The UofC EXAS Simulation System is composed of two types of computer resources and each addresses different data processing challenges.  The first is a two node cluster of IBM POWER 6 570 Servers.  Each p570 system consists of 32, 4.2GHz, tightly coupled SMP CPU cores with 256GB local RAM.  These systems are designed to run very large, complex models in the million of X. The machines are used individually and as the model sizes grow and the code continues to develop, they will be clustered to enable larger, more complex simulations.  The IBM P570 systems are unique in the following ways:

  • SMP design with 32 cores at 4.2Ghz each.  This allows for a dramatic increase in single thread application performance where the application has not been optimized for MPI.
  • Uniform, non-NUMA memory architecture.
  • Distributed on-chip, interchip switch fabric.
  • A 128-bit vector execution unit called Altivec SIMD.
  • Advanced memory subsystem enabling 32GB/s throughput.
  • Data integrity features such as Instruction Retry
  • Decimal Floating Point Support increasing numeric accuracy and performance
  • Large L3 cache (16MB) per chip
  • Advanced Hardware Virtualization
  • Symmetrical Multi Threading (SMT) allows each CPU core to split its many execution units into two logical CPU’s.  This allows each system to be viewed as having 64 CPU to the simulation.
  • GX+ Bus slots providing additional external expansion for up to 32 I/O drawers and 128PCI e/x cards per server

The second system is a cluster of 28 IBM POWER 6 blades. In addition to the large model simulators described above, there is the requirement for a large volume of smaller simulations and associated code development.  The blade systems fulfill this requirement while providing the opportunity to be easily clustered together as the code is optimized for MPI operation.  Each blade Server includes:

  • The same POWER 6 CPU architecture as the 570 systems
  • Four very fast 4.0 GHz POWER 6 cores and 16GB of RAM.  Each CPU supports Symmetrical Muti Threading (SMT) as well for up to 8 logical CPU’s per blade.
  • Small blade based infrastructure that is compatible with other IBM blades in use by the department.
  • In particular both these systems utilize the POWER 6 processor, the only in-order 64bit RISC processor with Altivec extensions capable of operating at greater than 4GHz, providing both the parallel and serial core execution performance required for the investigative work Dr. Chen is undertaking.

These two systems ensure Dr. Chen’s research will evolve in lock step with Industry as both single, large SMP MPI compute and clustered OpenMP programming and analysis models are being addressed.  These two systems reflect the current and future direction of complex reservoir simulators.  They are essentially newer versions of the same systems used by Industry and Dr. Chen’s partners which improves the transferability of the research results.