Comparing CAF and MPI-3 and Studying Fast Reconnection for Relativistic Two-Fluid Electrodynamics

Dinshaw Balsara, University of Notre Dame

Usage Details

As an approximation for plasmas, magnetohydrodynamics (MHD) has served us well. Even so, there are systems where the MHD approximation breaks down. In particular, if the plasma is relativistic, the fluid motions can be very rapid and the displacement current can change on timescales that are comparable to the fluid motions (i.e., the separation of timescales that was so useful in giving rise to the MHD approximation is no longer valid). Similarly, if the positive and negatively charged particles are made up of positrons and electrons, then the skin depth can be comparable to the scale of the system being considered. In such situations, the electrons and positrons have to be treated as two separate fluids which interact with each other and also with the entire system of Maxwell's equations. The RIEMANN code has recently been extended to provide such a capability (Balsara et al. 2015). While such conditions are extreme by terrestrial or solar system standards, such conditions routinely occur in the magnetospheres of pulsars and black holes (Lyutikov, Balsara and Mathews 2012).

This is a small request so it includes Balsara and his group. Balsara's postdoc, Sudip Garain, will run the detailed simulations. Dan Nagle and his postdoc, Alessandro Fanfarillo, are extending the capabilities of the GNU compiler to include Cray's Coarray Fortran (CAF). This team represents a very valuable group that will help drive the maturation of crucial technologies on NCSA's Blue Waters.

The core two-fluid relativistic electrodynamics algorithms in our RIEMANN code are based on higher order Godunov schemes including schemes that go well beyond the traditional second order. These algorithms are used in conjunction with divergence-free AMR-MHD. Multidimensional Riemann solvers are also used.

We have NSF funding (NSF-ACI-1307369; NSF-DMS-1361197) to compare CAF standard—which is available on Blue Waters—with MPI for a range of PDE applications. We report on early work that we have done in this regard using Blue Waters (Garain, Balsara & Reid 2015). Further work is also needed to show scalability of adaptive applications, which is one of the goals of this renewal proposal. At this point, we have developed two versions of AMR-MHD code. One version works with and the other with the newly-released MPI-3 standard. We wish to compare the two frontline capabilities. We also have new NSF funding (NSF-ACI-1533850) that builds on previous grants. The goal of this funding is to develop advanced compilers and fault-tolerant applications that are ready for exascale computing. While such an exascale machine does not exist, our plan for this year is to get a public domain GNU compiler for CAF incorporated on Blue Waters and compare its performance vis a vis Cray's compiler.