PETSc

PETSc , the Portable, Extensible Toolkit for Scientific computation, is provided as part of the Cray Programming Environment software stack. It provides sets of tools for the parallel (as well as serial), numerical solution of PDEs that require solving large-scale, nonlinear and sparse linear systems of equations. Cray provides an optimized version of the standard PETSc Library. This package is highly tuned for Cray hardware via Cray's custom Cray Adaptive Sparse Kernels (CASK).

CASK is an auto-tuned and adaptive framework for the optimization of Sparse Matrix Vector operations. CASK uses a custom code generator and the result of extensive offline research to select the best optimization method according to the problem characteristics. This adaptive approach is vital in the optimization of sparse linear solvers since the performance of numeric kernels is highly dependent on sparsity characteristics of the calling problem. CASK provides CSR (Compressed Sparse Row format) and FBSR (Fixed Block Sparse Row) tuned kernels that can improve PETSc performance by 5% to 30%. The exact performance advantage from CASK depends on the matrix class of the calling problem.

If you need to build a more recent version of PETSC or build a modified version of PETSC please see our Build your own PETSC page.

How to use PETSc

% module load cray-petsc

This will load the default version of PetSc. to load a specific version available on the system, use

% module avail cray-petsc

then load the specific version, for example,

% module load cray-petsc/<version>

You can now us cc, CC and ftn to compile and link without needing to specify paths for header files or libraries.

Note: For dynamically linked binaries the cray-petsc libraries cause the linker to link to specific cray-hdf5 or cray-hdf5-parallel libraries independant of what cray-hdf5 module is loaded due to specified libraries in the .so. The current defaults for cray-petsc and cray-hdf5-parallel should link fine. If using version 3.7.2  of PETSc then you should also load cray-hdf5 or cray-hdf5-parallel version 1.10.0. Indications of this issue would be from runtime errors like:

PETSC ERROR: Error in HDF5 call 

Additional Information

More information can be found in PETSc and the Cray Application Developer's Environment User's Guide. Always refer to docs.cray.com for the version available on your system or the latest version of the document.