CM1 is a three-dimensional, time-dependent, non-hydrostatic numerical model that has been developed primarily by George Bryan at The Pennsylvania State University (PSU) (circa 2000-2002) and at the National Center for Atmospheric Research (NCAR) (2003-present). CM1 is designed primarily for idealized research, particularly for deep precipitating convection (i.e., thunderstorms). CM1 is freely available to anybody that wants to use it, though restrictions on distribution apply. See CM1 home page http://www2.mmm.ucar.edu/people/bryan/cm1/ for additional information.
These instructions were tested with CM1 version 18 and 20.1. They may work with other versions as well.
B. How to download CM1
Please follow the instructions on the CM1 website where you have to agree to the CM1 license http://www2.mmm.ucar.edu/people/bryan/cm1/getcode.html.
Once you have downloaded the source tarball, do:
tar xf cm1r18.tar.gz
tar xf cm1r20.1.tar.gz
C. How to build CM1
The CM1 website describes building it in detail and should be taken as authorative, this page for the most part summarizes what is described there.
On Cray XE6 platform, CM1 can be built under the Cray environments as described in the Makefile. You can enable both NetCDF or HDF5 output by uncommenting the OUTPUTOPT option in the respective section of the Makefile in src/Makefie. Uncomment the compiler options in the Blue Waters section (search for bluewaters) and remove -lfast_mv from LINKOPTS since the libfast module is no longer supported. For version 18 you will also have to fix param.F in subroutine param, where the write directive is used incorrectly. For version 20.1 you will have to allow up to 1023 columns per line and reduce optimization level for domaindiag.F to avoid compiler failures.
module load PrgEnv-cray module load cray-netcdf cd cm1r*/src # fix up param.F for version 18 by removing a "," after write sed -i 's/write(outfile,[*]),/write(outfile,*) /' param.F # reduce optimization level for domaindiag.F (\t is a tab that make requires) echo -e >>Makefile ' domaindiag.o: domaindiag.F \t$(CPP) $(DM) $(DP) $(ADV) $(OUTPUTOPT) $*.F > $*.f90 \t$(FC) $(FFLAGS) $(OUTPUTINC) -O2 -Ovector0 -c $*.f90 hifrq.o: hifrq.F \t$(CPP) $(DM) $(DP) $(ADV) $(OUTPUTOPT) $*.F > $*.f90 \t$(FC) $(FFLAGS) $(OUTPUTINC) -O1 -c $*.f90 '
# this sets the options on the command line rather than edit the Makefile # these options are for a pure MPI build (no OpenMP) # There seem to be some missing dependency declarations in the Makefile # leading to make aborting if build in parallel. You can use higher values # but might have to restart make if an error is detected. make FC=ftn OPTS="-I../include -O3 -Ovector3 -Oscalar3 -Othread3 -h noomp -N 1023" \ LINKOPTS="" CPP="cpp -C -P -traditional" DM=-DMPI -j1
Successful compilation will create a cm1.exe binary file in the run directory, one level up from src.
The compilation was conducted on 2020-09-28 under the following environment:
Currently Loaded Modulefiles: 1) modules/22.214.171.124 17) PrgEnv-cray/5.2.82 2) eswrap/1.3.3-1.020200.1280.0 18) cray-mpich/7.7.4 3) cce/8.7.7 19) craype-interlagos 4) craype-network-gemini 20) torque/6.0.4 5) craype/2.5.16 21) moab/9.1.2-sles11 6) cray-libsci/18.12.1 22) openssh/7.5p1 7) udreg/2.3.2-1.0502.10518.2.17.gem 23) xalt/0.7.6.local 8) ugni/6.0-1.0502.10863.8.28.gem 24) scripts 9) pmi/5.0.14 25) OpenSSL/1.0.2m 10) dmapp/7.0.1-1.0502.11080.8.74.gem 26) cURL/7.59.0 11) gni-headers/4.0-1.0502.10859.7.8.gem 27) git/2.17.0 12) xpmem/0.1-2.0502.64982.5.3.gem 28) wget/1.19.4 13) dvs/2.5_0.9.0-1.0502.2188.1.113.gem 29) user-paths 14) alps/5.2.4-2.0502.9774.31.12.gem 30) gnuplot/5.0.5 15) rca/1.0.0-2.0502.60530.1.63.gem 31) darshan/3.1.3 16) atp/2.0.4 32) cray-netcdf/126.96.36.199
D. Sample test
CM1 provides some sample namelist.input files on its website (r18) or in the tarball (r20.1). To run the supercell case do
cd .. # now in main directory cp -r run ~/scratch/supercell rm ~/scratch/supercell/namelist.input # for r18 wget -O ~/scratch/supercell/namelist.input \ http://www2.mmm.ucar.edu/people/bryan/cm1/namelists/namelist.input.supercell.r18 # for 20.1 cp run/config_files/supercell/namelist.input ~/scratch/supercell/ cd ~/scratch/supercell
To reduce runtime to about 5 minutes, you should reduce the timax setting in namelist.input to 480 to stop after integrating 8 minutes of time.
sed -i 's/timax[^,]*,/timax = 480.,/' namelist.input
Then create a run.pbs file having the following content:
#!/bin/bash #PBS -l nodes=1:ppn=32:xe #PBS -l walltime=00:05:00 #PBS -q debug #PBS -N supercell cd $PBS_O_WORKDIR aprun ./cm1.exe >cm1.out 2>cm1.err
Submit the job
which will produce a file cm1.out which will contain lines like this
stattim = 60. 0 0.000000 min Entering writeout ... nwrite = 1 nloop = 1 Opening cm1out_s.dat 51, 1 rain 51, 2 prate 51, 3 sws 51, 4 svs 51, 5 sps
E. Known issues
For some (high resolution input), the model can hang without giving an error message, until its time in the queue runs out. To our knowledge, it only occurs when one has chosen to use the Morrison microphysics routine.
Here’s what the output file looked like when that was happening:
-------------------- nwrite = 1 Test_10m_0_000001_s.dat Test_10m_0_000001_u.dat Test_10m_0_000001_v.dat Test_10m_0_000001_w.dat 2d vars s vars -----------------
It turns out that for the Morrison scheme in CM1, there is a check for numerical convergence in the saturation adjustment routine, and if that is not met, then it prints out this error message and stops the model:
print * print *,' Convergence cannot be reached in satadj2 subroutine.' print * print *,' This may be a problem with the algorithm in satadj2.' print *,' However, the model may have became unstable somewhere' print *,' else and the symptoms first appeared here.' print * print *,' Try decreasing the timestep (dtl and/or nsound).' print * print *,' ... stopping cm1 ... ' print *
For some reason, at some resolutions this bit of code doesn't get executed, and things just hang, "spinning" down time in the queue. To fix the problem (not the code), one must just increase his/her value of the variable 'nsound' in the namelist.input file. If the user has already maxed that out (I think a value up to 12 is allowed), then they must decrease the main timestep ('dtl') in namelist.input.