NICE Desktop Cloud Visualization (DCV, https://www.nice-software.com/products/dcv) is a VNC server optimized for high-performance OpenGL applications such as the molecular visualization program VMD. DCV runs on a single XK node in cluster compatibility mode.
Client Software for Connecting to Blue Waters
You must download and install NICE DCV 2016.0 Endstation on your desktop or laptop to connect to a Blue Waters XK node with accelerated OpenGL performance. Earlier versions of NICE DCV Endstation appear to work with acceptable performance. Other VNC clients may also connect but OpenGL performance will be very poor. NICE DCV 2017.0 uses a different protocol that is incompatible with the DCV server software on Blue Waters.
Running DCV Endstation on MacOS
There is a problem with the DCV Endstation software that prevents it from launching on some versions of MacOS. If the DCV Endstation software quits immediately after launching, try removing the
Establishing Port Tunnels
To enable DCV Endstation to connect to your interactive session on a Blue Waters XK compute node you will need to ssh-tunnel ports 5901 and 7300-7310 (more or less, each OpenGL window will consume a port starting at 7300) through a Blue Waters login node. On a Linux or Mac client the easiest way to accomplish this is to create a script (named, e.g., bwforward) with the following (replace username with your Blue Waters login):
Launching an Interactive Visualization Job on Blue Waters
First, launch an interactive job on a single XK (GPU-accelerated) compute node:
If you require more memory than the 32GB on a regular XK node you may want to use a himem node, after using showbf to confirm that one is available and for how long:
When your interactive job starts (this may take ten minutes or more depending on the scheduler iteration time), you will be connected to a shell on a shared "mom" node, from which you must connect to your assigned compute node as follows:
After connecting to the login node you must launch the DCV server as follows:
The first time you launch DCV it will ask you to set a password.
The dcvlaunch script will print the internal (private) IP address of the compute node. You must open an ssh tunnel for ports 5901 and 7300... from your client desktop or laptop to the compute node through a login node. This can be done through the bwforward script given above.
A full transcript of launching a DCV session is shown below.
Leave the terminal connected, as killing it will terminate the interactive job and your dcv session.
In order to connect to this session from a client (i.e., a laptop or desktop) one would first establish the required ssh tunnels by running "
With the tunnels established you can then launch the NICE DCV Endstation software. In the dialogue, for "VNC Server" enter "
Exiting an Interactive Visualization Session
You may disconnect and reconnect to your visualization session as many times as you like. Exiting the window manager from within the viewer will kill the DCV server, but not your interactive job. The DCV server can be re-launched if it is killed. In order to kill the interactive job completely, you can use Control-D to exit first from the shell on the compute node, and then from the shell on the mom node. You can also run "
Running VMD for Interactive Visualization
VMD (Visual Molecular Dynamics, http://www.ks.uiuc.edu/Research/vmd) is a powerful molecular visualization and analysis package that relies on OpenGL for graphics acceleration.
Once you have connected to your DCV session, open a terminal and run
This will launch the pre-installed VMD optimized for OpenGL visualization. VMD will initialize a CUDA context on launch. Additional copies of VMD launched while the first is still running will report CUDA errors during startup but continue to run without CUDA acceleration (see explanation below). VMD CUDA initialization can be prevented by defining the
Limitations on CUDA Programs Due to Exclusive Mode
The GPUs on Blue Waters are set to the compute mode "Exclusive Process", which allows only one CUDA context per device, although that context can be accessed by multiple threads within the same process. As a result, if a process has accessed the GPU via CUDA any other process attempting to use CUDA will receive an error. This is different from the compute mode "Default" that is most commonly encountered, in which multiple processes can share the GPU freely, although with a loss of performance if their actual use overlaps.
Note that this restriction applies only to CUDA (and likely other GPU compute environments such as OpenACC and OpenCL). It specifically does not apply to OpenGL, so multiple OpenGL programs can run simultaneously (unless they also use CUDA).
In order to deal with this limitation, the