Login Node Policies

The login nods are shared resources intended for editing and managing files, submitting and monitoring jobs, as well as compiling and linking. Use of scp, sftp, or rsync for large file transfers should be avoided. 

Details on acceptable use of Blue Waters login nodes and methods to enforce these policies, are discussed below. Accpetable useages are those which do not violate the terms of use agreement and do not impact the login experience for other Blue Waters users. Use of the Blue Waters login nodes that does not fit within the terms of use or impacts other users will be cause for review by the Blue Waters Project Office.

Long running processes

To regulate the use of the login nodes (h2ologin[1-3]) a process watcher has been implemented. The process watcher monitors processes running on the login nodes recording the CPU utilization per task per user per login node. 

The policy for long running user tasks are:

  1. Terminate user process over 100 hours of cputime which is our max user process hard limit.  (Note: as of the summer of 2016, we are considering sharply lowering the length of time processes are allowed to run.  There will be an announcement before the policy is changed on the user login nodes.)
  2. Terminate user process over 4 hours of cputime with an average cpu usage over 20%.
  3. For each process killed, email with details will be sent to the process owner.
  4. Exemptions by approved request via a support ticket help+bw@ncsa.illinois.edu.
    1. Current exemptions include most processes used in building code, like the compilers and things like "make".  We always want people to be able to build code on the logins.  If you're running a build and something gets killed, please submit a ticket and let us know.
  5. In December 2016, we have tightened limits on certain I/O-intensive processes that effect the user experience of other users but tend not to hit the CPU limits.  These "targetted" processes will be limited to 1 hour wall time regardless of level of CPU use.  Processes in this class generally include anything that can recursively walk the file system, which puts a lot of load on the meta-data server.  If you need to run any of these processes longer than a few minutes, please consider running them in a job with aprun. The list of processes under these restrictions is dynamic, but includes rsync, cp, scp, diff, du, and tar.

Rsync

The use of rsync can impact the experience of all users of a login node if used improperly.  Rsync is recommended for transfer of source code and small amounts of data between Blue Waters and other systems. Large data transfers should use the recommended method of Globus Online.

To regulate usage, user rsync processes utilizing Blue Waters login nodes [h2ologin1-3] as a source or destination are restricted to one instance per user per login node. An error message will be displayed with our policy details for rsync processes originating from Blue Waters login nodes. External rsync processes above our limit of one will be denied connection with an error message "protocol version mismatch -- is your shell clean? (see the rsync man page for an explanation) rsync error: protocol incompatibility (code 2) at compat.c(171) [sender=3.0.4]"
 

Crontab

The use of crontab tool is not allowed on Blue Waters. This policy is implemented to prevent the potential interference of the user processes with maintenance work, and to improve the transparency of processes running on the service nodes. Upon execution of crontab tool, a message will be printed "You (username) are not allowed to use this program (crontab)".

File access date changes to avoid purging

The use of frequent modification of the access time of files on the scratch file system for the sole purpose of avoiding file puringing is a violation of Blue Waters policy. If there are issues with job scheduling and file system purging then please contact us at help+bw@ncsa.illinois.edu for assistance on changes to workflows to enable file staging from Nearline.