Login Node Policies
The login nods are shared resources intended for editing and managing files, submitting and monitoring jobs, as well as compiling and linking. Use of scp, sftp, or rsync for large file transfers should be avoided.
Long running processes
To regulate the use of the login nodes (h2ologin[1-3]) a process watcher has been implemented. The process watcher monitors processes running on the login nodes recording the CPU utilization per task per user per login node.
The policy for long running user tasks are:
- Terminate user process over 100 hours of cputime which is our max user process hard limit. (Note: as of the summer of 2016, we are considering sharply lowering the length of time processes are allowed to run. There will be an announcement before the policy is changed on the user login nodes.)
- Terminate user process over 4 hours of cputime with an average cpu usage over 20%.
- For each process killed, email with details will be sent to the process owner.
- Exemptions by approved request via a support ticket email@example.com.
- Current exemptions include most processes used in building code, like the compilers and things like "make". We always want people to be able to build code on the logins. If you're running a build and something gets killed, please submit a ticket and let us know.
- In December 2016, we have tightened limits on certain I/O-intensive processes that effect the user experience of other users but tend not to hit the CPU limits. These "targetted" processes will be limited to 1 hour wall time regardless of level of CPU use. Processes in this class generally include anything that can recursively walk the file system, which puts a lot of load on the meta-data server. If you need to run any of these processes longer than a few minutes, please consider running them in a job with aprun. The list of processes under these restrictions is dynamic, but includes rsync, cp, scp, diff, du, and tar.
The use of rsync can impact the experience of all users of a login node if used improperly. Rsync is recommended for transfer of source code and small amounts of data between Blue Waters and other systems. Large data transfers should use the recommended method of Globus Online.
To regulate usage, user rsync processes utilizing Blue Waters login nodes [h2ologin1-3] as a source or destination are restricted to one instance per user per login node. An error message will be displayed with our policy details for rsync processes originating from Blue Waters login nodes. External rsync processes above our limit of one will be denied connection with an error message "protocol version mismatch -- is your shell clean? (see the rsync man page for an explanation) rsync error: protocol incompatibility (code 2) at compat.c(171) [sender=3.0.4]"
The use of crontab tool is not allowed on Blue Waters. This policy is implemented to prevent the potential interference of the user processes with maintenance work, and to improve the transparency of processes running on the service nodes. Upon execution of crontab tool, a message will be printed "You (username) are not allowed to use this program (crontab)".
File access date changes to avoid purging
The use of frequent modification of the access time of files on the scratch file system for the sole purpose of avoiding file puringing is a violation of Blue Waters policy. If there are issues with job scheduling and file system purging then please contact us at firstname.lastname@example.org for assistance on changes to workflows to enable file staging from Nearline.