Blue Waters Partners:
Through December 19, 2019:
| Charge factors for completed jobs that meet one or more of the following criteria below are discounted by 25% for each of the following opportunities with a resulting maximum discount of 69% when compounded. Please note that job accounting will not immediately reflect the discounts and there may be a several day delay before the charging is adjusted. This will improve as automatic accounting operations are created. |
As of December 20, 2019 all charge factor discounts are discontinued.
As of March 28th, 2017 the wall clock accuracy and backfill discounts are not in place. The preemptee discount has been merged with the minwclimit (flexible wall clock limit) discount.
1. Job wall clock accuracy of 75%
A discount is applied to jobs that have wall clock accuracy of at least 75%. Wall clock accuracy is defined as actual elapsed wall clock time divided by requested/maximum wall clock time. (exceptions exist for discounts 3 & 4 below)
2. Job backfills available nodes
A discount is applied to jobs that run on backfill available nodes that are draining for other jobs. Backfill happens most often for jobs requesting several hundred nodes or less and wall clock times less than 6 hours. Backfill opportunities vary based on system load and are not guaranteed and the potential exists that any job could run on non-backfill nodes without the discount. Learn more about backfill on the portal page (https://bluewaters.ncsa.illinois.edu/why-isn-t-my-job-running#backfill).
3. Job Preemptibility OR Flexible wall clock time
Jobs marked as preemptible or specify flexible wall clock time will receive a charging discount. Job preemption is possible in normal and low queues. Even if marked preemptible, jobs are shielded from preemption for the initial 1 hour of runtime or the runtime requested if less than 1 hour is requested. To be eligible for this discount, jobs must utilize at least 1 hour wall clock time and be sized at or less than 1,024 XE nodes or 512 XK nodes. Preemption decisions are based on relative job priority. Preemption flags may be used in conjunction with flexible wall clock. Any job that gets preempted will automatically receive the wall clock accuracy discount as well.
The PBS job script syntax for preemption specification is
#PBS -l flags=preemptee
To be eligible for the flexible wall clock discount, jobs must utilize at least 4 hours wall clock time. Once the minimum wall clock time is reached, the job will become preemptible by higher priority jobs. We still recommend continued use of flexible wall clock option (minwclimit) for workloads that can make progress with a portion of the total requested walltime, as it usually results in an earlier start time, distinguishing it from use of the preemptee flag by itself.
The PBS job script syntax to enable flexible wall clock time in a job is:
#PBS -l minwclimit=[mintime] -l walltime=[maxtime]
Job interruptions due to discount 3 above will result in job cancellation. Future enhancements, pending resolution of some bugs, include automatic requeueing options as well as SIGTERM signals some time ahead of the SIGKILL to permit an opportunity for a final checkpoint.
Once the minimum wall clock time is reached, the job will become preemptible by higher priority jobs. We still recommend continued use of flexible wall clock option (minwclimit) for workloads that can make progress with a portion of the total requested walltime, as it usually results in an earlier start time, distinguishing it from use of the preemptee flag by itself.
Please send questions to help+bw@ncsa.illinois.edu.
