Blue Waters Innovation and Exploration Allocations

Overview

The National Geospatial-Intelligence Agency is funding the operations of Blue Waters to further the NGA mission to map and model the world. As a result, 90% of Blue Waters resources are available for projects relevant to NGA including:

~180 millions node hours (approximately 5.7 billion core hour equivalents)
23 Petabytes of usable, online storage
Staff to support development/porting/optimization of applications and workflows on the system

Blue Waters’ many cores provide a unique opportunity to perform groundbreaking work in computational and data science.

Projects of interest to NGA are geospatial in general, often focusing on mapping and modeling of the Earth. The production of Digital Elevation Models for the entire Earth is a priority, and account for the majority of these allocations. Projects should use the large-scale and highly-usable computing capabilities to innovative research and promote next-generation computationa-l and data-intensive applications and products.

Following the NGA Strategy 2025, (https://www.nga.mil/Documents/NGA_Strategy_2025.pdf) other topics of interest may include:

Foundational data - Digital Elevation Models either from stereoscopic or composited monoscopic satellite images, LIDAR and/or SAR processing, etc
Geodesy - Earth gravity models, Earth magnetic field models
Indications, assessment and warnings for earth surface use - e.g., crop yield prediction
Homeland defense and safety
Humanitarian and disaster relief
Human migration (due to economic, climate, etc)
Data analytics making use of foundational data (e.g., hydrologic mapping and modeling, supply routes, safety of navigation)
Efforts aligned with NGA strategic plans to move from just mapping to modeling initiatives (e.g., 2018 “MODELING THE WORLD” Workshop)
Machine learning/computer vision research related to the automated identification of features and classes of objects in still and video imagery such as from earth facing satellites.

Many areas of science that previously utilized large allocations on Blue Waters including computational biology and astrophysics are not in the scope for this allocation process.

Uniqueness of Blue Waters

Blue Waters is a powerful resource for computational and data science. Blue Waters is a Cray system with a combination of XE6 nodes (two AMD Interlagos CPU modules – four processor chips) and XK7 nodes (one AMD Interlagos CPU module and one NVIDIA Kepler K20X GPU), connected with Cray's Gemini interconnect, a high-performance network. A high-performance, high-capacity file system is also part of the system. A summary of the key capabilities is listed below:

Over 1.3 PetaFLOPS (10^15 floating-point operations per second) sustained performance (13.34 PetaFLOPS peak performance)
1.66 PetaBytes(PB) memory, 25PB Disk
4,228 XK7 GPU nodes with NVIDIA K20x GPUs
22,636 XE6 nodes with over 360,000 core modules of 720,000 integer cores
Over 9 Petaflops general purpose CPU performance and 4 PetaFLOPS peak GPU performance (typicallyb¼ PetaFLOPS sustained)
Over 1 TeraBytes/sec (10^12 bytes/second) sustained disk bandwidth
Over 400 Gbps external network performance

Additional details of the system design may be obtained from: https://bluewaters.ncsa.illinois.edu/hardware-summary

Eligibility

Only Principal Investigators who are faculty or senior staff at US research and development organizations, or who are employees or funded by US intelligence entities are eligible to submit an allocation proposal. The proposals can, however, include co-PIs and collaborators from other institutions.

Access to the Blue Waters system will be governed by the Blue Waters Terms of Use: https://bluewaters.ncsa.illinois.edu/terms-of-use.

Proposal Guidelines

A project proposal should contain the following:

Name(s) and contact information of PI and Co-PIs and if applicable, name(s) of collaborators and their institutions
Project Abstract which will be available on a public website
Project Overview
Project or Initiative Goals
Relevance to the NGA Mission and Goals
Description of Applications(s)
Support Requirements
Readiness, Usage Plans and Funding Source(s)
Resources Required
Requested Start Date and Duration

Explanations and guidelines are described in detail below.

Principal Investigator

The Principal Investigator must be a faculty or staff member of a US institution engaged in research and development and/or use of geospatial information. Multiple Co-PIs may be listed, and they may be postdocs, or faculty/staff members from the PI's home institution or from other institutions. Young investigators are encouraged to apply. Only one Principal Investigator should be listed for each proposal. The proposal should include the PI and Co-PIs name, title, department, university and contact information.

Project Abstract

A one-paragraph (about 150 words) project summary including why Blue Waters is necessary for this activity and how this activity furthers the NGA’s mission. This summary will be shared on the Blue Waters website.

Project Overview

Include a one-page overview of the project that describes the science/engineering problem to be solved and the computational approach, including challenges. Also describe the possible scientific impact to the specific field of science, the NGA mission. This overview will be shared on the Blue Waters website.

Project or Initiative Goals

A description of the specific research question(s) that the resources requested will be used to answer and the scientific and societal impact of the proposed work. Include an explanation of why a petascale resource of the capability that Blue Waters represents is necessary to address this research and development. This section should also clearly state how the proposed work maps to the mission of NGA.

Relevance to the NGA Mission and Goals

The proposal must explain how the planned work is relevant and supports the goals and mission of the National Geospatial Intelligence Agency, or its customers and stakeholders.

Description of Applications(s)

Describe the structure of the application codes that you intend to use. These may either currently exist, might require enhancement, or they may be in development. Include descriptions of any novel computational or data driven approaches. Please include details about the algorithms involved and the approach that you intend to use to ensure that the code scales effectively on the Blue Waters architecture. Describe how your code(s) will exploit the major system elements: the memory hierarchy, the communications network, the computational elements, CPU and GPU nodes, scalability and the I/O subsystem. Identify which system element(s) is/are likely to be the main bottlenecks and how the design of your application minimizes the impact of these bottlenecks. Describe how you intend to analyze the output resulting from your use of Blue Waters. IMPORTANT: Please describe any run-time libraries or special system software or program development environment features that you will require and the types of graphics support that you would find most useful.

Description of Support Requirements

Provide a description for any special support requirements and an explanation of why this support is needed. This could include requests for

Support for software that is not in the Blue Waters supported software category
Development and programming support
Training and educational support
Support for special handling of data (export control, limited distribution, etc.)
Requirements for special hardware
Support for on-premises (your site) services and visits
Project management support for your project
Public Affairs support
Visualization consultation and services
Specialized data science tools and frameworks
Special Data transport support and software

Readiness, Usage Plans and Funding Source(s)

Describe the current state of readiness of the application codes that you intend to use and your plans for developing these to the point where they are ready to run on the Blue Waters system. Evidence of suitability for running on Blue Waters may include data on the efficiency of the code and analysis and/or demonstration of scalability.

Provide an estimated Blue Waters usage schedule. The estimate should be per quarter and may be represented as a percent of the requested allocation (e.g., Q1: 10%, Q2: 20%, Q3: 50%, Q4: 20%). Preference will be given to projects that will use their time in earlier quarters.

Resources Required

Describe the Blue Waters resources required to complete research and development on the Target Problem. This description should include the number of system nodes needed for your runs, the anticipated actual memory usage, the expected numbers of each major class of arithmetic and logical operation, the expected numbers of local and remote memory accesses, the total number of node-hours required, the anticipated input and output requirements, the amount of data that you anticipate transferring to or from the Blue Waters enclave, the amount and type of storage required and any other system resource needs that you anticipate. For assistance in computing node hours for Blue Waters see: https://bluewaters.ncsa.illinois.edu/node_core_comparison

A typical proposal will be for 30,000 to several million node hours and will be 3-5 pages in length and will provide sufficient detail to justify the merits of the project are for periods of 3-12 months.

For a description of the default storage quotas see: https://bluewaters.ncsa.illinois.edu/storage

If your project will require storage limits that exceed the standard quotas, provide a justification in support of your request.

Requested Start Date and Duration

Usage of allocation is expected to begin immediately upon award and complete within the allocated allocation period. Projects should time their request to align with their other resources so the usage of Blue Waters can begin upon allocation award.

Related Information

Review Criteria

Projects will be judged on their scientific merit, their suitability for Blue Waters; and the degree to which the proposed work furthers the mission of NGA and innovation. Reviews will be conducted by a steering committee consisting of representatives of the Blue Waters Project, the University of Minnesota Polar Geospatial Center and the NGA.

NCSA Staff Assistance

Blue Waters information is available to guide potential proposals. System and programming environment information is available at the Blue Waters website, training material from past workshops, as well as complete documentation about using Blue Waters is available at the Blue Waters website. See Training: https://bluewaters.ncsa.illinois.edu/training or Documentation: https://bluewaters.ncsa.illinois.edu/documentation. Blue Waters support staff are available during the workday to provide limited assistance for proposal writers such as answering specific questions, discussing algorithms and method approaches, scoping problems, providing general performance metrics, etc. Contacting the Blue Waters Support by submitting a service request via email to help+bw@ncsa.illinois.edu or calling 217- 244-6689 between 9 a.m. and 5 p.m. Monday through Friday.

Reporting Requirements

Reports are required for all awards under this program. The report should include accomplishments achieved with research and development conducted using Blue Waters — including publications, data products shared, software developed, grants submitted, grants obtained, and talks presented, as well as details of any issues or problems encountered with the use of Blue Waters. Instructions for the final report are available at: https://bluewaters.ncsa.illinois.edu/reports. PIs will be responsible for submitting a final report and also a status update mid-allocation. Projects are encouraged to contact Blue Waters support about issues or problems as they occur rather than using these reports to bring issues to the attention of the Blue Waters project.

Acknowledgement Line

The text found on Acknowledging Support from Blue Waters or something similar must be included in publications of all kinds that report on work performed and/or share the data using an allocation on Blue Waters made under this program.

Questions? Contact the Blue Waters Project Office at help+bw@ncsa.illinois.edu.

Approved for public release – 20-035.