About Blue Waters
Blue Waters is one of the most powerful supercomputers in the world. It can complete more than 1 quadrillion calculations per second on a sustained basis and more than 13 times that at peak speed. The peak speed is almost 3 million times faster than the average laptop. Blue Waters is also the fastest supercomputer at a university anywhere in the world.
It is not a powerhouse only in terms of speed, though. The machine architecture balances processing speed with data storage, memory and communication within itself and to the outside world in order to cater to a wide variety of scientific endeavors. A project may simulate the evolution of the cosmos, delve into fine-scale processes in molecular dynamics, or anything in between. Many of the projects require a large portion of the thousands of processors that constitute Blue Waters and would be difficult or impossible to run elsewhere.
Blue Waters is supported by the National Science Foundation and the University of Illinois; the National Center for Supercomputing Applications manages the Blue Waters project and provides expertise to help scientists and engineers take full advantage of the system for their research. The system opened up to the science community at large on March 28, 2013.
Cray Inc. supplied the hardware: 22,640 Cray XE6 nodes and 4,224 Cray XK7 nodes that include NVIDIA graphics processor acceleration. The XE6 nodes boast 64 GB of memory per node and the XK7s have 32 GB of memory. Blue Waters' dedicated storage system includes 26 petabytes of useable online storage for quick access while jobs are running and 380 petabytes of usable nearline tape storage for longer term data storage. For complete system details, see the System Summary.
Compare this to a typical laptop, which has one processor—1/16 of an XE node or 1/8 of a XK node—with 4 GB of memory and half a terabyte of storage.
Speaking of storage, the Blue Waters 3 file systems (home, projects, and scratch) provide room for over 24 PB of data with a combined 1 TB/s read/write rate. The 3 filesystems are assembled around Cray's Sonexion Lustre appliances. The scratch file system is one of the largest and fastest file systems Cray has ever provided.
To backup or store data from the file systems for longer periods a nearline tape environment was built using Spectra Logic T-Finity tape libraries, a DDN disk cache and IBM's HPSS. This system provides over 300 PB of tape storage and a 1.2 PB disk cache.
The supercomputer lives in a 20,000-square-foot machine room at the National Petascale Computing Facility (NPCF) on the western edge of the University of Illinois campus in Champaign, Illinois. NPCF earned a gold certification from the U.S. Green Building Council's Leadership in Energy and Environmental Design (LEED) for its sustainable design.
Researchers interested in using Blue Waters can apply for an allocation through the NSF PRAC program. Members of the University of Illinois and the Great Lakes Consortium for Petascale Computing (GLCPC) may apply through their organizations, too. There are also pathways for educational and industry projects.
A Science and Engineering Team Advisory Committee (SETAC) brings together a diverse group of Blue Waters users to provide guidance on the project's growth and development. Representatives will make annual recommendations for positive change and improvement within the Blue Waters project.
Read the history of the Blue Waters project, from the awarding of the grant in 2007 to the system's launch into full operations.
Photos and video
If you would like to visit Blue Waters and learn more about NCSA, request a tour. Tours are scheduled based on staff availability Monday through Friday during standard working hours. It is best to contact us at least a week in advance to schedule your tour.