Blue Waters history
On Aug. 8, 2007, the National Science Foundation (NSF) approved a $208 million grant to the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign for a new petascale supercomputing project. NCSA dubbed the project and the supercomputer "Blue Waters", a name inspired by Illinois' orange and blue school colors, by the role of water in cooling the system, and by the support provided to the overall project by the Great Lakes Consortium for Petascale Computation.
NCSA initially proposed to work with IBM to build the Blue Waters supercomputer using the company's POWER7 hardware.
The plan for a building to house the new supercomputer was announced in October 2008. The following month, construction started on the University of Illinois-funded National Petascale Computing Facility on the western edge of the campus in Champaign, Ill. The LEED Gold facility was finished ahead of schedule and under budget in April 2010. More than 1,000 people visited the facility during its first open house in June 2010, and many more continue to take tours today.
In the meantime, NSF announced the first Petascale Computing Resource Allocations. These awards help research teams collaborate with NCSA staff to prepare their codes to fully and effectively utilize the massive Blue Waters petascale compute and data system. NSF also pledged an additional $775,000 to educate scientists and engineers about petascale computing and help them prepare to use Blue Waters.
On Aug. 6, 2011, IBM terminated its contract with the University of Illinois to supply the hardware for Blue Waters, indicating that the project had grown more complex and required more financial and technical support than IBM had anticipated. IBM returned all project funds and NCSA returned all IBM equipment.
NCSA quickly acted to assess other hardware options and to review proposals from various vendors. A little more than three months after IBM's withdrawal, the university signed a new contract with Cray, Inc. The new configuration called for a combination of XE6 compute nodes and XK6 (later XK7) GPU-accelerated nodes.
Cray delivered the first cabinet within a few weeks, and by March 2012 a 48-cabinet Early Science System (15 percent of the full Blue Waters system) was available to multiple research teams.
By November 2012 the full Blue Waters system was installed and operational: 237 racks of XE6 CPU nodes, 32 racks of XK7 GPU-accelerated nodes, and more than 26 petabytes of fast online storage. Select teams were granted access to the full system in "friendly-user" mode.
By January 2013 four large-scale science applications in the Blue Waters Petascale Performance Suite had achieved more than 1 petaflop per second or better sustained computing performance. Another application reached 2.2 petaflops per second sustained.
The Blue Waters team decided not to enter the Top500, a list of the 500 supercomputers with the fastest peak computing speed that is compiled twice each year. The Top500 assesses performance using a benchmark called Linpack that accounts for only a single characteristic of a supercomputer and tests only one type of application. The benchmark does not represent the computing needs of the scientific community. Instead of chasing bragging rights, Blue Waters aims to be a highly productive resource for breakthrough science and engineering work.
"Blue Waters truly is the most data-focused, data-intensive system available to the U.S. science and engineering community," says Blue Waters Project Director Bill Kramer.
On March 28, 2013, NCSA, the University of Illinois, and NSF celebrated the launch of Blue Waters into operations making the system available for 24/7 science.
In August of 2013 an additional 12 racks of XK7 compute nodes were added completing the final 288 Blue Waters configuration (244 racks of XE6 compute and service nodes, 44 racks of XK7 compute nodes).
On December 19, 2019 Blue Waters ended its role as the NFS track 1 supercomputer after 6.75 years of production. During that production time frame Science teams utilized Blue Waters to produce many groundbreaking research successes documented elsewhere on the portal. One of the most significant successes was the creation of digital elevation maps of the Arctic and Antarctic regions with partners at the Polar Center at the University of Minnesota, the Ohio State University and the National Geospatial-Intelligence agency. This partnership has resulted in the continuing operation Blue Waters for the primary purpose of producing digital elevation maps under funding provided by the NGA. This work was highlighted in a speech by Vice Admiral Robert Sharp, Director of the NGA.
On March 31st, 2020 the Blue Waters nearline subsystem was retired after a 90 day grace period to allow all teams the opportunity to transfer data stored on nearline or Blue Waters file systems to other resources.