2016 Blue Waters Symposium: Abstracts and Presentations

Improved Molecular Dynamics Model Suggests a Novel Mechanism for Epigenetic Control of Chromatin Compaction

Project PI: Aleksei Aksimentiev, University of Illinois at Urbana-Champaign

Presented by: Jejoong Yoo, University of Illinois at Urbana-Champaign

Abstract: Mounting experimental evidences suggest that the physical structure of chromosomes controls gene transcription. Regions of the genome containing predominantly AT basepairs are commonly found forming compact structures, making their genetic information inaccessible to the gene transcription machinery. Conversely, regions of the genome containing predominantly GC basepair are loosely folded and accessible. Interestingly, the GC-rich parts of the genome can be conditionally inactivated by hypermethylation of DNA. Despite decades-long research efforts, the microscopic mechanism underlying such gene activation or deactivation control has remained unknown. Using Blue Waters, we have determined that both AT content and DNA hypermethylation strengthen direct DNA-DNA association mediated by polyamines. Single-molecule fluorescence experiments performed in the laboratory of Dr. Ha (University of Illinois) confirmed the predictions of our MD simulations. The results of our study suggest a novel physical mechanism of gene regulation in eukaryotes, with possible implication to cell differentiation and cancer.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Quantum-classical Path Integral Simulation of Electron Transfer in a Bacterial Photosynthetic Reaction Center

Project PI: Thomas Allen, University of Illinois at Urbana-Champaign

Abstract: The conversion of solar energy to ATP lies at the heart of life on Earth, and autotrophic organisms have evolved highly efficient light harvesting complexes which human engineers have often tried to imitate. However, a full theoretical understanding of the electron transfer involved in photosynthesis requires both a quantum mechanical description of the transferring electron as well as a detailed atomistic picture of its environment. For the first time, we report on a completely rigorous, atomically detailed quantum-classical simulation of electron transfer dynamics in the reaction center of Blastochloris viridis. We are performing these simulations using an efficient, highly-parallelized implementation of the quantum-classical path integral method recently developed in our group, and from this work we can obtain not only information about the electron transfer itself, but also data which provide the first direct examination of the validity of the linear response approximation in a biomolecular electron transfer reaction.

Field of Science: Chemistry

Watch the presentation | View presentation slides (PDF)

Molybdenum Disulfide (MoS₂) as a Novel 2D Nano-porous Membrane for Water Desalination

Project PI: Narayana Aluru, University of Illinois at Urbana-Champaign

Presented by: Mohammad Heiranian, University of Illinois at Urbana-Champaign

Abstract: We demonstrate molybdenum disulfide (MoS₂) as a novel 2D nano-porous membrane for water desalination. We performed extensive molecular dynamics (MD) simulations which involve up to 60,000 nodes hours on the Blue Waters (BWs). The MD packages (LAMMPS & NAMD) we used scale well with the number of cores on BWs. We find that a single-layer MoS₂ nanopore can effectively reject ions and allow transport of water at very high rates. More than 80% of ions are rejected by membranes having pore areas ranging from 20 to 60 Ų. Water flux through the membranes is found to be several orders of magnitude greater than that of other known nano-membrane technologies. In another project, using BWs, we find that a nanopore in a DNA origami sheet, mounted on top of a 2D membrane, is a promising material for DNA base detection. The nanopore retards DNA translocation speed resulting in high residence times.

Field of Science: Materials Science

Watch the presentation | View presentation slides (PDF)

Petascale Quantum Simulations of Nano Systems and Biomolecules

Project PI: Jerzy Bernholc, NC State University

Abstract: Our simulations use the Real Space Multigrid (RMG) electronic structure codes, of which we are the primary developers. RMG employs real-space grids, a multigrid pre-conditioner, and subspace diagonalization to solve the Kohn-Sham equations. It runs at petaflops speeds on massively parallel supercomputers and scales effectively to 200,000 cores and 20,000 GPUs. CPU and GPU binaries for Blue Waters can be downloaded from bluewaters.ncsa.illinois.edu/rmg, while the source codes and binaries for Linux, Windows and MacIntosh systems reside at sourceforge.net/projects/rmgdft/. We will describe the newest enhancements to RMG, which will be released in version 2.0, including improved small-problem performance without impacting large-scale performance, extended support of exchange-correlation functionals, and much reduced memory footprint. We will also discuss the latest applications, such as quantum transport (NEGF) calculations for prototypical devices containing ~10,000 atoms and nanotube/polymerase nano-circuits that can monitor DNA replication in real-time and potentially enable electrical readout of DNA sequences.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Custom Genotyping Chip for African Populations

Project PI: Gerrit Botha, University of Cape Town

Abstract: This computational project aimed to produce genomic variant calls for the design of a cost-effective genotyping chip that would capture the genetic diversity in populations of African origin, including African-Americans.

This work will enable the identification of genetic variation specific to African populations, which will help better understand the links between genotype and disease in people of African origin, and thus extend the principles of personalized medicine to these underserved populations. It will also permit deeper study of African genetic diversity, which will bring important insights into the history and evolution of humans in general.

We also demonstrated, in a production-grade project, the capability of Blue Waters to conduct high-throughput analysis of human genomes. Lots of benchmarking data were collected, and the computational workflow was hardened with many quality control steps to ensure delivery of correct results. The code is posted on GitHub, to be shared with the community.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Modeling a Mobile Ecosystem: Eddies and Sargassum in the North Atlantic

Project PI: Maureen Brooks, University of Maryland Center for Environmental Science

Abstract: The pelagic macroalgae Sargassum fluitans and Sargassum natans are keystone species in the Atlantic Ocean and Gulf of Mexico, where they serve as habitat and forage for a diverse floating ecosystem. Recent Sargassum wash-ups along the coastline in the Caribbean highlight the need for improved understanding of this organism. The seasonal distribution of Sargassum throughout the tropics and subtropics established via satellite and ship-board observations is at odds with a basin-wide circulation that tends to aggregate buoyant material in the central North Atlantic Subtropical Gyre. Mesoscale features such as eddies can reconcile these differences by physically dispersing Sargassum and altering local growing conditions through nutrient upwelling. Using Blue Waters, a series of four coupled models including ocean circulation, biogeochemistry, Lagrangian trajectories, and Sargassum physiology, were applied to examine the effects of eddies on Sargassum growth and dispersal.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Evolutionary Dynamics of the Protein Structure-Function Relation in Metabolic Networks

Project PI: Gustavo Caetano-Anolles, University of Illinois at Urbana-Champaign

Presented by: Fizza Mughal, University of Illinois at Urbana-Champaign

Abstract: Protein dynamics manifests in protein function by virtue of the flexibility offered by unstructured regions, the protein loops. Previous studies have shown that these loop regions, while not completely rigid, are not completely disordered and may possess a set of distinct motions that are characteristic of the molecule. We argue that biophysical variables may be key to studying this preference of motions during the course of evolution. In our current Blue Waters allocation, we have completed a first round of molecular dynamic simulation studies of loops regions of aminoacyl-tRNA synthetase (aaRS) enzymes. We are exploring if protein motions related to protein functions of these enzymes are at the core of the genetic code's origin. We also aim to analyze metabolic enzymes. We have initiated analysis of metaconsensus enzymes derived from the overlap of three prominent comparative bioinformatic approached that focus on sequence, structure and metabolic reactions.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Improving Checkpoint-Restart with Lossy Compression

Project PI: Jon Calhoun, University of Illinois at Urbana-Champaign

Abstract: I/O is starting to dominate execution time and is becoming a limit to scalability for numerous HPC applications. The mean time between failures on future extreme scale machines is expected to decrease which will add more I/O pressure with reading and writing of checkpoints. Compression techniques can be employed to reduce data size before I/O operations. Compared to traditional lossless compression which yields only minor reductions in dataset size, lossy compression is able to deliver a 5-50x reduction in data set size, but at the expense of adding error into the data set. In this talk, we investigate the use of lossy compression when restarting from lossy compressed checkpoints in the applications PlascomCM and Nek5000. We highlight how simulation properties such as truncation error can help guide the selection of a compression error tolerance.

Field of Science: Computer Science

Watch the presentation | View presentation slides (PDF)

Cellulosome Structure Determination: Combining Atomistic Simulations and Experimental Assays

Project PI: Isaac Cann, University of Illinois at Urbana-Champaign

Presented by: Rafael Bernardi, University of Illinois at Urbana-Champaign

Abstract: Adopting the strategy that some bacteria employ, which takes advantage of synergistic enzymatic action in large complexes, namely cellulosomes, offers a promising approach to reduce advanced-biofuel production cost. The importance of cellulolytic enzymes for the production of renewable fuels and chemicals from biomass has highlighted an urgent need for improved fundamental understanding of how cellulosomal networks achieve their impressive catalytic activity. Our group aims, employing Molecular Dynamics techniques combined with biochemical experiments, to study the detailed mechanism of cellulase complexes, in particular, cellulosomes. Using stochastic search algorithms connected to molecular dynamics tools implemented in Blue Waters, we recently built the first comprehensive structure of a cellulosome. We expect that a complete model of cellulosome's structure will shed light on the mechanism that allow these enzymatic complexes to be highly efficient. A full understanding of cellulosomes mechanism is key for the development of purpose-designed cellulosomes for biofuel industry.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Hydrogen under Extreme Conditions

Project PI: David Ceperley, University of Illinois at Urbana-Champaign

Abstract: Hydrogen accounts for much of the visible mass in the universe. The properties of hydrogen and helium are important for understanding the giant planets, Jupiter and Saturn, but experiments under the relevant conditions are challenging. Even though hydrogen is the first element in the periodic table, calculating its properties is not simple since both the electronic and protonic correlations are quantum and correlated. It has long been an open question how hydrogen makes a transition from a molecular insulating state to an atomic metallic state. We have developed new Quantum Monte Carlo simulation methods to treat such systems and using them, have studied molecular dissociation in liquid hydrogen and have observed clear evidence of an extra liquid-liquid phase transition. During the past year, two experiments have reported observations of the transition we predicted, however, the observations do not agree with each other, differing in pressure by a factor of two. This motivated us to repeat our earlier calculations on Blue Waters, to better control the convergence and utilize recent improvements in methodology.

We use a quantum Monte Carlo method (Coupled Electron Ion Monte Carlo) where we start with the true interaction between the electrons and protons and treat both fully quantum mechanically. In contrast to density functional calculations, all effects of electronic correlation are explicitly included. This is particularly important in hydrogen, because of possible self-interaction effects, difficulty in treating the hydrogen bond breaking and the large van der Waals interactions. We model hydrogen with about 100 electrons and protons in a periodic cell. Special methods are used to extrapolate to the thermodynamic limit. With our method, we simulated hydrogen for temperatures in the range of 200K up to 5000K, and at relevant pressures, 100GPa to 500GPa.

For temperatures below 2000K we observe a first order transition between an insulating molecular liquid and a more dense metallic atomic liquid. Our predicted transition pressure are intermediate between the two experimental observations. Future work will be to perform further simulations and analysis to understand the divergent results of the experiments and of the unusual properties of the molecular and atomic liquid. It is essential for progress in the high pressure community to resolve the difference between the experiments and computation. After validation, the method can be used with more confidence in modeling the wide variety of astrophysical objects being observed, composed largely of hydrogen and helium under extreme conditions.

Field of Science: Physics

Watch the presentation | View presentation slides (PDF)

Ensembles of MD Simulations to Assess, Validate, Improve, and Enable Force Fields for Exploring RNA Structure and Dynamics

Project PI: Thomas Cheatham, University of Utah

Abstract: By coupling together efficient and highly optimized molecular dynamics simulations on GPUs, we are very rapidly able to converge the conformational structural and dynamic ensemble of various RNA motifs. This allows critical evaluation of the available molecular mechanical force fields and highlights key limitations that we have been able to exploit to improve RNA force fields. With force fields that have been validated and assessed in reproducing accurate structural and dynamic distributions of RNA, we are then able to use the improved methods to give better insight into RNA structure, dynamics, and ultimately function. Even without perfect force fields, the applied methods in combination with biases from experiment (such as NMR derived NOE, residual dipolar coupling and other restraints), we are able to significantly and much more rapidly characterize RNA structure and dynamics.

Field of Science: Chemistry

Watch the presentation | View presentation slides (PDF)

Computational Methods Allow the Connection of Microscopic Models to Emergent Phenomena

Project PI: Bryan Clark, University of Illinois at Urbana-Champaign

Abstract: In this talk we report on simulations of two emergent phenomena arising in quantum many-body systems. The first phenomena, many-body localization, happens in interacting disordered systems and results in the breakdown of statistical mechanics and the unprecedented emergence of quantum effects at infinite temperature. In the second phenomena we report on the numerical discovery of a simple microscopic model which results in a strange metal phase, a phase which regularly shows up in the phase diagram of superconducting materials. In both cases, we show how the computational power of Blue Waters combined with sophisticated algorithms to compensate for the naive exponential scaling in simulating quantum mechanics has been critical in elucidating these phenomena.

Field of Science: Physics

Watch the presentation | View presentation slides (PDF)

Innovative ab initio Symmetry-adapted No-core Shell Model for Advancing Fundamental Physics and Astrophysics

Project PI: Jerry Draayer, Louisiana State Univerisity

Presented by: Robert Baker, Louisiana State University

Abstract: A standing goal in nuclear physics is to determine nuclear structure and reaction information from first principles (or ab initio), that is, informed by quantum chromodynamics considerations. Coupled with the computational capabilities of Blue Waters, our recently developed ab initio symmetry-adapted no-core shell model (SA-NCSM) can reach nuclear regions that were previously inaccessible. We have achieved this through an innovative concept that utilizes a physically relevant basis and a highly scalable hybrid MPI/OpenMP implementation. Only possible with Blue Waters, we have begun using the SA-NCSM to investigate nuclei from oxygen through titanium, including isotopes of interest to next-generation experimental facilities and whose properties will have significant impact on astrophysical models of nucleosynthesis and probing physics beyond the Standard Model. I will also report on recent progress toward leveraging GPUs and Intel MIC coprocessors to improve performance of the methods for calculating structure and reaction observables in the SA-NCSM framework.

Field of Science: Physics

Watch the presentation | View presentation slides (PDF)

First Galaxies and Quasars in the BlueTides Simulation

Project PI: Tiziana Di Matteo, Carnegie Mellon University

Presented by: Yu Feng, University of California Berkeley

Abstract: We will report the recent progress on the BlueTides simulation (Project: jp6). BlueTides is a hydrodynamical cosmology simulation that utilizes the full capability of the Blue Waters supercomputer at NCSA. The simulation combines state of art physical models and advanced parallel algorithms to make the first numerical predictions for the entire population of galaxies and supermassive black holes (and the intergalactic medium) that existed when the Universe was less than a billion years old.

This talk consists of four parts: 1) I will review the important technical aspects of the simulation code MP-Gadget; 2) I will give a briefing of recent science results from 6 submitted/published papers; 3) I will report on progress made investigating the performance of the massively parallel I/O module (bigfile) that is used in the BlueTides simulation via the BW-PAID program [with Markus Scheucher]; 4) I will also discuss the status of our recent efforts to improve the scaling of the hydrodynamics module.

Field of Science: Astronomy & Astrophysics

Watch the presentation | View presentation slides (PDF)

Solvation Thermodynamics of Oligoglycine with Respect to Chain Length and Flexibility: Implications for Aggregation and Collapse

Project PI: Justin Drake, University of Texas Medical Branch

Abstract: Oligoglycine is a backbone mimic for all proteins and is prevalent in intrinsically disordered proteins/regions. We computed a thermodynamic decomposition of the solvation free energy (ΔG_^sol) of Gly2-5 into enthalpic (ΔH_^sol) and entropic (ΔS_^sol) components as well as their van der Waals (vdw) and electrostatic (elec) contributions as a function of chain length, conformational flexibility, and force field (CHARMM36 and Amber ff12SB). For both rigid-extended and flexible oligoglycine models, the decrease in ΔG_^sol with chain length is enthalpically driven with only weak entropic compensation. However, the apparent rates of decrease of ΔG_^sol, ΔH_^sol, ΔS_^sol, and their elec and vdw components differ for the rigid and flexible models. Thus we find solvation entropy does not drive aggregation for this system and may not explain the collapse of long oligoglycines. Additionally, both force fields yield very similar thermodynamic scaling relationships despite both force fields generating different conformational ensembles of oligoglycine chains.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Dispersion of Finite Size Droplets and Solid Particles in Isotropic Turbulence

Project PI: Said Elghobashi, University of California, Irvine

Presented by: Michele Rosso, University of California, Irvine

Abstract: The paper presents a comparison between the dispersion characteristics of finite size liquid droplets and finite size solid particles in isotropic turbulence at moderate values of Rλ. The droplets and particles have equal diameters (larger than the Kolmogorov length scale) and equal densities. The immersed boundary method is used for direct numerical simulations (DNS) of the solid particles. The level set method is used for DNS of the droplets where a variable-density projection method is used to impose the incompressibility constraint.

We discuss the effects of varying the surface tension (Weber number) of the liquid droplets on their dispersion and acceleration characteristics.

Field of Science: Fluid Systems

Watch the presentation | View presentation slides (PDF)

QMCDB: A Living Database to Accelerate Worldwide Development and Usage of Quantum Monte Carlo Methods

Project PI: Elif Ertekin, University of Illinois at Urbana-Champaign

Abstract: Quantum Monte Carlo (QMC) methods are a suite of tools for direct stochastic solution of the many-body Schrodinger Equation. Although QMC methods are one of the highest-accuracy materials modeling methods available, their usage for materials design and discovery has historically been limited by large computational cost. With the capabilities of Blue Waters, however, it is now possible to extend the method to this realm. The goal of our work is to develop the first database of materials based on quantum Monte Carlo results, QMCDB. Blue Waters is critical to carrying out the comprehensive QMC calculations that populate our database. QMC methods exhibit near-linear scaling on the entire platform, which has allowed us to calculate properties of an extensive materials set including semiconductors, transition metal oxides, and others. We expect that this database will serve as a shared community resource to accelerate the use of this high-accuracy method.

Field of Science: Materials Science

Watch the presentation | View presentation slides (PDF)

Large-scale Computational Mapping of Protein-DNA Binding Affinity Landscapes

Project PI: Peter Freddolino, University of Michigan

Abstract: Transcription factors and other DNA-binding proteins shape the behavior of all cells, coordinating gene expression patterns in response to internal or external cues. Our ability to understand and manipulate cellular behavior thus relies on our understanding of their transcriptional regulatory networks, including the locations and occupancies of transcription factor binding sites. Those binding sites, in turn, are dictated by the binding affinity of each transcription factor for different DNA sequences. These binding affinity landscapes can at present be determined only through costly and laborious experiments. We are applying the massive computing resources provided by Blue Waters to map the binding affinity landscapes of several human transcription factors, using rigorous, fully atomistic molecular dynamics simulations on the interactions of those transcription factors with varied DNA sequences. The results of these calculations are providing us with unprecedented insight into the biophysics underlying transcription factor specificity, which we will subsequently use to enable more efficient computational estimations of transcription factor binding affinity landscapes that could routinely supplement or replace experimental results.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Large Eddy Simulation of Sediment Transport and Hydrodynamics at River Bifurcations using a Highly Scalable Spectral Element-based CFD Solver

Project PI: Marcelo Garcia, University of Illinois at Urbana-Champaign

Presented by: Som Dutta, University of Illinois at Urbana-Champaign

Abstract: Bifurcations are fundamental features of all river systems. The current study focuses on a specific class of bifurcations called diversions. Experiments have shown that the distribution of near-bed sediment between the downstream channels at a diversion is not proportional to the water flow distribution, with a disproportionately higher amount of sediment going into the lateral channel. A better understanding of the aforementioned non-linear phenomena will help in efficient design of river diversions, which among uses like navigation and flood-mitigation have also been put forth as a solution for reclaiming deltas going under the sea due to increasing sea level. The current study investigates the mechanisms behind this phenomenon through Large Eddy Simulations (LES) and Direct Numerical Simulations (DNS) of the flow in idealized diversions of different configurations, with the sediment being modeled as Lagrangian particles. The simulations have been conducted using a highly scalable spectral element based incompressible Navier-Stokes solver, Nek5000. The simulation conditions are comparable to the laboratory experiments, which make these simulations one of the largest and most complex to date in the area of river mechanics. Blue Waters provides the computing resources to undertake these highly-resolved simulations. Currently, simulations have been conducted for up to 243.648 million computational points, with strong scaling being shown up to 32,768 MPI ranks.

Field of Science: Engineering

Watch the presentation | View presentation slides (PDF)

Sequence Similarity Networks for the Protein Universe

Project PI: John Gerlt, University of Illinois at Urbana-Champaign

Presented by: Ken Yokoyama, University of Illinois Urbana-Champaign

Abstract: Our project is enabling generation of a library of precomputed sequence similarity networks for all Pfam families in the UniProtKB database for dissemination to the scientific community. At present, the Pfam database contains 16,295 families (Release 2.0; December 2015). To enable construction of the library, we are using Blue Waters to calculate 1) all-by all BLAST sequence relationships, 2) statistical analyses of the BLAST results; and 3) merged sets of input sequences based on sequence identity. We are working toward updating the library with an eight-week refresh cycle so that the library will remain current with the InterPro databases as new sequences are released. We will provide an update on our progress to achieve these goals.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

LES Study of Suction Side Film Cooling at Different Free Stream Turbulence Levels

Project PI: Chaitanya Ghodke, GE Global Research

Abstract: In this study Large Eddy Simulations of adiabatic film cooling effectiveness are carried out on a row of shaped holes placed on the suction side of a transonic vane. The film cooling consists of 5 shaped holes located upstream of the passage throat. The pitch and spanwise boundaries of the vane passage are modeled using periodicity. Different levels of turbulence intensity affecting the film cooling are achieved by placing a grid of cylindrical bars at different axial positions upstream of the vane. The LES simulations are carried out using two solvers. A commercial solver that uses a finite volume second-order accurate spatial discretization and a high order flux reconstruction/correction procedure via reconstruction (FR/CPR) unstructured grid solver. The impact of free stream turbulence on the film mixing in the main gas path is analyzed and comparisons are made between second order and high order LES analysis.

Field of Science: Engineering

Dissipative Particle Dynamics Simulations of Star Polymer Microdroplets

Project PI: Sharon Glotzer, University of Michigan

Presented by: Ryan Marson, University of Michigan

Abstract: Star polymer droplets can be used used as scaffolds for tissue regeneration. Experiments of these systems demonstrate a variety of hollow, non-hollow, and porous structures. We perform dissipative particle dynamics simulations (DPD) of 10 million particles using HOOMD-blue on Blue Waters XK7 GPU nodes. Our simulations map the phase diagram of droplet morphologies as a function of hydroxyl density and star polymer architecture (arm length and number) and offers predictive power to direct future experiments. This system demonstrates the possibility of functional droplets composed of complex networks, with a hierarchy of scales that can be tuned for specific applications.

Field of Science: Engineering

Watch the presentation | View presentation slides (PDF)

Nuclear-Electronic Orbital Calculations on Molecular Systems

Project PI: Sharon Hammes-Schiffer, University of Illinois at Urbana-Champaign

Presented by: Kurt Brorsen, University of Illinois Urbana-Champaign

Abstract: The nuclear-electronic orbital (NEO) method treats electrons and select nuclei quantum mechanically on the same level using an orbital-based formalism with the goal of obtaining a computationally tractable method that includes non-Born-Oppenheimer effects as well as nuclear quantum effects. The NEO method is ideal for studying chemical phenomena such as proton-coupled electron transfer (PCET) because the timescale for proton tunneling is often faster than the timescale for electronic transitions, thereby leading to a breakdown of the Born-Oppenheimer approximation. In applications of the NEO method to PCET, all electrons and one or a few protons are treated quantum mechanically, and a mixed nuclear-electronic time-independent Schrödinger equation is solved using explicitly correlated wavefunctions. Recent advances to the NEO method involving wavefunction and density functional theory will be discussed and benchmarking calculations on small molecules for basis sets will be presented.

Field of Science: Chemistry

Watch the presentation | View presentation slides (PDF)

Policy Responses to Climate Change in a Dynamic Stochastic Economy

Project PI: Lars Hansen, University of Chicago

Presented by: Yongyang Cai, University of Chicago

Abstract: We extend our Integrated Assessment Model framework, called DSICE (Dynamic Stochastic Integration of Climate and the Economy), for evaluating alternative policy responses to future climate change under both economic and climate uncertainty. We incorporate five interacting climate tipping points into DSICE, and find that the present social cost of carbon (SCC) increases nearly eightfold. Moreover, passing some tipping points increases the likelihood of other tipping points occurring, so that the SCC increases abruptly. The optimal mitigation policy requires zero industrial emission after this midcentury and leads to a path of global average temperature less than 1.5 degree Celsius. We also examine the impact of Bayesian learning of uncertain critical parameters (e.g., climate sensitivity) to decision rules. Furthermore, we address the problem of decision making with uncertainty in the context of climate change that affects the world in a non-uniform manner (more warming in the polar regions).

Field of Science: Social Sciences

View presentation slides (PDF)

Computational Approach to Designing Antibody for Ebola Virus

Project PI: Eric Jakobsson, University of Illinois at Urbana-Champaign

Abstract: Our work on Blue Waters successfully demonstrated the feasibility of computational design of synthetic antibodies against evolving Ebola infections. We simulated multiple cycles of viral mutation and redesign of a synthetic antibody to successfully counter the mutations and restore high affinity binding of the antibody to the virus. Viral mutations were selected by random walk biased according to statistical propensity for amino acid substitution. Trial substitutions for redesign were selected according to statistical propensity for forming favorable interfaces. Success of redesign was evaluated by using molecular dynamics to compute viral protein-antibody binding energy. Blue Waters provided the essential compute power to do the many simulations essential to test ability to successfully redesign. The approach should be extendable to other viruses. In combination with experimental sequencing and structure determination, our approach should enable rapid design and redesign of synthetic antibody therapy in response to rapidly evolving viral challenges.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

The Power of Many: Runtime Environment for Executing Dynamic Heterogeneous Workloads

Project PI: Shantenu Jha, Rutgers University

Abstract: High-performance computing systems such as Blue Waters have been optimized to support mostly monolithic and single-job workloads. Many important scientific applications however, have workloads comprised of multiple heterogeneous tasks that are not known in advance and temporally vary in the resources required. The default runtime environment is often unable to meet the requirements of such workloads. We will discuss how RADICAL-Pilot has been designed to serve as a runtime system that meets the requirements of heterogeneous and dynamic workloads on Blue Waters. We will discuss how it is engineered to spawn up to 100 tasks/second and maintain a steady state of up to 8,000 concurrent tasks. We discuss the architecture of the RADICAL-Pilot agent and how RADICAL-Pilot is interfaced with the ORTE layer to achieve necessary performance. We will conclude with a discussion of several scientific questions under way using RADICAL-Pilot on Blue Waters.

Field of Science: Computer Science

Watch the presentation | View presentation slides (PDF)

High Accuracy 3D Radiative Transfer in Cloudy Atmospheres

Project PI: Alexandra Jones, Princeton University/NOAA-GFDL

Abstract: One of the most important roles clouds play in the atmosphere is in redistributing radiative energy from the sun and that is emitted from the Earth and atmosphere. However, radiative transfer in the atmospheric sciences is generally modeled crudely because of the perceived computational expense. A community 3D broadband Monte Carlo Radiative Transfer model (MCBRaT-3D) has been developed for massively parallel use on Blue Waters to produce simulations at higher accuracy than ever before that can be used as standards of comparison for cruder models more frequently employed. The unique large, high fidelity databases of liquid water droplet and gas radiative properties needed to run high accurate simulations are products that can be utilized by the radiative transfer community and mined to evaluate and create new parameterizations for simpler radiative transfer models. Challenges in scaling this model to large problem sizes include finding the right balance of memory footprint, load, and communication latency per processor.

Field of Science: Atmospheric & Climate Science

Watch the presentation | View presentation slides (PDF)

Physics-based Strong Ground Motion Simulations using Blue Waters

Project PI: Thomas Jordan, University of Southern California

Presented by: Christine Goulet, SCEC, University of Southern California

Abstract: Important societal infrastructure remains vulnerable to large magnitude earthquakes. Over the past two decades, a multidisciplinary group of earth scientists, engineers, and computer scientists affiliated with the Southern California Earthquake Center (SCEC) have used high-performance computing to advance our knowledge of earthquake science. We have used Blue Waters to perform state-of-the-art physics-based earthquake simulations to provide researchers and practitioners with information that can help significantly reduce uncertainty in seismic hazard assessments. This past year, Blue Waters enabled SCEC teams to simulate deterministic ground motions up to 8 Hz while introducing new physics required for more realistic ground motions (i.e., rough-fault geometrical complexity, frequency-dependent attenuation modeling, nonlinear effects, small-scale material heterogeneities, and surface topography), and increasing computational performance (through our CPU-GPU hybrid, parallel I/O, and workflow management applications). We have also improved the underlying seismic velocity models through tomographic inversions and used these models to validate simulations against past earthquake recordings.

Field of Science: Geophysics

Watch the presentation | View presentation slides (PDF)

Pixel-level Uncertainty Quantification, Optimized for Blue Waters

Project PI: Athol Kemball, University of Illinois at Urbana-Champaign

Field of Science: Astronomy & Astrophysics

Sensing the Environment: A Glimpse into the Microscopic Mechanisms of Pain

Project PI: Michael Klein, Temple University

Presented by: Vincenzo Carnevale, Institute for Computational Molecular Science, Temple University

Abstract: TRP channels are central to environmental sensation in animals, fungi, and unicellular eukaryotes. All known TRP channels are nonselective cation channels that open in response to wide array of factors. Clarifying how TRP channels convert physical and chemical stimuli from the environment into the allosteric signals underlying channel activation is key to understand how they control cell excitability in both physiological and pathological conditions. Their relevance in the molecular pathways mediating pain makes them promising targets of novel classes of analgesics. Building on the structural information made recently available for TRPV1 thanks to a series of cryo-microsopy experiments, we performed free energy (metadynamics) simulations on models of TRPV1 embedded in a lipid bilayer. Harnessing the computation capabilities of Blue Waters, we explored several pathways of activation and characterized ion channel conductance and selectivity. Our calculations reveal a novel mechanism for sensing temperature and osmolarity.

Field of Science: Biosciences

Nanoelectronics Modeling on Blue Waters with NEMO5

Project PI: Gerhard Klimeck, Purdue University

Presented by: Jim Fonseca, Purdue University

Abstract: Semiconductor device size has already reached countable number of atoms in critical dimensions and materials and designs become more dependent on atomic details. NEMO5 is a nanoelectronics modeling package designed for comprehending quantum effects such as tunneling, state quantization, and atomic disorder, which begin to dominate characteristics of these nano-scale devices. Recent research powered by Blue Waters includes 1) Contributions to the International Technology Roadmap for Semiconductors' Process Integration, Devices, and Structures Chapter for ultra-thin body and nanowire performance projections 2) Universal behavior of strain in quantum dots based on shape and aspect ratios and 3) Assessment of literature scattering approximations and their effect on the current-voltage performance of silicon tunnel-FET circular nanowires.

Field of Science: Materials Science

Watch the presentation | View presentation slides (PDF)

High-resolution Simulations of Cumulus Entrainment

Project PI: Sonia Lasher-Trapp, University of Illinois at Urbana-Champaign

Abstract: Deep cumulus clouds produce the majority of the earth's precipitation, but accurate predictions of their initial development remain elusive. Of particular importance is the proper quantification of entrainment, the process by which clouds bring dry air from outside the cloud inward that may ultimately lead to their early demise. A long-standing problem in meteorology has been to reproduce how quickly entrainment dilutes a cumulus cloud. Currently, all models fail. It has been assumed to be a problem of inadequate spatial resolution, but could also result from a deficiency in conceptual models of entrainment. Simulations of small cumuli at ultra-high resolution are being conducted on Blue Waters and compared to aircraft observations. We have determined the simulations require 10-20 m resolution or higher, which over 3D domain lengths of 10 km consist of 1 billion grid points, in order to represent entrainment accurately.

Field of Science: Atmospheric & Climate Science

Core-collapse Supernovae through Cosmic Time

Project PI: Eric Lentz, University of Tennessee

Abstract: Core-collapse supernovae (CCSNe) are an important link in the chain of stellar evolution and critically contribute to the chemical enrichment of the Universe.

Numerical modeling of CCSNe remains a challenge due to the inherent multidimensional and multi-physics nature of the problem and the diversity of initial conditions; yet, recent progress has resulted 3D simulations with detailed physics at reasonable resolutions.

We present recent results from 3D simulations that examine resolution and progenitor diversity computed with our detailed "Chimera" code.

A low-mass, zero-metal star with low density envelope explodes easily with behaviors contrasting those in more massive stars that require longer for neutrino-driven convection to drive an explosion.

Analysis of the explosion hydrodynamics and direct signals from neutrinos and gravitational waves will enhance our understanding of the CCSN mechanism and its variations, while analysis of the ejecta composition will advance our understanding of chemical evolution.

Field of Science: Astronomy & Astrophysics

Simulating 4-D Subduction and Mantle Flow beneath the Americas using Data Assimilation Models

Project PI: Lijun Liu, University of Illinois at Urbana-Champaign

Abstract: Our team has been using Blue Waters to simulate the temporal evolution of subduction and mantle convection underneath both South and North Americas. For South America, we attempt to address the question on what controls the formation of flat-slab subduction, a process the down-going oceanic plate moves sub-horizontally beneath the continent before plunging into the mantle. For North American, we focus on testing different hypotheses for the origin of the enigmatic Yellowstone Volcanic Province.

So far, we have made good progress on both projects. For South America, we find that the flat slabs were affected by several physical mechanisms with the most important one being the subduction of buoyant oceanic plateaus. For Yellowstone, we find that the traditionally believed Yellowstone mantle plume actually plays a minor role. Instead, it is the progressively intruding hot upper mantle from the oceanic side that ultimately formed the western U.S. volcanic activities.

Field of Science: Geophysics

Watch the presentation | View presentation slides (PDF)

Simulating Ribosome Biogenesis in Replicating Whole Cells

Project PI: Zaida Luthey-Schulten, University of Illinois at Urbana-Champaign

Presented by: Tyler Earnest, University of Illinois at Urbana-Champaign

Abstract: Central to all life is the assembly of the ribosome: a process involving the association of ~50 proteins to 3 RNA molecules in a hierarchical, coordinated manner. Through the synthesis of data from disparate sources such as in vitro kinetics, cryo-electron tomography, single particle diffusion assays, mRNA expression, and genomics data, we have developed a spatially-resolved stochastic model of the biogenesis of the ribosomal small subunit in Escherichia coli. Using our GPU-accelerated Lattice Microbes (LM) software on Blue Waters, we have observed biogenesis in modeled cells at timescales up to 2 hours. Our model reproduces the correct assembly times and predicts the spatial distribution of assembly intermediates. We then extend this model to include cell cycle-dependent effects including growth, DNA replication, and cell division. Blue Waters provided the necessary computing power to complete this research as well as directed the development of LM to maximize performance.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Instrumenting Human Variant Calling Workflow on Blue Waters

Project PI: Liudmila Mainzer, University of Illinois at Urbana-Champaign

Abstract: High throughput Human Variant Calling Workflow on Blue Waters. If whole genome sequencing and analysis become part of the standard of care in many hospitals within the next few years, then human genetic variant calling will need to be performed on hundreds of incoming patients on any given day. At this scale, the standard workflow widely accepted in the research and medical community, will use thousands of nodes at a time and have i/o bottlenecks that could affect performance even on a major cluster like Blue Waters. In this presentation, we will discuss the kinds of computational bottlenecks that can be expected, as well as the tools and methods to overcome them. Specifically, we will cover the bottlenecks associated with the large number of small files created by the workflow, saturated i/o bandwidth for parts of the workflow, and unbalanced data load on the file system.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Quantum-classical Path Integral Simulations of Ferrocene-Ferrocenium Charge Transfer in Solution

Project PI: Nancy Makri, University of Illinois at Urbana-Champaign

Presented by: Peter Walters, University of Illinois at Urbana-Champaign

Abstract: Condensed phase electron transfer reactions play a vital role in most biological and synthetic energy transfer pathways. The cost of quantum mechanical calculations scale exponentially with system size; thus, performing highly accurate simulations of realistic condensed phase reactions is notoriously demanding. Quantum-classical path integral (QCPI) is a recently developed highly parallelizable methodology designed to efficiently and accurately simulate the dynamics of a quantum system immersed in a condensed phase environment. It combines a classical treatment of the environment with a path integral representation of the system. The local nature of the quantum paths allows for the system-solvent interaction to be treated exactly, free of approximation. Using Blue Waters' resources, we utilized QCPI to simulate the charge transfer process of the ferrocene-ferrocenium pair in solution with unprecedented accuracy.

Field of Science: Chemistry

Watch the presentation | View presentation slides (PDF)

Advanced Computational Methods and Non-Newtonian Blood Models for Blood-artery Interactions in Patient Specific Geometries

Project PI: Arif Masud, University of Illinois at Urbana-Champaign

Abstract: We have developed non-Newtonian models for blood that account for its shear-rate response, and have embedded them in a new developed hierarchical multiscale finite element method. The proposed method possesses local and global (coarse- and fine) description of the variational formulation that results in a telescopic depth in mathematical representation of spatial and temporal scales. This scale split leads to two coupled nonlinear systems, namely, the coarse-scale and the fine-scale sub-systems. Fine-scale models that are extracted from the residual-driven fine-scale sub-problems are then variationally embedded in the coarse-scale description of the problem and it leads to the class of Hierarchical Multiscale methods for non-Newtonian fluids with enhanced stabilization properties.

From computational and algorithmic perspective the new method leads to substantially reduced global communications in favor of increased local computing, a feature which is of tremendous benefit in massively parallel computing. With five percent increase in the cost of computation of the stiffness matrices and the residual force vectors, we were able to reduce the mesh size to less than half the nodes that would otherwise be needed for an equivalent mathematical as well as engineering accuracy. This unique aspect of the method and its link with the computing platform are highlighted.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Large-eddy Simulations of Aero-optic Distortions

Project PI: Edwin Mathews, University of Notre Dame

Abstract: Using Blue Waters, large-eddy simulations are performed to study aero-optic distortions, or the distortion of light by compressible turbulent flow, with a goal to further understand the connection between flow physics and optical distortions. In the past year, research has progressed on two fronts. First, the relationship between turbulent density fluctuations and optical distortions has been explored using high-fidelity simulations of weakly compressible mixing layers. Optical results from the simulation are compared with theoretical expressions for the spectral behavior of phase distortions, demonstrating a direct relationship to the spectral behavior of density fluctuations. Secondly, investigation of the database generated from the simulation of a realistic optical turret has improved our knowledge about the effects of turbulent wake structure, viewing angle, and aperture size on optical distortions found in a common beam transmission platform.

Field of Science: Fluid Systems

Watch the presentation | View presentation slides (PDF)

Experiments and Large-Scale Simulations of the Triboelectric Charging of Volcanic Ash

Project PI: Joshua Mendez, Georgia Institute of Technology

Abstract: Volcanic plumes, like many other turbulent and collisional granular systems in nature, have the tendency to become electrostatically charged [Hatakeyama, 1943; Hatakeyama and Uchikawa, 1951; Hatakeyama, 1958; Anderson et al.1965; Lane and Gilbert, 1992; Miura et al. 1996, 2002; James et al.1998, 2008; McNutt and Davis, 2000; Mather and Harrison, 2006; Aizawa et al. 2010; Harrison et al. 2010; McNutt and Williams, 2010; Nicora et al. 2013]. Perhaps the most dramatic and evident consequence of this electrification are the impressive lightning displays often observed during vigorous eruptions. While volcanic lightning has been reported for millennia [Letters of Pliny, 2001 translation], the physics that generate and separate charge in plumes still require clarification. Investigated mechanisms for electrification include fractocharging or charging resulting from the break-up of magma in the conduit [James et al, 2000], triboelectric charging arising from particle-particle collisions [Forward et al. 2009c; Lacks and Sankaran, 2011], inductive charging of dry materials [Pähtz et al. 2010], and charging mechanisms similar to those found in thunderstorms related to the interaction of hydrometeors [Herzog et al.1998; McNutt and Williams, 2004; Mansell et al. 2005].

The observed variability in electrical behavior between eruptions and between volcanoes suggests that the generation of charge is modulated by specific eruption parameters such as fragmentation behavior, the properties of the ejected materials (both ash and volatiles), environmental conditions, as well as the dynamics of the plume itself. Therefore, understanding the coupling between eruption parameters and electrical activity, much of which can be studied remotely, may yield information about the internal dynamics of an eruption which would otherwise be opaque to observation [James et al.1998]. Indeed, Behnke and Bruning [2015] have recently shown that lightning (and by association, charging processes) could be used to infer changes in eruption kinematics.

Here, we present the results of a set experiments and numerical simulations specifically designed to link the dynamics of a flow of ash to its electrification behavior. Based on the devices described by Forward et al. [2009a] and Bilici et al. [2014], we constructed an spouted-bed instrument capable of characterizing the triboelectric charging of ash due to solely particle-particle collisions. The apparatus is capable of measuring the transient electrification behavior of the fountain as well as measure the absolute charges on individual grains in the flow with a resolution of 1 fC. While the systems works well to quantify electrostatic behavior, extracting fountain dynamics, specifically collisions rates and energies, remains difficult. To address these shortcomings, the internal fountain kinematics were characterized using a large-scale, full 3-D discrete element model (DEM) using 10 million particles run on the NCSA Blue Waters supercomputer. Our results provide a microphysical framework to interpret recent observations of lightning and field changes at active volcanoes such as Redoubt and Sakurajima.

Field of Science: Geophysics

Watch the presentation | View presentation slides (PDF)

Petascale Particle-in-Cell Simulation of Kinetic Effects in High Energy Density Plasmas

Project PI: Warren Mori, UCLA

Presented by: Frank Tsung, UCLA

Abstract: The UCLA Simulation Group has been on the frontier of high performance kinetic simulation of plasmas since its early days with its founder John Dawson. Over the subsequent 40+ years, the group has developed a full suite of codes (with various models for fields and particles) to study kinetic effects in plasmas for a variety of applications, including plasma-based accelerators, laser fusion, and basic plasma physics. The computational resources at Blue Waters have been instrumental in allowing our group to perform large-scale simulations which have allowed us to study higher dimensional effects in inertial fusion plasmas, and to make quantitative comparisons between simulations and experiments in plasma-based accelerator.

In 2015, simulations performed on Blue Waters have been published in many high impact journals, including Nature. These simulation results and our code development efforts, including the latest timing results on GPU's and the Intel Xeon Phi processors, will be presented.

Field of Science: Physics

Watch the presentation | View presentation slides (PDF)

Predicting the Transient Signals from Galactic Centers: Circumbinary Disks and Tidal Disruptions around Black Holes

Project PI: Scott Noble, The University of Tulsa

Abstract: Active supermassive black holes (SMBHs) at the centers of galaxies are home to the most energetic phenomena in the universe and are systems of great interest for high-cadence sky surveys such as the LSST. Insight into their transient nature is contingent on the community developing detailed simulations of electromagnetic emission from SMBHs and/or SMBH binaries (SMBHBs), which is our goal. We will report on three sets of simulations: SMBHB accretion dynamics, tidal disruption events (TDEs), and misaligned or "warped" accretion disks. Each set of runs advances the field's existing resolution and sophistication limits. We will report how the growth of periodic electromagnetic signals depends on the strength of magnetic field stresses in SMBHB simulations. We will show how the tidal truncation radius of mini-disks depends on BHB separation in the post-Newtonian limit. We will report on our first multi-patch calculations of stellar disruptions which will pave the way for the first ever MHD evolutions of a TDE. Lastly, we will share our latest misaligned accretion disk simulations that is revolutionizing our understanding of how accretion disks warp when they are tilted relative to the spin or orbital angular momentum of the central BH or BHs.

Field of Science: Astronomy & Astrophysics

Watch the presentation | View presentation slides (PDF)

Realistic Simulations of the Intergalactic Medium: The Search for Missing Physics

Project PI: Michael Norman, San Diego Supercomputer Center, UCSD

Abstract: Recent, more precise observations of the intergalactic medium (IGM)—the hydrogen and helium gas between the galaxies produced in the Big Bang—have revealed a discrepancy with the well-established predictions of our computational models. In particular, precision observations of the IGM using the Keck telescopes in Hawaii show that the temperature and ionization state of the IGM is not what our standard cosmological simulations predict. We present simulations which examine the hypothesis that hard UV radiation from hundreds of quasars provides the missing heat. We solve the equations of multigroup radiation diffusion for a collection of quasars in a large cosmological volume coupled to cosmological hydrodynamics. The size of the computational mesh and the implicit treatment of the 5 radiation group model make this a Blue Waters problem. We compare synthetic hydrogen and helium absorption spectra to a standard simulation to look for improved agreement with observations.

Field of Science: Astronomy & Astrophysics

Watch the presentation | View presentation slides (PDF)

Simulating the Earliest Generations of Galaxies with Enzo and Blue Waters

Project PI: Brian O'Shea, Michigan State University

Abstract: Galaxies are complex—many physical processes operate simultaneously, and over a huge range of scales in space and time. As a result, accurately modeling the formation and evolution of galaxies over the lifetime of the universe presents tremendous technical challenges. In this talk I will describe some of the important unanswered questions regarding galaxy formation, discuss in general terms how we simulate the formation of galaxies on a computer, and present simulations (and accompanying published results) that the Enzo collaboration has recently done on the Blue Waters supercomputer. In particular, I will focus on the transition from metal-free to metal-enriched star formation in the universe, as well as the luminosity function of the earliest generations of galaxies and how we might observe it with the upcoming James Webb Space Telescope.

Field of Science: Astronomy & Astrophysics

Watch the presentation | View presentation slides (PDF)

Insights into Opiate Binding and Activation of μ-Opioid Receptors

Project PI: Vijay Pande, Stanford University

Presented by: Amir Barati Farimani, Stanford University

Abstract: Many important analgesics relieve pain in human through binding to μ-Opioid receptors (μOR). To design effective pain medications, we should gain a deep understanding of the activation mechanism of μOR and how they attract drugs to bind. Since the nature of conformational changes in the activation pathway of these receptors is very subtle and encompass Microsecond timescales, we took advantage of Blue Waters to run thousands of MD simulations to be able to shed light into drug-receptor interaction. We used Markov State Models (MSMs) and Machine Learning techniques to characterize the conformational changes induced by ligand. Time-Structure Based Independent Component Analysis (tICA) enabled us to find and pick the most significant reaction coordinates and their associated trajectories among the massive number of trajectories produced by Blue Waters. We found significant intermediates in the binding pathway of ligand and activation of the receptor.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

From Sequence to Structure to Binding using Physics and GPUs

Project PI: Alberto Perez, Stony Brook University

Abstract: A long-standing challenge in the field of computational biology is how to derive the structure of proteins given their sequence. Proteins are the workhorses of the cell and knowing their structure opens the door to acting on their mechanisms of action and allows rational drug design. We know over 50 million protein sequences—and yet we only know the structure of about 100,000 or so. And this gap is widening. New tools and technology are needed and using physics in the pipeline poses answers not only to the protein structure problem but also opens the doors to other foldameric and self-assembling materials.

In this talk I will present our MELD technology—a physics based approach combining replica exchange molecular dynamics simulations with sparse, noisy and ambiguous data. I will show some of the blind protein structure prediction success stories with the methodologies as well as some of the new challenges we are tackling with the computer resources of Blue Waters, both in structure prediction and fleixible peptides binding to proteins.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Modeling Heliophysics and Astrophysics Phenomena with a Multi-Scale Fluid-Kinetic Simulation Suite

Project PI: Nikolai Pogorelov, University of Alabama in Huntsville

Abstract: The solar wind flow colliding with the local interstellar medium involves neutral atoms, and thermal and nonthermal ions. The ion-ion collisions are negligible while ion scattering on magnetic field fluctuations justify the application of MHD equations. Ion-neutral collisions should be treated kinetically. We have implemented this approach in the Multi-Scale Fluid-Kinetic Simulations Suite. Our time allocation on Blue Waters through the NSF PRAC program allowed us to perform breakthrough simulations and interpret a number of spacecraft observations: the structure of the heliopause, the effect of charge exchange and interstellar magnetic field on the heliotail topology; the influence of the heliosphere on the TeV cosmic ray anisotropy, creation of energetic neutral atoms, and modifications to the interstellar flow. We present these results together with the code scaling studies.

Field of Science: Astronomy & Astrophysics

Watch the presentation | View presentation slides (PDF)

Evolution of the Small Galaxy Population from High Redshift to the Present

Project PI: Thomas Quinn, University of Washington

Abstract: Creating robust models of the formation and evolution of galaxies requires the simulation of a cosmologically significant volume with sufficient resolution and subgrid physics to model individual star forming regions within galaxies. This project is aiming to do this modelling with the specific goal of interpreting Hubble Space Telescope observations of high redshift galaxies. To do this modelling, we are using the highly scalable N-body/Smooth Particle Hydrodynamics code, ChaNGa, based on the Charm++ runtime system on Blue Waters to perform a simulation of a 25 Mpc cubed volume of the Universe with a physically motivated star formation/supernovae feedback model. We also implemented a new model for black hole growth, mergers, and feedback to realistically simulate the properties of the higher mass galaxies.

Field of Science: Astronomy & Astrophysics

Watch the presentation | View presentation slides (PDF)

A Turbulent Dynamo in Rotating Magnetized Core-Collapse Supernovae

Project PI: David Radice, California Institute of Technology

Abstract: I will report on the result of a series of extremely high resolution simulations of rotating, magnetized, core-collapse supernova simulations that our team performed on Blue Waters. These are the first global simulations able to resolve the fastest growing mode of the so-called Magneto-Rotational Instability (MRI). This is an instability of weakly magnetized shear flows that results in the fast growth of the magnetic field. We find that the MRI also operates in core-collapse supernovae and can efficiently build magnetar-level magnetic fields. We also find that magneto-turbulence generated by the MRI results in dynamo action building large scale magnetic fields that could potentially power relativistic outflows and jet-driven supernovae.

Field of Science: Astronomy & Astrophysics

Watch the presentation | View presentation slides (PDF)

Effects of Forcing Scheme on the Flow and the Relative Motion of Inertial Particles in DNS of Isotropic Turbulence

Project PI: Sarma Rani, University of Alabama in Huntsville

Presented by: Rohit Dhariwal, University of Alabama in Huntsville

Abstract: In direct numerical simulations (DNS) of homogeneous isotropic turbulence, statistical stationarity is achieved through a forcing scheme that adds energy at the large scales. The forcing schemes may be broadly classified into deter- ministic and stochastic forcing schemes. In deterministic schemes, the tur- bulent kinetic energy dissipated during a time step is resupplied, whereas in stochastic schemes, forcing is determined based on the evolution of Ornstein- Uhlenbeck processes. Both approaches add the forcing within a band of wavenumbers at the low-wavenumber end of the energy spectrum. The goal of the current study is to investigate the effects of the forcing schemes on the flow, as well as on the relative motion of inertial particles in DNS of isotropic turbulence. An important parameter in stochastic forcing is the characteristic time scale of the applied forcing, TF . In this study, DNS was performed using the deterministic forcing, as well as stochastic forcing for five values of TF = Teddy/4, Teddy/2, Teddy, 2Teddy, 4Teddy. Here Teddy is the eddy turnover time obtained from the DNS with deterministic forcing. Two Taylor micro-scale Reynolds numbers Reλ = 76, 131, and twelve particle Stokes numbers based on the Kolmogorov time-scale, Stη ranging from 0.05 to 40 were considered. Detailed analysis of the effects of forcing time scales on both fluid and particle statistics was undertaken.

Field of Science: Fluid Systems

Watch the presentation | View presentation slides (PDF)

High Resolution Earth System Modeling using Blue Waters Capabilities

Project PI: Robert Rauber

Presented by: Susan Bates, NCAR

Abstract: This project aims to address uncertainties associated with numerical modeling of Earth's climate system and with modeled present and future climate change by conducting high-resolution simulations with the Community Earth System Model (CESM). This presentation will update the Blue Waters community on some of the activities during 2015 and lay the path forward for 2016. Building on 2014 results, further progress was made concerning tropical cyclone behavior within the CESM framework as well as changes in behavior in future scenarios. Pushing model resolution, our group is conducting the first-of-its-kind simulations including the fully-coupled 1/4° atm/lnd coupled to the 1° ocn/ice configuration and the fully-coupled 1/4° atm/lnd coupled to a 1/10° ocn/ice RCP8.5 scenario. The former set is used to assess climate sensitivity, 20th Century climate changes, and future climate changes while the latter simulation allows exploration of very high resolution impacts on regional features and climate change processes.

Field of Science: Atmospheric & Climate Science

Watch the presentation | View presentation slides (PDF)

Simulated Effects of Urban Environments on the Dynamics of a Supercell Thunderstorm

Project PI: Larissa Reames, The University of Oklahoma

Abstract: The goal of this study is to quantify the impacts of urban morphology on supercell thunderstorm dynamics. The Advanced Research Weather Research and Forecasting (ARW-WRF) model will be used to simulate an isolated supercell storm that traverses parts of Oklahoma. An initial simulation (CTRL) is performed wherein all model representations of urban areas have been removed. Further model runs are conducted in which the land use patterns of various cities are placed within the model domain in or near the path of the supercell. By comparing the simulations with urban areas to CTRL, the effects of the storm passing across different parts of the city or upwind/downwind of the city, as well as the storm lifecycle stage during this interaction, are investigated. These comparisons concentrate on differences in boundary layer characteristics prior to storm formation as well as changes in supercell structure, dynamics and evolution.

Field of Science: Atmospheric & Climate Science

Watch the presentation | View presentation slides (PDF)

Exploring the Fundamental Optical Properties of Methyl-Ammonia Lead Iodide Solar Cell Materials: A Computational Study

Project PI: Andre Schleife, University of Illinois Urbana-Champaign

Presented by: Joshua Leveillee, University of Illinois Urbana-Champaign

Abstract: Methyl-ammonia lead-iodide (MAPbI₃) perovskite solar cells have rapidly increased in photo-conversion efficiency and engineering synthesis viability over the past four years. Despite experimental progress, the underlying physics of how creation, lifetime, and separation of light-induced electron-hole pairs and the connection to high device efficiency in these unique materials is still heavily debated in the scientific community. In this study, we take a first principles approach to understanding the fundamental electronic-optical behavior of these perovskite solar cell materials. We use density functional theory (DFT) to obtain equilibrium atomic geometries and calculate the ground state of the different organic-metal halide perovskite material. Then, we use the Bethe-Salpeter equation (BSE) to calculate the optical polarization function. Finally, we compute optical and absorption spectra including excitonic effects. The Blue Waters Super Computer allows our team to rapidly compute large excitonic Hamiltonians, using the entire memory of several tens of nodes, and in some cases even pushing the memory available. We find that the organic cation, methyl-ammonia, potentially plays a central role by contributing high lattice screening of to the electron-hole interaction. This effect allows easier separation of electrons and holes. Charges may then flow freely, leading to efficient solar cell materials.

Field of Science: Materials Science

Watch the presentation | View presentation slides (PDF)

Molecular Dynamics Simulations of Large Macromolecular Complexes

Project PI: Klaus Schulten, University of Illinois at Urbana-Champaign

Presented by: Till Rudack, University of Illinois Urbana-Champaign

Abstract: Connecting dynamics to structural data from diverse experimental sources, molecular dynamics simulations permit the exploration of biological phenomena in unparalleled detail. Advances in simulations are moving the atomic resolution descriptions of biological systems into the million-to-billion atom regime, in which numerous cell functions reside. In this talk, I present the progress, driven by large-scale molecular dynamics simulations, in the study of viruses and bacterial systems. These examples highlight the utility of molecular dynamics simulations in the critical task of relating atomic detail to the function of supramolecular complexes, a task that cannot be achieved by smaller-scale simulations or existing experimental approaches alone.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Computational Ancestral Gene Resurrection for Investigating Activation Mechanisms of Cellular Signaling Proteins

Project PI: Diwakar Shukla, University of Illinois Urbana-Champaign

Presented by: Alexander Moffett, University of Illinois Urbana-Champaign

Abstract: Kinases are cellular signaling proteins involved in various physiological functions. Small molecules and other allosteric enhancers bind to these proteins and modulate their function. The drug binding sites of kinases share a high degree of similarity but differ significantly in their selectivity towards the same binding partner. Designing selective molecules and elucidating functional mechanisms is a key challenge in the drug discovery pipeline. Since the conformational changes of these proteins occur at long time scales, powerful computational resources are required to study these complex systems. Using Blue Waters supercomputer, we have been able to study these rare biological events at multi-microsecond time scales. In particular, we employed computational ancestral gene resurrection methodology to investigate ancestral proteins along the evolutionary tree connecting the modern proteins to shed light on the molecular origin of the differences in selectivity of the modern proteins. Our results reveal the molecular mechanisms of ligand selectivity in kinases along with the atomic level details of conformational changes associated with these binding events. We have also investigated Kinases associated with plant growth to elucidate molecular mechanisms responsible for plant growth under adverse environmental conditions. These simulations have revealed new mechanisms for conformational control of kinase activity in plants.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Location-dependent Space Weather Hazards of Societal Significance within the Earth-Ionosphere Waveguide

Project PI: Jamesina Simpson, University of Utah

Abstract: The historical record indicates the possibility of extreme geomagnetic storms on Earth resulting from coronal mass ejections on the sun. These geomagnetic storms can create intense electromagnetic fields at the Earth's surface that can disrupt electric power grid operations, communications, geolocation, etc. The goal of the proposed work is to greatly improve our ability to understand and predict space weather hazards to: (1) electric power grids; and (2) geolocation. The methodology of the proposed work is to advance and apply detailed, high-resolution Maxwell's equations models of the Earth-ionosphere waveguide developed by the PI over the past decade. These models are based on the full-vector Maxwell's equations finite-difference time-domain (FDTD) method. They uniquely account for the Earth's complete topography, oceans, lithosphere composition, geomagnetic field, and magnetized ionospheric plasma according to altitude, position, and time of day.

Blue Waters is permitting us to run unique, highly detailed simulations of geomagnetic storms. For example, to properly study geomagnetic storm effects on surface electromagnetic fields at ocean-continent boundaries, memory intensive global grid resolutions on the order of meters are required. Further, Blue Waters is permitting us to extend the model further up into the ionosphere than previously possible.

Field of Science: Atmospheric & Climate Science

Watch the presentation | View presentation slides (PDF)

Extreme-scale Graph Analysis on Blue Waters

Project PI: George Slota, Penn State / Sandia Labs

Abstract: In recent years, many graph processing frameworks have been introduced with the goal to simplify analysis of real-world graphs on commodity hardware. However, these popular frameworks lack scalability to modern massive-scale datasets. This work introduces a methodology for graph processing on distributed HPC systems that is simple to implement, generalizable to broad classes of graph algorithms, and scales to systems with hundreds of thousands of cores and graphs of billions of vertices and trillions of edges. We demonstrate our approach to be orders-of-magnitude faster than other distributed graph processing frameworks for several graph analytics. Additionally, we show how our methods aren't limited to simple algorithms: We also implement a distributed version of the PuLP graph partitioner and use it to partition a real-world graph with over a hundred billion edges and synthetic graphs with over a trillion edges on Blue Waters; each partition computation completes in only minutes. This work opens the door for the complex study of the large and irregular interaction datasets that are ubiquitous throughout the social and physical sciences.

Field of Science: Computer Science

Watch the presentation | View presentation slides (PDF)

Simulation-based Policy Analysis to Reduce Ebola Transmission Risk in Air Travel

Project PI: Ashok Srinivasan, Florida State University

Abstract: Air travel is a major cause of the spread of infections. This led to calls for ban on air travel during the recent Ebola outbreak. However, such bans have serious human and economic consequences. The goal of our project is to identify policies and procedures that can provide the same benefit without the negative consequences. This is accomplished using a fine-scale model that tracks individual passenger movement in airplanes. Inherent uncertainties in human behavior make it difficult to accurately predict the consequences of any particular policy choice. Instead, it is more fruitful to determine vulnerabilities under a variety of possible scenarios. We parameterize the sources of uncertainty and perform simulations to cover the range of uncertainties. This approach leads to high computational cost, which is handled through massive parallelization on Blue Waters. Our results show potential for substantial reduction in Ebola spread by changing current boarding and disembarkation procedures.

Field of Science: Computer Science

Watch the presentation | View presentation slides (PDF)

Analyzing Tropical Cyclone-climate Interactions using the Community Earth System Model (CESM)

Project PI: Ryan Sriver, University of Illinois at Urbana-Champaign

Presented by: Hui Li, University of Illinois at Urbana-Champaign

Abstract: Tropical cyclones (TCs) have the potential to influence regional and global climate through their interactions with the upper ocean. Here we highlight results from a series of ocean-only simulations featuring a high-resolution state-of-the-art Earth system model (Community Earth System Model -- CESM), in which we analyze the effect of tropical cyclone wind forcing on the global ocean using three different horizontal ocean grid spacing (3°, 1°, and 0.1°). Findings indicate that TCs significantly contribute to global ocean heat and energy budgets, pointing to important connections between TCs and ocean dynamics, which can influence seasonal to interannual climate variability and large-scale circulation patterns in the atmosphere and ocean. Furthermore, TCs' contribution to the climate system may have important implications for anthropogenic global warming through feedbacks in the coupled system, which is one of the scientific questions we seek to answer in the ongoing work.

Field of Science: Atmospheric & Climate Science

Watch the presentation | View presentation slides (PDF)

Lattice QCD on Blue Waters

Project PI: Robert Sugar, University of California Santa Barbara

Presented by: Steve Gottlieb, Indiana University

Abstract: Using highly optimized code for QCD on Blue Waters we have carried out calculations of major importance in high energy and nuclear physics. We generated gauge configurations (samples of the QCD vacuum) with both the HISQ and Wilson-Clover actions. On Blue Waters we are able to reduce the masses of the up and down quarks to the value they have in Nature. These leads to a major improvement as we no longer have to extrapolate in quark masses. Smaller lattice spacings of 0.042 and 0.03 fm are being used to study leptonic decays of pi, K, D, Ds, B and Bs mesons, and semileptonic decays K-> pi l nu, D-> K l nu, and D-> pi l nu. The Wilson-Clover project explores the isovector meson spectrum. Finite volume techniques are being used to determine hadron resonance parameters in the scattering of pion, kaon, and eta mesons. We computed quark propagators on $32³ \times 256$ and $40³ \times 256$ ensemble with 230 MeV pions. New $64³ \times 96$ and $72³\times 196$ ensembles will allow exploration of possible exotic meson states with energies around 2 GeV.

Field of Science: Physics

Watch the presentation | View presentation slides (PDF)

DeepPep: Deep Proteome Inference from Peptide Profiling

Project PI: Ilias Tagkopoulos, University of California at Davis

Presented by: Minseung Kim, University of California at Davis

Abstract: Proteome reconstruction is a fundamental challenge in molecular biology. In many cases, especially in food science, such reconstruction has been elusive until now because of partial degradation of the proteome through different natural processes. Here we present a novel deep-convolutional neural network framework called DeepPep that predicts the minimal set of proteins that can generate a given profile of short peptides. DeepPep was able to reconstruct accurately the protein set from dipeptides at various cellular states and low detection thresholds. These results argue that a peptide's sequence location is a sufficient informative feature to predict protein presence and prevalence from metabolomic mass spectrometry samples. DeepPep was also able to perform protein reconstruction from longer peptide samples with better accuracy when compared to other traditional machine learning techniques, thus implying that deep learning methods can be competitively applied in the proteomics field.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Fluid-flow and Stress Analysis of Steel Continuous Casting

Project PI: Brian Thomas, University of Illinois at Urbana-Champaign

Presented by: Seong-Mook Cho, University of Illinois at Urbana-Champaign

Abstract: Many defects in steel products are caused by transient fluid flow, such as level fluctuations capturing slag inclusions at the top surface of the mold. This project aims to develop advanced computational models of the continuous casting of steel, including multiphase turbulent fluid flow, MagnetoHydroDynamics, particle transport, interfacial behavior, heat transfer, solidification, and thermal-mechanical behavior; to apply these models to better understand the multiphysics phenomena related to defect formation; and to evaluate ways to improve this important commercial process. Recently, an important new relation between swirl in the nozzle and mold surface flow variations has been discovered and quantified from Large Eddy Simulation results obtained using a special multi-license version of ANSYS Fluent on Blue Waters. To capture the oscillating swirl behavior in the turbulent flow requires extremely small time steps, fine computational grids and long times, which is enabled by Blue Waters, with speedup >3,000 times for some cases.

Field of Science: Engineering

Watch the presentation | View presentation slides (PDF)

Nonthermal Electron Energization from Magnetic Reconnection in Laser-driven Plasmas

Project PI: Samuel Totorica, Stanford University

Abstract: Particle acceleration induced by magnetic reconnection is thought to be a promising candidate for producing the nonthermal emissions associated with explosive phenomena such as solar flares, pulsar wind nebulae, and jets from active galactic nuclei; however, the dominant acceleration mechanisms, their efficiency, and spectral signatures are not yet fully understood. Laboratory experiments can play an important role in the study of particle acceleration by reconnection. We have used particle-in-cell simulations to study particle acceleration in regimes associated with laser-driven plasma experiments. For current experimental conditions, we show that nonthermal electrons can be accelerated to energies more than an order of magnitude larger than the initial thermal energy. We provide an analytical estimate of the maximum electron energy and threshold condition for observing suprathermal electron acceleration in terms of experimentally tunable parameters. These results open the way for a new platform for the experimental study of particle acceleration induced by reconnection.

Field of Science: Physics

Watch the presentation | View presentation slides (PDF)

The Realization of Extreme Tornadic Storm Events under Future Anthropogenic Climate Change

Project PI: Robert Trapp, University of Illinois at Urbana-Champaign

Abstract: This research seeks to answer the basic question of how current-day extreme tornadic storm events might be realized under future anthropogenic climate change. The "e;pseudo-global warming"e; (PGW) methodology was adapted for this purpose. In our application of this methodology, global climate model simulations under historical and future (RCP8.5) conditions were used to modify the initial and boundary conditions applied to an ensemble of WRF-model simulations of three high-end tornadic events. Control simulations (CTRL) without such modifications facilitated assessment of PGW effects..

In contrast to the robust development of supercellular convection in each CTRL, convective storms failed to initiate in many of the PGW experiments. However, the PGW experiments that did support storm initiation also tended to generate rotating convective updrafts that were stronger than in CTRL. Notably, the PGW modifications did not induce a change in the convective morphology in any of the experiments with significant convective storminess.

Field of Science: Atmospheric & Climate Science

View presentation slides (PDF)

Dependence of Directional Intensity and Polarization of Light Scattered by Small Ice Crystals on Microphysical Properties: Application to Forward-scattering Probes, Satellite Retrieval, and Numerical Models

Project PI: Junshik Um, University of Illinois at Urbana-Champaign

Abstract: Current in-situ airborne probes measure sizes of ice crystals smaller than 50 micrometers assuming that the intensity of light scattered by a particle in the forward and sometimes backward direction is a function of particle size. The relationship between particle size and scattered light used in current forward scattering probes is based on Mie theory, which assumes the refractive index of a particle is known and all particles are spherical. Not only are small crystals not spherical, but also there are a wide variety of non-spherical shapes. To improve in-situ airborne measurements of small crystals, satellite retrieval algorithms, and numerical models, precise relationships between the light scattering by a particle and its size and shape are required, and are based on accurate calculations of single-scattering properties of crystals. However, such calculations using numerically exact methods require a large amount of computing time and memory, which increase with ice crystal size. We use Blue Waters for these calculations.

Field of Science: Atmospheric & Climate Science

Watch the presentation | View presentation slides (PDF)

Dynamic Coarse-grained Models for Simulations of Large-Scale Biophysical Phenomena

Project PI: Gregory Voth, University of Chicago

Presented by: John Grime, University of Chicago

Abstract: Atomic-resolution computer simulations can provide highly detailed information about biomolecular systems, but are restricted to time- and length-scales that may be insufficient for the study of certain biological phenomena. One alternative technique is the use of "coarse-grained" (CG) models, where degrees of freedom are removed to generate simpler molecular representations while retaining the essential characteristics of the detailed system. Due to their computational efficiency, coarse-grained models provide an appealing tool for the study of relatively large-scale biomolecular phenomena. The application of CG models and software to the study of model systems such as the nucleation and growth of HIV-1 capsid protein lattice will be discussed, with specific emphasis placed on the use of the Blue Waters platform.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Advancing Genome-scale Phylogenomic Analysis

Project PI: Tandy Warnow, University of Illinois at Urbana-Champaign

Abstract: Research Goals: The project has three main aims, each geared towards advancing the accuracy of large-scale estimation of evolutionary history: method development for multiple sequence alignment, maximum likelihood phylogeny estimation, and species tree estimation. These scientific challenges are also computationally very difficult, because nearly all desirable approaches are computational methods that attempt to solve NP-hard optimization problems, or that seek to find optimal statistical models in a high-dimensional parameter space. Many biological datasets require months of analysis, and some recent biological studies (e.g., the Avian phylogenomics project (2)) have spent hundreds of CPU years in computing phylogenies for their phylogenomic datasets.

Research Highlights: 1. Scalable versions of BAli-Phy (3), a Bayesian method for statistical co-estimation of multiple sequence alignments and trees so that they can analyze large datasets (our current implementations show scalability to 10,000 sequences, whereas the original implementation could only analyze about 100 sequences). 2. A new method (HIP-HOP) for classifying sequences into gene families, that substantially improves the accuracy compared to all current alternative methods (including BLAST (1)). 3. New supertree methods with improved accuracy and scalability, that will enable the development of divide-and-conquer methods for estimating large phylogenetic trees.

Field of Science: Biosciences

Watch the presentation | View presentation slides (PDF)

Unlocking the Mysteries of the Most Violent Tornadoes

Project PI: Robert Wilhelmson, University of Illinois at Urbana-Champaign

Presented by: Leigh Orf, University of Wisconsin - Madison

Abstract: Devastating, long-lived tornadoes are rare, but the death and destruction they cause is significant. In this presentation, we review nearly four years of our work on Blue Waters, focusing on recent supercell thunderstorm simulations in which long-lived, violent tornadoes occur. We will present an overview of the challenges we needed to overcome in order to simulate and visualize tornadoes embedded within their parent thunderstorm at ultra-high spatial and temporal resolution. Newly discovered flow features identified in our simulations indicate mechanisms that may play an important role in discerning the most devastating, tornado-producing thunderstorms from those that are much more common and less destructive. Recent simulations run with 20 meter isotropic grid spacing reveal the internal structure of a long-lived, devastating tornado, which undergoes morphological transitions from a narrow, single-celled tornado to a wide, multiple-vortex tornado.

Field of Science: Atmospheric & Climate Science

Watch the presentation

The Hydrogen Ingestion Flash in a Low-Z AGB Star of the Early Universe and the Special Challenges it Presents to Computation

Project PI: Paul Woodward, University of Minnesota

Abstract: Nucleosynthesis in low-mass stars in the early universe can be strongly influenced by hydrogen ingestion events. Mixing at convection zone boundaries brings highly combustible fuel into regions hot enough that it burns with a flash of very strong energy release. This process can be brief enough to permit it to be simulated in 3D on Blue Waters. Results of such a simulation will be presented. The behavior is very different from 1-D stellar evolution treatments of the process. Simulation of these events demands that calculations be carried out over many dynamical times, involving millions of time steps. To treat more complex problems of this type, such as the merger of multiple burning shells in massive stars, multiple time and length scales must also be accommodated. Efforts to address these challenges with assistance from the Blue Waters project through its "PAID" program will be discussed.

Field of Science: Astronomy & Astrophysics

Watch the presentation | View presentation slides (PDF)

Particle Tracking and Turbulent Dispersion at High Reynolds Number on Blue Waters

Project PI: Pui-kuen Yeung, Georgia Institute of Technology

Presented by: Dhawal Buaria, Georgia Institute of Technology

Abstract: As reported last year, a half-trillion-grid points simulation of turbulence performed on Blue Waters has provided clear evidence of extreme events where local measures of the deformation and rotation of local fluid elements are much stronger than and have topological properties different from previously thought. In this talk we focus on turbulent dispersion at high Reynolds number in the Lagrangian reference frame of an observer moving with the flow. A substantial algorithmic change in the interpolation of fluid particle velocities on a distributed domain has led to a dramatic improvement in the scalability of the particle tracking algorithm at 262,144 cores. We present a number of important results, such as the Lagrangian acceleration autocorrelation, and the scaling of velocity increment statistics conditioned upon the dissipation rate. Particle pair trajectories are also followed both forward and backward in time, which provide unique insights for the physics of enhanced mixing in turbulence.

Field of Science: Fluid Systems

Watch the presentation | View presentation slides (PDF)

Modeling the Ocean Energetics through Horizontal Convection

Project PI: Varvara Zemskova, University of North Carolina at Chapel Hill

Abstract: The driving forces of the Meridional Overturning Circulation (MOC) are one of the central questions in oceanography and climate, and it is important to understand its mechanism and sensitivity to changing ambient conditions. The key to the energetics of the MOC is in the relative importance of mechanical forcing due to surface winds and surface buoyancy forcing. We investigate the energetics of a 3D Direct Numerical Simulation runs of a simplified model of the Southern Ocean, where contributions from both surface buoyancy and wind forcing are important. The domain is a rectangular basin with buoyancy forcing applied at the surface with cooling near the pole and heat at mid-latitudes and wind stresses of variable magnitudes. For each simulation, generation and conversion terms in the energy budget are calculated using the local APE framework to investigate the effects of surface forcing on the large-scale overturning, baroclinic eddy generation in the ACC, and diapycnal mixing.

Field of Science: Fluid Systems

Watch the presentation | View presentation slides (PDF)

Blue Waters is supported by the National Science Foundation (awards OCI-0725070 and ACI-1238993) the State of Illinois, and the National Geospatial-Intelligence Agency.

Contact Blue Waters Team with questions regarding this page.
Copyright 2024 Board of Trustees of the University of Illinois. All rights reserved. Web privacy notice.