Powering the Future of Cosmology With AI

Researchers at Carnegie Mellon University and partners at the University of Washington are collaborating with the U.S. Department of Energy's SLAC National Accelerator Laboratory on a new Genesis Mission-funded project to help scientists use artificial intelligence to better understand the universe.

Modern astronomy is producing more data than ever before. Powerful telescopes and instruments from around the world are collecting detailed information about billions of stars, galaxies and other cosmic objects. These observations help researchers investigate some of the biggest mysteries in science, including the nature of dark matter and dark energy, how the universe evolved over time and what it is made of.

Projects such as the NSF-DOE Vera C. Rubin Observatory's Legacy Survey of Space and Time (LSST)(opens in new window), the Dark Energy Spectroscopic Instrument (DESI)(opens in new window), and future cosmic microwave background experiments are expected to generate enormous amounts of information in the coming years.

But there is a challenge: much of that data is stored in different formats, housed at different institutions and difficult to combine.

As a result, scientists often spend significant time preparing data before they can begin analyzing it.

“The scientific opportunities and the data analysis challenges are incredible,” said Rachel Mandelbaum(opens in new window), head of Carnegie Mellon’s Department of Physics(opens in new window) and a member of the McWilliams Center for Cosmology and Astrophysics(opens in new window).

Mandelbaum is the principal investigator for a project to create a shared data service to allow researchers to seamlessly access and combine information from multiple astronomy experiments. Rather than moving massive datasets from one location to another, the system will allow the data to remain where it is stored while making it available through a unified platform.

The data infrastructure developed as part of this project will be available to the astronomical community at the SLAC-hosted Rubin Observatory’s U.S. Data Facility (USDF) and via the American Science Cloud, which integrates the nation’s most advanced high-performance computing systems, scientific facilities, data resources and production capabilities into a single, coordinated AI-driven discovery system.

In addition to Mandelbaum, co-investigators include: Jeremy Kubica, director of engineering for LSST Interdisciplinary network for Collaboration and Computing (LINCC) Frameworks at Carnegie Mellon; Andrew Connolly, associate vice provost for data science, professor in the department of astronomy and director of the eScience Institute at the University of Washington; Neven Caplar, research scientist/engineer at the University of Washington; and Adam Bolton, Senior Staff Scientist at SLAC and at Stanford’s/SLAC’s Kavli Institute for Particle Astrophysics and Cosmology (KIPAC) and lead for the USDF. Additional modeling will be provided by Francois Lanusse of the French National Centre for Scientific Research. Their work is an extension of the Schmidt Sciences-supported LINCC Frameworks program, a partnership led jointly by CMU and UW.

"A new generation of telescopes and surveys will each change the way we understand our universe, but it is when we bring these data together to look at the universe from a unified perspective that these discoveries will be truly transformative," said Connolly.  

The infrastructure also will support a growing area of AI known as foundation models. These AI systems are trained on large and diverse datasets, enabling them to recognize patterns and connections that might otherwise go unnoticed.

In astronomy, foundation models could help researchers analyze many different types of observations at once, including images, measurements of light from distant objects and records of how those objects change over time. By bringing these data sources together, scientists hope to uncover new insights about the universe more quickly and efficiently.

The collaboration will build infrastructure that can be used by the broader cosmology community, including researchers working at Department of Energy national laboratories and universities around the world. By making it easier to combine and analyze data, the project is expected to accelerate scientific discovery and enable new AI-driven approaches to understanding the cosmos.

Ultimately, the team hopes to transform the vast collections of astronomical data being gathered today into a long-lasting scientific resource, helping researchers answer some of humanity's most fundamental questions about the origin, evolution and makeup of the universe.