In 2015, the Python-ARM Radar Toolkit (Py-ART) made its open-source debut. After 4 years, and with contributions from 34 individual editors, it is now a staple in radar science. The toolkit helps scientists analyze radar data to improve models of the Earth’s systems; its growth illustrates the power of community software.
Py-ART is an architecture for working with radar data in the Python programming language. It ingests data from a wide variety of atmospheric radars to produce visualizations that enable users to draw meaningful conclusions. Institutions across the world — including the National Weather Service, MeteoSwiss, IBM and the University of Illinois — use Py-ART to organize and analyze radar data.
Atmospheric scientist Scott Collis originally developed the software to retrieve environmental information from the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) User Facility’s diverse radar network, which consists of radars that output data in a range of formats. In 2012, Collis, who works at DOE’s Argonne National Laboratory, attended the Scientific Computing with Python Conference and, inspired by the open-source software he saw there, uploaded Py-ART to the community development platform GitHub.
“The old software lived and died with a few people developing it in silos rather than bringing in users to contribute. Now radar science isn’t limited to large institutions that have people dedicated to looking after software — we’ve democratized working with radar data.” — Scott Collis, Argonne atmospheric scientist
“I wanted to put my code out there so that it grew organically to suit the needs of the community,” said Collis. “I took my ideas to ARM and they gave us the support we needed to release Py-ART.”
The toolkit quickly grew in popularity with a range of third parties, including universities, military organizations, the private sector, meteorological agencies and national laboratories. As the number of users grew, so did the number of developers, resulting in an ecosystem of applications built on top of Py-ART.
“Open-source software is better software,” said Collis, “and we at the DOE benefit from our users contributing code back to Py-ART because we can use it to meet the nation’s need for improved earth system modeling capabilities.”
Py-ART is also increasing the number of people working with radar data. Before Py-ART, many institutions were working with complicated, outdated software that required very specialized environments to function. Py-ART, however, is easy to learn and can work on virtually any computer.
“The old software lived and died with a few people developing it in silos rather than bringing in users to contribute,” said Collis. “Now radar science isn’t limited to large institutions that have people dedicated to looking after software — we’ve democratized working with radar data.”
Although originally intended to work with ARM’s radars, Py-ART has collected a suite of application-based codes that have attracted users across many disciplines.
An example of the software’s usefulness is RELAMPAGO-CACTI (Remote sensing of Electrification, Lightning, And Mesoscale/microscale Processes with Adaptive Ground Observations – Clouds, Aerosols, and Complex Terrain Interactions), a field campaign studying the development and effects of thunderstorms in Argentina, funded by the National Science Foundation and DOE’s ARM facility. From November 1 to December 15, scientists collected and analyzed extensive data from radar in the La Planta River basin, the site of some of the world’s most intense storms.
With Py-ART, scientists can merge information — like moisture levels or elevation and direction of antennae — from many types of radar whose data all come in different formats. The software can then use this information to produce real-time operational displays and 3-D maps to track and forecast the storms.
When scientists better understand what leads to these storms, they can improve models to help predict storms in Argentina and other parts of the world.
“Having a complete radar toolkit that syncs with many other Python plotting and numerical toolkits makes Py-ART a go-to resource for our research group,” said Professor Steve Nesbitt, a scientist and principal investigator on the RELAMPAGO-CACTI team. “Gone are the days of using tools that take hours to set up on each machine. Now we can be up and running in minutes on our own machines, on clusters and in the cloud.”
Nesbitt is also an associate professor at the University of Illinois Urbana-Champaign, where he uses Py-ART for educational and research purposes. “Our research group uses Py-ART in many different ways, from visualizing weather data to doing advanced comparison of aircraft and radar data,” said Nesbitt, “and I have used Py-ART to teach several radar training courses.”
Because Py-ART is open source, and U.S. weather radar data is openly available on the Amazon cloud, students can solve realistic research problems using the software. “Py-ART is a conduit for radar-specific science to reach people who might not be radar experts,” said Collis.
Inspired by Py-ART’s success, scientists have launched the OpenRadar Partnership, an informal collaboration across Europe, Canada and the United States on open-source radar software education and inter-compatibility.
Although scientists primarily use Py-ART to study and predict the weather, its applications are vast. The American Meteor Society, for example, is using Py-ART to develop techniques for locating land-falling meteorites using weather radar. Scientists have also used it in machine learning algorithms that track birds using radar data.
Py-ART currently has more than 35,000 downloads; more than 50 published papers and theses have cited the software. It already powers several web pages, and its popularity and contributor community is growing.
Virtually anyone can contribute to Py-ART, and Collis and the other editors keep a website to show updates to the code. They also have a mailing list, Twitter and Facebook pages.
Py-ART is funded by the DOE Office of Science. ARM is a DOE Office of Science user facility.
Argonne National Laboratory seeks solutions to pressing national problems in science and technology. The nation’s first national laboratory, Argonne conducts leading-edge basic and applied scientific research in virtually every scientific discipline. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state and municipal agencies to help them solve their specific problems, advance America’s scientific leadership and prepare the nation for a better future. With employees from more than 60 nations, Argonne is managed by UChicago Argonne, LLC for the U.S. Department of Energy’s Office of Science.
The U.S. Department of Energy’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visit https://energy.gov/science.