When One Drone Isn’t Enough: CMU Builds Swarms for High-Stakes Response Efforts

Response teams in high-stakes environments like natural disasters, mass casualty events and battlefields face a lot of risks. Carnegie Mellon University is developing coordinated groups of drones — called drone swarms — and AI systems that can be deployed to keep more people out of danger.

Training drones to autonomously assess human vitals

Researchers at CMU are building AI systems for drone swarms that can use cameras to scan people’s bodies and assess their vitals from a distance. This work is part of the Defense Advanced Research Projects Agency (DARPA) Triage Challenge(opens in new window), aimed at improving emergency response in mass casualty and battlefield scenarios. 

“Our goal is to identify who needs the most help and what kind of help they need,” said Artur Dubrawski(opens in new window), Alumni Research Professor of Computer Science in the Robotics Institute(opens in new window) and member of Auton Lab(opens in new window). “By using drone swarms to do this, we can assess the same victim from multiple angles to create more data.”

Dubrawski said they plan to employ teams of drones of varying sizes with different capabilities: Smaller robots can access tight spaces, while larger ones can carry heavier equipment like cameras. 

Drones with cameras will be able to scan wounded bodies found in the field for vital signs. For example, drones equipped with AI models the team developed can recognize slight changes in facial color to estimate heart rate. 

“We are training our models to assess vital information like respiratory rate, heart rate and body temperature using cameras that scan the body from three meters away,” Dubrawski said. 

According to Dubrawski, some of this assessment technology has been around for years, but only worked successfully in very controlled lab environments. Outside the controlled conditions of the lab, rain, shadows and partially hidden faces prevented the AI from accurately scanning the people it was instructed to assess.

“Nothing we did in the lab worked as advertised in the field,” Dubrawski said. “It took us about two years to develop robust AI methods that are at least partially immune to these challenges.”

Now, he feels that these models can triage people in complicated environments: in the dark, through smoke, and even faces partially obscured by rubble, mud or blood. 

“What we have now is a basic but reasonably advanced proof of concept,” he said. “We’re still a few years away from deploying it, but we can make this work in the real world.”

Mapping rescue missions faster than humanly possible

Wennie Tabib(opens in new window), a systems scientist at CMU’s Resilient Intelligent Systems Lab(opens in new window), believes that search and rescue drones can help emergency responders navigate the subterranean paths of caves and unstable earthquake rubble more safely. 

“Cave rescues happen fairly rarely, but when it does happen it can be catastrophic,” Tabib said. “The Thai cave(opens in new window) rescue operation to save a team of young soccer players was very challenging. That situation has a lot of parallels to searching for people in collapsed buildings after an earthquake. We are lacking robotic tools that can speed up search and rescue operations where time is of the essence.”

Rescue efforts are always working against the clock, as the survival rate for victims trapped in natural disaster situations plummets(opens in new window) after 72 hours. 

“My dream would be that we get these vehicles miniaturized to such an extent that you can have someone pull out a handful of drones, throw them, have them quickly search a cave or disaster scenario and collaboratively build a map of the scene,” Tabib said.

But just like humans, drones have their own set of limitations. Namely, Tabib said, the robot’s size and the battery life required to power a robot that’s constantly flying or hovering. To that end, they’re experimenting with ways to make drones smaller and lighter so they can fly farther and fit inside tighter spaces. 

“The algorithms have to be very computationally efficient because they run on size, weight and power-constrained systems,” she said. “We need the algorithm to run quickly to get the drone to fly fast, search as much of the cave or disaster area as possible and return safely.”

Another problem is that once the robot goes into a remote location, operators can’t communicate with it, Tabib said.

There’s no cloud or internet communications available in these extreme environments. That means all the photos and data it collects has to be stored locally on the device that the rescue team accesses when it gets back,” she said.

Tabib’s team is also working on how to increase the physical capabilities of the drones they’re designing to make even more detailed maps of hazardous environments. Recently, they figured out how to get a drone to flip over.

“Imagine you have a drone with one camera pointed at a 45 degree angle down to see the floor ahead,” she said. “You could quickly flip that drone upside down and take pictures of the ceiling without substantially increasing the payload.” 

Tabib believes the applications of drone teams can go far beyond caves. The search and rescue drones she works on have already garnered interest from people across the country worried about increasing natural disasters caused by climate change. 

“Floods create a particular challenge,” Tabib said. “We’ve seen entire towns go underwater and people are waiting to be rescued while they’re standing on their roofs. Having a swarm of drones that conduct an automated search for survivors is an efficient solution. Drones can fly through the air at 100 kilometers an hour.”

Tabib, credits the collaborative effort of many other researchers at the Robotics Institute and the National Robotics Engineering Center(opens in new window) as key to the progress CMU has made in this line of research. 

“We’re all driving toward the same vision,” she said. “It’s definitely a team effort.”