Hands-on exhibits are staples of science and children's museums around the world. The exhibits invite children to explore scientific concepts in fun and playful ways.

But do kids actually learn from them? Ideally, museum staff, parents or caregivers are on hand to help guide the children through the exhibits and facilitate learning, but that is not always possible.

Researchers from Carnegie Mellon University's Human-Computer Interaction Institute(opens in new window) (HCII) have demonstrated a more effective way to support learning and increase engagement. They used artificial intelligence (AI) to create a new genre of interactive, hands-on exhibits that includes a virtual assistant to interact with visitors.

When the researchers compared their intelligent exhibit to a traditional one, they found that the intelligent exhibit increased learning and the time spent at the exhibit.

"Having artificial intelligence and computer vision turned the play into learning," said Nesra Yannier(opens in new window), HCII faculty member and head of the project, who called the results "purposeful play."

Earthquake tables are popular exhibits. In a typical example, kids build towers and then watch them tumble on a shaking table. Signs around the exhibit try to engage kids in thinking about science as they play, but it is not clear how well these work or how often they are even read.

Yannier led a team of researchers that built an AI-enhanced earthquake table outfitted with a camera, touchscreen, large display and an intelligent agent, NoRilla, that replaced the signs. NoRilla — a virtual gorilla — interacts with participants, taking them through different challenges and asking questions about why towers did or didn't fall along the way and helping them make scientific discoveries.

The team — Yannier, Ken Koedinger(opens in new window) and Scott Hudson(opens in new window) from CMU; Kevin Crowley of the University of Pittsburgh; and Youngwook Do of the Georgia Institute of Technology — tested their intelligent earthquake exhibit at the Carnegie Science Center in Pittsburgh. Elementary-school-aged children attending a summer camp interacted with either the intelligent or traditional exhibit and completed pre- and post-tests as well as surveys to gauge what they learned and how much they enjoyed the experiment. Researchers also observed visitors interacting with the exhibit during regular hours.

The pre- and post-tests and surveys revealed that children learned significantly more from the AI-enhanced intelligent science exhibit compared to the traditional exhibit while having just as much fun. A surprising result was that even though children were doing more building in the traditional exhibit, their building skills did not improve at all, as they mostly engaged in random tweaking rather than understanding the underlying concepts. The AI-enhanced exhibit not only helped children understand the [underlying] scientific concepts better but also transferred to better building and engineering skills as well.

Their experiment at the Science Center also showed that people spent about six minutes at the intelligent exhibit, four times the 90-second average of the traditional one.

"What's particularly impressive to me is how the system engages kids in doing real scientific experimentation and thinking," said Koedinger, a professor in HCII. "The kids not only get it, they also have more fun than with usual exhibits even though more thinking is required."

Parents of children who experienced the exhibit said it was more interactive, directed and instructional and offered two-way communication compared to other exhibits. They also commented that "it employs inquiry learning, which is the heart of how kids learn, but is also a play model, so it does not seem like a learning activity."

"Our exhibit automated the guidance and support that make hands-on physical experimentation a valuable learning experience," Yannier said. "In museums, parents may not have the relevant knowledge to help their children, and staff may not always be available. Using AI and computer vision, we can offer this experience to more children of different backgrounds and at a wider scale."

The team's research started at the Children's Museum of Pittsburgh, where they tested the design of their intelligent exhibit and made improvements based on feedback from people who interacted with it.

"This research will have lasting implications for future exhibit experiences at the Science Center," said Jason Brown, the Henry Buhl Jr. director of the Carnegie Science Center. "Creating hands-on fun and inspirational exhibit experiences that scaffold science, technology, engineering or mathematics learning and discovery is what positions us as one of the most unique museums in the region."

The team recently published its findings in the Journal of the Learning Sciences(opens in new window). The intelligent science exhibit remains at the Carnegie Science Center as a long-term exhibit. It is also at the Children's Museum of Atlanta and will soon be at the Please Touch Museum in Philadelphia and the Children's Discovery Museum of San Jose in California.

"The Children's Museum of Atlanta is enjoying being a part of this research study. As we have observed the NoRilla in action, we see high levels of 'stay time' for children and adults as they work to meet the challenges through the combination of hands-on activities with computer-based challenges," said Karen Kelly, the director of exhibits and education at the Atlanta museum. "We love that this experience aligns with our mission of sparking every child's imagination, sense of discovery and learning through the power of play."

The CMU team is already working on creating other intelligent science exhibits using computer vision and AI to teach different scientific topics. Future projects include an exhibit with ramps and one with a balance scale.

Yannier stressed that this technology will not only enhance lessons in a museum, but could also assist students learning in the classroom or at home.

This research was funded by an Advancing Informal STEM Learning grant from the National Science Foundation.