Navigating the AI Era with a CMU Focus on Critical Thinking

Carnegie Mellon University is weaving AI into coursework across campus to ignite student curiosity, creativity and critical rigor. Whether exploring the ethics of algorithmic decision-making or debating the societal impact of new technology, CMU faculty are equipping students with the agency to define and shape the AI of tomorrow, not just master the AI of today.

“At CMU, we train students how to be contributors to the next generation of tools,” said Tom Cortina(opens in new window), associate dean for undergraduate programs for the School of Computer Science(opens in new window) (SCS) and teaching professor in the Computer Science Department(opens in new window). “They’re going to use these AI tools, but they have enough knowledge that they can work on the tools themselves, and that means we as faculty have to adjust our teaching to focus on problem-solving and deeper thinking.”

The best way to learn is to struggle with a concept and then work through a problem to find a solution, said Zico Kolter(opens in new window), associate professor of computer science and director of the Machine Learning Department in SCS. 

If AI easily explains away the struggle, then students need to approach learning in new ways.

“This is a transformational technology that is going to fundamentally change the way we think about education,” Kolter said. “We need to find a way to teach effectively in a world where students have AI at their disposal to basically solve any homework problem.”

Learning best practices starts with faculty

Recognizing the need for faculty members to share best practices for teaching with generative AI, SCS Dean Martial Hebert(opens in new window) tasked Cortina and Kolter with convening a summit last summer.

“We weren't expecting to have any particular answers per se, but the idea was to share some of the techniques that faculty were trying,” Cortina said.

To open one session, Tom Mitchell(opens in new window), Founders University Professor in the Machine Learning Department, discussed the history of technological change and how quickly the proliferation and adoption of generative AI tools has affected day-to-day life.

“What I took away was tread with caution, but we have to tread — we have to move forward with this,” Cortina said. “What’s the right way to do this? We’re not sure, but we should be experimenting.”

Students have to know less about writing a lot of code from scratch, but they need to know how computer code works, he said. 

“If we as faculty believe that our content is still valuable, then we have to learn how to teach that in a way that is robust to AI,” Kolter said. “We've outsourced a lot of teaching to independent exercises that students do at home, and it's arguable that this is no longer as effective as it was.”

In SCS, designing courses to be more aligned with AI may mean programming homework assignments that include more tests and program code reviews, where teaching assistants will interview students about their projects so they can explain the process behind their creations.

“They will work at a higher level, doing things like planning and designing software,” Cortina said. “We have to be a bit more agile, in terms of adjusting assignments … in the end, it still needs to be the student’s program, even though AI has created some of the code.”

Beyond coding, students explore bigger ideas

Last spring, Mike Taylor(opens in new window), assistant teaching professor in the Computer Science Department, incorporated what he calls “algorithmic thinking” into his “Effective Coding With AI”(opens in new window) course by focusing on the process of problem-solving rather than the end result.

“The idea behind the course is to have it be a sandbox, or safe environment, with an explicit goal of answering together this question of what's the best way to use AI, while helping them build their portfolios and explore their interests,” he said. “At the end of the semester, we got a wider range of projects than I ever thought we would get.”

Taylor said he found that “the students who learned the most were the ones who refused to let the AI think for them.” After every assignment, he surveyed students — whose majors included computer science and electrical and computer engineering, but also statistics and data science, information systems, architecture and neuroscience — and compiled a list of best practices, which included advice like “verify everything,” “work incrementally” and “use multiple models deliberately.” 

Taylor, who submitted his findings in a paper for the spring Technical Symposium on Computer Science Education(opens in new window), said one of his goals was to weave ethics into a technical course.

After class discussions that sprung from science fiction reading assignments, Taylor said the students thoughtfully considered the ways technology shapes society, worked well together, and engaged in creative projects.

“They started to focus more on big issues, and took pride in the work they were doing,” he said. “My hope is that they start seeing programming less as one chance to meet someone else's expectations, and more as a chance to cross something off their list of ideas,” he said. “If it works, excellent; if it doesn’t work, let’s learn from it and still celebrate that process.”

CMU students are often ready to explore new concepts, Cortina said. Instead of dedicating time and problem-solving to coding, using AI allows them to focus on developing innovative ideas.

“They are open to going to more cutting-edge topics much earlier,” he said. “They're really interested in exploring, experimenting and looking for opportunities to create. I've always been amazed and proud to be here when I see the projects and research work that the students do.”

Across campus, faculty research learning with AI

CMU’s Eberly Center for Teaching Excellence and Educational Innovation(opens in new window) established the Generative Artificial Intelligence Teaching as Research(opens in new window) (GAITAR) Initiative in 2023 to promote instructor-led innovations and educational research designs, measuring the impacts on student learning. 

Chris McComb(opens in new window), professor of mechanical engineering and the director of the Human + AI Design Initiative(opens in new window) in the College of Engineering(opens in new window), was one of 27 CMU professors chosen as GAITAR Fellows(opens in new window), whose projects received a $5,000 grant and in-kind support from the Eberly Center to complete a yearlong research project examining if generative AI tools affected student learning and equity. The next round of fellowship applications will be accepted through March.

Chad Hershock(opens in new window), executive director of the Eberly Center, said the effects of generative AI on learning outcomes remain an open empirical question, especially the pedagogical details that drive positive or negative outcomes.

“Our goal is to put meaningful data in instructors’ hands to inform their decisions,” he said. “We’re letting the data tell the story and the fellowship projects like Chris’ are letting many data-informed teaching innovations bloom.”

For his research, McComb instructed students in his “Mechanics II: 3D Design” course to use a generative AI chatbot on eight homework assignments using pre-determined prompts to make deliberate mistakes for the students to identify. Instead of positioning the AI as an intelligent tutor, McComb structured the research(opens in new window) to make the students into tutors for the AI by incorporating specific mistakes to mimic a novice student. 

McComb found that students in the course significantly outperformed students from the previous semester in three of four concept areas.

“I honestly did not expect to see as large of an effect on their skill set,” he said, adding that, while beneficial, relying on AI for this type of one-on-one, easily accessible learning could erode community trust, such as when students reach out and rely on one another for help.

“When students engaged authentically, treating the AI agent like another student, they generally had pretty good experiences. On the other hand, students who didn't attempt or were incapable of engaging in that way, really just didn't get it and had more negative feedback,” McComb said. “For faculty, that goes two ways: What tools are you bringing into your class, and are they being explicitly put in a place where it's going to support student learning? These expectations should be communicated clearly.”

Hershock said, so far, the impact generative AI has on outcomes depends, in part, on whether AI is used as a production assistant, where it creates a deliverable and outsources human thinking, or if AI is used as a thought partner, where it only aids the student in critical thinking. A “transfer task,” which Hershock described as a subsequent task given to the student which must be completed without AI, helps researchers discover the amount of impact.

“Sometimes we find there's no difference compared to another teaching strategy that leverages the same evidence-based learning principles,” he said. “However, we’ve found learning is enhanced when generative AI creates a ‘thought partner’ learning opportunity that wouldn't exist otherwise.”

Teaching tactics come into focus

For his course, “Responsible AI,” Anand Rao(opens in new window), distinguished service professor of applied data science and AI in the Heinz College of Information Systems and Public Policy(opens in new window), encourages what he calls “meta-thinking,” considering first how to frame and deconstruct a problem, so that any AI assistance returns more focused, differentiated perspectives.

“Yes, it is shaped by the AI, but it should be shaped by both of you, like having a discussion with another person,” said Rao, who has served as co-chair of the Master of Science in Artificial Intelligence Management program(opens in new window) established in 2024. “Both of you are brainstorming, and, at the end of the day, you both come to a joint draft.”

For coding assignments in “Ethics, Safety and Social Impact in NLP and LLMs,” Maarten Sap(opens in new window), assistant professor with the Language Technologies Institute(opens in new window) and Human-Computer Interaction Institute(opens in new window) in SCS, has started giving intentionally vague instructions.

“Historically, our mindset with assignments has been to make them as clear-cut as possible, so students can't push back or haggle for their grade. But in the real world, there will be underspecification, and that's where human thinking can really come in,” he said. “You still have to decide how to operationalize your prompt. So a large language model might still be able to fill in the gaps and the details, but using it this way forces more thinking.”