Bridging academic worlds with virtual production: U-M’s XR stage expands global education

March 18, 2024

Yeah, I think we have Hollywood here in Ann Arbor.

So, behind me is an example of light frame wood construction, also known as stick framing, to build floors, walls and ceilings. Where openings are present…

We’re sitting in our new XR studio. XR is extended reality.

It’s an umbrella phrase that includes augmented virtual and mixed reality technologies. Over the last few years, primarily because of the TV show The Mandalorian, figured out how to use these large LED screens to project 3D environments into the screen.

And produce high quality video content without having to do a lot of visual effects work. Through the Center for Academic Innovation, faculty, innovators and student fellows are helping us shape the future of higher education and think about what’s next in teaching and learning.

We’ve created the XR studio here at the Center for Academic Innovation to help us continue pushing the boundaries of online and hybrid learning for students on campus and around the world.

With this space we’re going to focus on immersive learning.
We think this is the future.

This kind of studio, gives us an opportunity to pull students into different environments that would be very difficult to create.

Imagine inside of an operating room, a courtroom for law students in ancient Rome.

This allows us to help students understand complex environments.

Our center works with faculty to take an idea they have about a course and help design that for an open kind of global audience.

So it’s not like we just go into the classroom and film them lecturing to students.

Here, we sit down with our design teams and help design out what are the learning goals, learning objectives, How do we want to present this material in a way to make it accessible to a wider audience?

We have a whole media design team here that works with filming the faculty.

So instead of just kind of showing a kind of typical like presentation style, we can now immerse them in these environments.

These are high end cameras that allow us to both capture cinematic looks and feel, but also more traditional video broadcast.

This technology’s a step forward in making education accessible to everybody.

Imagine being a University of Michigan student or a learner anywhere in the world, logging on to an online course. Instead of seeing a talking head delivering a lecture, you see your instructor walking through ancient Cairo, in an operating room in Tokyo, or on a construction site in Rome.

When instructors step into the new extended reality stage at U-M’s Center for Academic Innovation, ​they can transport learners anywhere or any time in the world. The center is building online learning opportunities so students can engage and dive into an immersive virtual journey. ​There are no more limits to teaching and learning.

The XR studio can incorporate augmented, virtual and mixed reality into virtual production and was built upon the technological milestones achieved by the Disney+ series The Mandalorian. Producers figured out how to use large LED screens and cameras to render 3D environments where the cast is immersed in real-time. The technology produces high-quality video content and doesn’t require a lot of post-production work after filming is completed.

The center’s XR studio is one of six studios at its new facility in downtown Ann Arbor dedicated to developing accessible online learning experiences. While that is a long way from virtual production’s Hollywood origins, the new studios have the potential to change the way people teach and learn just as virtual production changed the film and TV industry. That transformative power comes from the combined expertise of U-M’s faculty and the center’s digital artists, media producers and learning experience designers.

Eric Schreffler, XR developer at the Center for Academic Innovation, works in the XR studio control room. Image credit: Jeremy Marble, Michigan News
Eric Schreffler, XR developer at the Center for Academic Innovation, works in the XR studio control room. Image credit: Jeremy Marble, Michigan News

“The studio opens a new chapter on scalable, personalized and immersive learning technologies, giving us an opportunity to pull students into different environments that would be very difficult to create on campus,” said James DeVaney, associate vice provost for academic innovation and founding executive director of the center.

“This allows our learners to practice in high-stakes environments while reducing risk. It helps students develop a sense of belonging in complex areas and solve complex problems in an increasingly uncertain world.”

Jonathan Rule, assistant professor at the U-M Taubman College of Architecture and Urban Planning, set foot on the new stage looking for a more engaging way to teach his classes in construction and design.

“My research has always been aligned toward studying things that would be beneficial for use in practice,” he said. “That might be how we use new technologies to shape teaching and practice. Then, once a student finishes school, he will be prepared to thrive in architecture.”

Rule leads “Augmented Tectonics,” a new project that leverages VR and AR and demystifies construction concepts currently presented through abstract drawings and images. Rule is exploring the use of XR to redefine the meaning of immersive and hands-on education for students and learners beyond campus.

“By creating an immersive environment, construction concepts become more tangible and supplement current learning methods through drawings and images,” he said. “Teaching on the XR stage, I can categorize, animate and highlight things. I can pick up augmented reality objects and move them around. You can actually pick up the virtual elements as if they were physical.”

And there is more.

Jonathan Rule, clinical assistant professor of architecture and urban planning, practices his course on the CAI XR studio. Image credit: Jeremy Marble, Michigan News

“I can pick up the intersection of two walls, for example, so one is perpendicular to the other,” Rule said. “It allows me to show viewers how those members come together to create a corner and where you can attach the finished material to it. Things that are complex layers are easily made visible through these types of technologies.

“This also begins to grab the attention and sparks the interest of a student or a viewer, as opposed to a static image. It’s interactive. It’s in motion.”

The new studio is set up like a broadcast facility that can support asynchronous and live multi-camera productions as well as cinematic-quality projects. In combination with augmented reality, camera tracking brings an unprecedented level of interactivity and engagement to course creation, making it a more engaging experience for learners.

“We have Hollywood here in Ann Arbor, and our faculty and student communities have access to this technology,” said Jeremy Nelson, senior director of XR, media design and production at the center. “This technology will shape the future and is a step forward in making education accessible to everybody.”

According to Nelson, some universities have similar studios to teach film students how to use this technology to create movies and broadcasts. But U-M’s new XR studio is the first in North America to use technology to support teaching and learning goals in online courses for all subjects and disciplines.

The center is experimenting with different types of environments from digital twins of the U-M Diag or the Ford Nuclear Reactor to tiny worlds like the inside of a DNA helix to fantastical worlds on other planets or deep in space.

Another add-on of the new stage is the ability to “teleport” another person from anywhere in the world.

“If a scholar is home, let’s say, in California and has a green screen behind him, we can beam him onto the stage in Ann Arbor,” Nelson said. “We are using the studio to collaborate and create courses in engineering, mathematics, science, dentistry, medicine and many other fields. We have a studio here to create online learning of the same caliber as ‘The Mandalorian.'”

Business Education

The center’s XR studio builds on the university’s first investment in digital education studios at Michigan’s Ross School of Business. Since 2019, students in the Online MBA and Executive Education programs have benefited from instruction delivered by faculty in the school’s studio. Featuring graphics technology similar to immersive video games, the studio creates a 3D environment where online students become part of a fully engaged virtual classroom community.

“Our XR studio expands our ability to connect with students around the globe, allowing learners with busy jobs and lives to still invest in their growth and development from afar,” said Lindy Greer, U-M professor of management and organizations. “The technology makes the classroom experience seamless, so faculty and students can interact fluidly and leverage the benefits that technology-fueled education can offer.”