NASA flight surgeon Josef Schmid holoported onto the ISS. (ESA/Thomas Pesquet)
There’s never been a house call quite like this. In a first for telepresence communication, a NASA flight surgeon was ‘holoported’ to the International Space Station (ISS), appearing and conversing as a virtual presence in real time, hundreds of miles above the surface of Earth.
If it sounds like Star Trek, you’re not too far off. (after all, Star Trek: Voyager did feature an artificial physician who was a holographic projection.)
But this isn’t science fiction. When NASA flight surgeon Josef Schmid was beamed up to the ISS in October of last year, the illusion was made possible thanks to Microsoft’s ‘holoportation’ technology, which lets users interact with 3D representations of remote participants in real time.
“This is [a] completely new manner of human communication across vast distances,” says Schmid. “It is a brand-new way of human exploration, where our human entity is able to travel off the planet.”
Schmid and other team members during the holoportation session. (ESA/Thomas Pesquet)
Unlike traditional holographic projections that appear to hover in the air for anybody to see, holoportation requires the use of an augmented reality headset, such as Microsoft’s HoloLens technology, for the wearer to be able to perceive (and interact with) the remotely captured individual(s), who are filmed with a multiple-camera setup in their actual location.
In this case, European Space Agency (ESA) astronaut Thomas Pesquet, who was on board the ISS and wearing such a headset, had a two-way conversation with Schmid and members of his medical team, along with Fernando De La Pena Llaca, the CEO of AEXA Aerospace, which develops custom holoportation software (the kind that made this ISS session possible).
While Microsoft’s holoportation technology has existed – in various stages of development – for several years, it’s never been used for something as ambitious as this before: connecting Earth-based medical researchers with astronauts on mission, orbiting the planet hundreds of miles up in the sky.
Yet it’s this exact kind of capability – bridging physical gaps to connect people over huge distances in space – that could be important for future space-exploration missions. This way, scientists could virtually interact with real-time 3D representations of remote participants on Earth, space stations, or other spacecraft, enabling collaborations that can be much more involving and immersive than standard 2D video calls.
“Our physical body is not there, but our human entity absolutely is there,” says Schmid.
“Imagine you can bring the best instructor or the actual designer of a particularly complex technology right beside you wherever you might be working on it.”
NASA flight surgeon Josef Schmid holoported onto the ISS. (ESA/Thomas Pesquet)
The next step in the technology’s evolution is to enable fully two-way holoportation interactions.
During this experiment, Pesquet was the only participant wearing an augmented reality headset that enabled him to perceive the other participants as digital 3D holograms, as Schmid and the other participants did not wear such devices themselves.
Once all participants are similarly equipped, however, the possibilities to jump into someone else’s reality could become even more instructive and transformative for off-world astronauts – whether you’re consulting Earth-bound doctors about a medical issue, or exchanging important ideas about mission objectives with NASA researchers.
“What it really plays into is opportunities for more longer duration spaceflight and more deep spaceflight,” Christian Maender, a research director at space infrastructure company Axiom Space, explained to the Verge in 2021.
“Where you are really talking about wanting to create a human connection between your crew – no matter where they’re traveling – and back to someone on the planet.”