by Sasha Kaurov
Many museums and planetariums are struggling under current social distancing policies and will likely continue to be affected for months, if not years. Thus, it might be the time for experiments with engaging audiences by other means. In this post I will discuss how the initiative I started with OmniScope, an organization to explore novel media for science communication, is testing virtual worlds as a tool for bringing the immersive and interactive experiences online to reach audiences around the world.
In contrast to other methods of engaging online audiences — like social media or video streaming services — the embodiment of a character in the virtual world enables STEM role modeling for young people otherwise lacking the opportunity to meet face-to-face with scientists. Additionally, with no need to travel physically, virtual museums and planetariums simplify logistics and consequently reduce the cost and environmental impact. Virtual reality also allows one to create experiences that are impossible to achieve in a regular lecture room, such as visiting outer space or the deep ocean. Thus, virtual worlds provide new ways to engage with and feel part of scientific discoveries.
In an effort to bring science to audiences currently stuck at home, we at OmniScope have developed a virtual planetarium show that allowed us to explore the possibilities of virtual worlds.
In what follows I briefly describe (1) what platform we chose and why, (2) the proof-of-concept that we made, and I will finish with (3) a call for collaboration.
1. What virtual worlds to use?
Even before the pandemic, trends in the game industry reveal the demand for virtual worlds. Furthermore, the prominence of social media is drawing the future of massive online virtual worlds toward social interactions.1 Even the game worlds start to diverge from pure gaming experience.2 In this environment it is natural to explore how all of these game-like interactions of embodied avatars can be utilized for science outreach.There is a large variety of existing virtual worlds that can be immediately adopted. Among those are multiplayer games like Minecraft or Fortnite. It might sound ridiculous, but why not create a planetarium show in Minecraft? The benefit of using an existing popular game as medium is that the players of those games would be immediately comfortable in the environment. The downside is that there will be a fraction of the population not willing to buy or install some additional software on their devices and learn the controls.
On the other side of the spectrum are purely social worlds like Second Life. In the past decade there were attempts to organize research institutions inside Second Life3 and conferences.4 However, the declining interest of academia in Second Life shows that maintaining your personal avatar and presence in yet another social network is too bothersome. Thus, we do not want the visitors to commit to a social network in order to enter our virtual museum.
We want to have the ability to customize the environment and minimize dependency on the platform. The development of the virtual world includes creation of custom assets (e.g. rooms, buildings, models of planets) and it requires investments. Thus, we certainly do not want to lose these investments if the platform closes or decides to dramatically change its policies
Finally, Virtual Reality headsets have become more accessible and widespread — the stand-alone headset like Oculus Quest now costs similar to a game console (under $400). Also, almost any modern phone supports a good Augmented Reality experience. This immersive technology is not a gimmick anymore, and it has the potential to dramatically enhance our lectures.
Thus, ideally we are looking for something that is:
Accessible. Does not require a gaming PC and can run on the majority of consumer devices.
Decoupled from social media. No need to register and “maintain” your social network profile.
Customizable. Ability to change the visuals and game logic.
For us the best platform is the open source project by Mozilla Hubs Foundation — Mozilla Hubs.5
- The open source nature of Hubs makes it easy to endorse in a blogpost. :)
- Frictionless use. Hubs are web driven, no installation required, everything runs in a web browser. People can use any device they own — laptops, computers, smartphone and tablets.
- Open source. It enables us to modify the virtual world by adding new interaction and mini-games. Also, it potentially allows for integration with Learning Management Systems if necessary.
- The cost of software is free. The server cost is tiny — on the order of cents per active online user per hour.
- Modern web technology allows for use of VR/AR in browsers. Thus, Hubs supports full VR presence and runs on less powerful stand-alone headsets.
2. Pilot: Standing on the surface of an asteroid
For our pilot show we chose JAXA’s Hayabusa2 mission that will return a sample from the asteroid Ryugu later this year. We created three 3D spaces and a couple of 3D assets specifically for the show in order to experiment what works best for the lectures in virtual worlds.
If you are an attendee the experience goes as follows. At a given time you follow the link to a website and you appear in a big meeting space. It is a real room in a real building at the Earth Life Science Institute (ELSI, Tokyo Institute of Technology), and it is recreated in detail to be realistic. You spend a few minutes by yourself learning how to move and pick up objects and wait for other participants to arrive. Some of them will be on their phones, others will be using VR headsets. You can greet them and make small talk. Then, the lecturer introduces herself and starts the lecture. She might pull a couple of 3D models into the space and talk about them. For instance, she can show the instruments on the model of the Hayabusa2 spacecraft. The model is 1:1 scale in this world and the surrounding common objects in the room help to feel the scale. You can make a copy of the spacecraft and look at it by yourself. If you have a VR headset, you can hold and rotate the model spacecraft in your hands.
After 5-10 minutes everyone teleports to a fairytale palace after which the asteroid is named. You spend a few minutes exploring the new environment and learning how it was named. The sound from the lecturer is spatial, meaning that you need to be closer to hear better. You can approach your friends and whisper something to them.
Finally, to contrast the previous scene, we teleport to the surface of the asteroid after another 5-10 minutes. Its landscape is recreated from real pictures taken by the spacecraft. We can fly around it and look at some media together, for instance, the video taken by one of the rovers. The lecturer can take a pen and show our location on the miniature version of the asteroid.
Altogether the show takes about 30 minutes. Here are a couple of tweets containing example videos that demonstrate the experience:
We had a great time testing our Virtual Reality Planetarium show about #JAXA Hayabusa2 mission (@haya2e_jaxa) using @MozillaHubs inside a virtual ELSI (@ELSI_origins)#VR #onlineeducation pic.twitter.com/0JNNrZxVZ3— OmniScope (@OmniScope_org) April 13, 2020
Today, we tried out a virtual reality talk on JAXA’s @haya2e_jaxa mission using fantastic scenes designed by @OmniScope_org that allowed us to handle the spacecraft, explore the underwater dragon palace and stand on the surface of asteroid Ryugu. High-res: https://t.co/GHfh3jJ3y7 pic.twitter.com/ZgE0Oe7vZW— Elizabeth Tasker (@girlandkat) April 28, 2020
It is fun not only for the attendees. The lecturer can also be in a VR headset and use their hands to gesture and point to things. Looking at the avatars of others one can determine who listens, and who got distracted by a tree. You can read more about our experience on JAXA website or Many Worlds blog (articles by Elizabeth Tasker).
You can also go and look at these 3 worlds right now by following this link: https://www.omniscope.org/outreach
3. Let’s do it together!
Currently we are developing a few more scenes and shows, primarily focused on space. However, the virtual world environment can be certainly adopted for outreach in all branches of science. We would be happy to share our experience in more detail and provide our expertise. Thus, if you have an idea of a show that can work in virtual worlds and in VR, or if you just want to talk more about it, let’s get in contact and discuss privately or on our discord channel (invite link). Follow our experiments in virtual worlds on Twitter @OmniScope_org!
References and Resources
1 Introducing 'Facebook Horizon'
2 Fortnite’s new experimental mode is about partying, not fighting
3 Djorgovski SG, Hut P, McMillan S, Vesperini E, Knop R, Farr W, et al. Exploring the Use of Virtual Worlds as a Scientific Research Platform: The Meta-Institute for Computational Astrophysics (MICA). In: Lehmann-Grube F, Sablatnig J, editors. Facets of Virtual Environments. Berlin, Heidelberg: Springer Berlin Heidelberg; 2010. pp. 29–43.
5 There are many other platforms worth mentioning that might be a better choice for a different set of priorities. The incomplete list includes AltspaceVR, RecRoom, VRChat, framevr.io.
Sasha (Alexander) Kaurov
Ph.D. Astronomy & Astrophysics
Postdoctoral IBM Einstein fellow at the Institute of Advanced Study
Alexander Kaurov’s research interests range from the physics of neutron stars to the epoch of reionization. At IAS, he is working on developing theoretical models of reionization and investigating techniques for analyzing the data from the upcoming probes of the early universe.