Author: Margarida Lira Gomes De Almeida Belo

Video

It’s a little bit shaky since it was recorded in Unreal’s VR viewport. It showcases the entire experience nonetheless.

WARNING

ATTENTION: When playing the project, sometimes, the player’s height is a little off, transporting the player to a lower or higher spot than where it should be. In case this happens, please refer to the project file and adjust player height as necessary. I am still trying to figure out why it’s happening. It hasn’t happened with the built file, so hopefully, no one will encounter this issue during the assessment period.

Critical Appraisal

Creating Scenes’ Layout

Lobby

For the first scene, my goal was to create a space station or lobby, where the player could admire the wonders of space while also serving as a transition scene between all the major levels. So it had to have some sort of impact on the player. So it served as a great starting point and was my first prototype.

For the basic layout and assets of this scene, I used two very important sources. A YouTube tutorial to create the planet and sky, and a ue5 package I found that had a space station I used in my scene.

Solar System

For the solar system, I wanted to create a standard solar system that looks like the one in my concept drawing. To achieve this, I once again found a very helpful tutorial on YouTube that guided me not only through the creation of each planet material, but it also gave me the real measurements, rotation and scale of all planets in unreal units. Of course, the distance between the planets and the sun is not at the same scale, otherwise it would be impossible to showcase them together.

Mars

For the Mars Scene, I also followed a tutorial and downloaded another package to use for the Mars terrain. I had to adapt it a bit though because the tutorial focuses more on a cinematic sequence. I mostly just used it to find out about the unreal package, which apparently had 90Gb… I just used the assets I needed and then deleted it.

For the sky, I adapted it to look like a Mars sunset.

Black hole

For the Black hole scene, I also followed an extremely complex tutorial in order to create a black hole texture that felt realistic and immersive.

Earth

Finally came the Earth scene. As the closing scene of the experience, I needed it to feel somewhat nostalgic and with an overwhelming sense of smallness when facing the huge infinity of our universe. So I created a huge grassy field under the night sky, where the viewer could wrap up the experience by having a more realistic perspective of the space, and once again feel “back on earth”, as if they took off the headset. To make it visible, I added a small lamp to serve as lighting for the scene.

Lamp asset link: https://sketchfab.com/3d-models/gas-lamp-0a5013605cc848e89ae5b2e3396504da

Grass: UE5 package called “City Environment Collection”

Desk and chair

As I’ve mentioned before, in the final experience the player will be sitting down at all times. And my goal was to have a desk reminiscent of school environments. To achieve this, I used the assets from the lobby package I’d downloaded. I simply used a pre-made blueprint, deleted all the elements I didn’t need, and just left the desk and the chair.

Afterwards, I added it to all the scenes along with the player start pawn, adjusting it to the view I wanted and the player’s position and proportion in accordance with the chair and desk’s height.

However, for the earth scene I used some quixel assets to create a more realistic desk and chair that matched our planet.

Robot

For my robot character, I wanted something that seemed somewhat human (humanoid robot), to really achieve that human-like motion and feeling I chose to portray. After a long time looking for the ideal asset, I came across the following character:

I really enjoyed the overall look this character has. It’s smooth and modern without looking aggressive, like most other models I’d found. The only change I made to it was the material which I changed to metallic, since it matched the darker colours better. I also didn’t include the shiny bit as it kept giving me problems by being a separate element in the animation sequences. So the robot simply became grey and metallic.

After downloading it, I rigged it using mixamo.

Collaborations

Collab 1 – Lecture narration and Sound (and Incorporation)

For my first collaboration, I worked alongside Vlad from BA Sound Arts year 3. He’s collaborated with our course before. Last year, he was with Miranda’s group and they produced some amazing work. This year Miranda recommended him to me so I reached out through whatsapp and our collaboration began.

I am so happy to say that he is a great person to work with! Very proactive and produces great material. Always very self-reflective and looking to improve in any way he can! I couldn’t have asked for a better work partner!

During our work, we shared files and ideas through both Whatsapp and OneDrive, where we uploaded all the videos, scripts and audio files. This made the entire process smooth and hassle-free.

In terms of robot content, he recorded all the scripts I’d sent him, including the voice-over for all the national geographic videos, which he recorded according to the timing of the original ones so that the robot’s acting could match them. It was perfect!

In the end, he produced the narration for the robot Teacher I used in my project. And once again I am delighted to say that his work was great and fit my project incredibly well.

Sound Incorporation

There isn’t much to say about the sound incorporation in Unreal. It’s a very easy process that doesn’t take more than 10 minutes. Overall very smooth step of this project.

Collab 2 – Motion Capture (and incorporation)

For my second collaboration, I worked alongside Kabir from BA Acting and Performance year 2. Kabir is also a great worker, very dedicated and talented! Once I mentioned to him we’d be doing motion capture, he became delighted and excited to work with such a relevant medium to today’s media production.

Our goal here was for him to perform as the robot teacher throughout all the scenes, by acting out the movements according to the audio Vlad had sent me. As a result, there wasn’t much preparation beforehand since Kabir didn’t have to memorise any lines. It was pure improvisation, with help from my directing and notes.

We ended up producing 5 takes, meaning one per scene. And I was extremely happy with the results as well. The movement was fluid and he was able to match the audio and body language to the audio seamlessly.

With the help of Kay (Digital Arts Specialist Technician), I was able to have the files exported by him through Rokoko. He sent me everything through teams and was willing to help me with any problems I might’ve had.

Animation Incorporation

Step 1 – Blender

The incorporation of all the animation clips was a long and complex process that took trial and error when it came to understanding how it worked. Step one was to add the animation to the rigged character in Blender by using the Rokoko plugin. Afterwards, I exported it as an FBX file. This was fairly easy and I didn’t encounter too many issues besides the exporting time and which exporting settings I should use.

Step 2 – Import Into Unreal

This is where I encountered my first issue. I will not get too much into it because it was honestly very stupid and I don’t know why it’s even a problem. Basically, every time I tried importing an FBX file, it gave me an error and I couldn’t do anything. I tried re-exporting many fbx files from Blender, checked the settings, and watched tons of videos and nothing… Until one video where the guy was exporting and creating a folder with the character skeleton and before importing an fbx file he clicked on the skeleton and only afterwards did he click on importing a file. I did it, and it worked… ._.

It was such a hassle at the time and I didn’t know what to do… I even contacted Kay about it. But eventually, I sorted everything out.

Step 3 – Level sequence

The next step was to create a level sequence and add the animated character to adjust the actions to the sound and to make sure the animation started and ended where I wanted it to. Afterwards, I exported it as an animation sequence file.

Step 4 – Animation Blueprints (start Issue)

The last step was to create an animation blueprint so that it would not only play but also loop the last sequence I told it to. This was so that after the dialogue, the robot would have a repetitive, NPC-like motion forever.

And this is where I also found a major issue. I had no idea how to tell it to only play the animation when I told it to, instead of playing it right away when I played the level. This was an especially big problem for the lobby scene because once the viewer put on the headset, there would be no warning or pause, nothing. It would play audio and animation right away. And I wanted to have the first menu panel before the robot teacher even appeared.

I’d had enough of Unreal blueprints and trying to figure out how to do even the most basic of things. And I did try to find solutions to this problem. I tried making it so the character only showed up after a button was pressed, which didn’t work because it was a different type of blueprint (animation blueprint).

So I gave up and came up with another solution. A new scene. I added a new scene before the lobby and called it “Menu Scene”, which takes place in outer space with the desk simply floating in space. This way, I could have the menu panel and a peaceful environment before the animation and narration started. This was a major change since it added an entirely new scene to my project. However, it was a basic layout that simply served as a transition to the experience.

Designing interactions and mechanics

Menu

Note: This Scene has no interactions besides the UI.

The Menu scene has no interaction or any mechanics. It is simply a transition scene that features the menu panel with controller instructions and an “About” page.

Lobby

Note: This Scene has no interactions besides the UI.

This scene is like the previous one. There are no special interactions or mechanics. It features the main panel of the experience, which lets the viewer pick which lecture they want to learn from or exit to the final scene.

This scene had initially an issue with lagging, which was fixed by removing the animation from the comet ring in the level.

UI

The UI was also very basic, simply serving to move from one level to the other

Solar System

This scene was the most complex to program and to set the correct timing. The goal was to have the viewer watch the lecture in a screen which then disappears, activating a menu which lets the player explore details about the sun and the planets by setting off animations that move the planets up close. The experience ends when the viewer presses the “Finish” button.

Mechanics

• Pop-off screen mechanic – a timer is set to deactivate the lecture screen once it’s over
• Play level sequence mechanic – Once a planet/sun button is pressed, the menu disappears, it plays the animation for the selected element and activates the “Go Back” button and information panels.
• Play in reverse mechanic – Once the “Go Back” button is pressed, the animation plays in reverse, the button disappears and the menu is reactivated
• Finish mechanic – Once the “Finish” button is pressed, the player goes back to the lobby

Issues

Issue 1 – I had a major issue initially because my plan was to have the player teleported to a location in front of each planet. But after many tries and prototypes, I realised it was more complicated than I needed it to be. So instead, I had the planets come to the player with simple animation sequences triggered by the UI.

Issue 2 – Initially instead of UI, I wanted to have a 3D menu with the planets/sun. Upon cooling with one of them and pressing the trigger button would trigger the animation just like the UI menu does. But again, this proved to be overly complicated so I replaced it with UI. Here is a video of that prototype:

Mars

This level was fairly easy to code. The goal was to have the viewer watch the lecture on a screen which then disappears, activating a menu which lets the player explore details about NASA’s Curiosity Rover or witness a hypothetical scenario of a human colony on Mars and learn about them up close. The experience ends when the viewer presses the “Finish” button.

Mechanics

• Pop-off screen mechanic – a timer is set to deactivate the lecture screen once it’s over and activates the Mars menu
• rover mechanic – on “rover” button pressed, the rover and rover’s UI appear (if the colony was activated first, it now disappears)
• Colony mechanic – on “colony” button pressed, the colony and colony’s UI appear (if the rover was activated first, it now disappears)
• Finish mechanic – Once the “Finish” button is pressed, the player goes back to the lobby

Issues

I had only one issue with this scene. And That was every time I assigned the hologram material to the colony mesh, it completely deformed it. The solution was to remove the transparency property from the material and have it be opaque instead.

Black hole

This level was again, more complex since it demanded visual distortion effects and assets moving towards the player. The goal was to have the viewer watch the lecture on a screen which then disappears, activating a button that once pressed lets the player experience being sucked into a black hole, and learn about its real-life effects. The experience ends when the viewer presses the “Finish” button.

Mechanics

• Pop-off screen mechanic – a timer is set to deactivate the lecture screen once it’s over and activates the Mars menu
• Go button mechanic – once “Go” button is pressed, a sequence animation of a post-process volume with distortion is played (changing its weight gradually), giving the illusion of physical distortion and movement towards the black holes. At the same time, black hole animation plays and the black hole moves closer to the player. Finally, it also triggers a fade-to-black sequence that reaches its max once the player is inside the black hole. The info UI is activated at the end.
• Finish mechanic – Once the “Finish” button is pressed, the player goes back to the lobby

Issues

The only issue here was similar to the one in the solar system scene. I wanted to move the player towards the blackhole, but it was easier to move the black hole towards the player along with other elements moving away from the player giving the illusion that the player is the one moving. So that’s what I did instead.

Earth

Note: This Scene has no interactions and no UI.

This level was the most basic one, with nothing but a timer and fading to black mechanics, which were set to match the end of the narration and mark the end of the experience.

UI/UX Design and Incorporation

Unfortunately, after looking for UI students who were willing to collaborate with me, I couldn’t find anyone that was willing to give me a hand. I even went with Abhay to their class but neither of us was able to find what we were looking for. And with not that much time left, I had to figure it out myself. So during break, I researched and designed all the UI by myself. To accomplish this, I used Adobe Illustrator to get the best resolution possible and to make it as precise as I could.

In terms of design, I’ve mentioned my inspiration in the previous page, but overall I was trying to go for a modern look reminiscent of Apple UI and the meta’s video UI as well. I worked with darker tones as I feel like it was easier on the eyes, making the entire experience more comfortable and less overwhelming. Plus it just looks more coherent with the darkness of the entire project.

As I started adding the UI to Unreal, I found two big issues.

1 – The resolution was bad, but it was an easy fix. I simply had to make the UI huge and scale it down on the scene.

2 – As I was working with pngs it always had a few resolution imperfections, so I ended up redoing all the UI within Unreal, based on my original illustrator designs.

After fixing these, I simply scaled them down as I saw fit, and transferred all the blueprint functions from the widget prototypes to the final designs. And then I did the same thing for the blueprints that governed each widget. It was a long repetitive process but it presented no issues whatsoever.

And I must say I am extremely happy with the final results that I got. Everything looks crisp and modern, just as I initially planned.

Playtests and feedback

Abhay

My first volunteer for the playtesting sessions was Abhay. Overall he really enjoyed the experience, especially the Mars scene and being able to look at the Mars rover up close. However, he had a few notes for me:

• Fixing a repeated title in one of the UI buttons in the Mars scene
• He thinks the screens should be in the middle rather than on the side
• the menu from the solar system scene should be more vertical instead of longitudinal
• Overall, “Very cool”

I think he felt a bit bored watching the lectures though, so I have to consider adding some sort of interaction for the viewer, while each lecture is playing.

Zedan

Zedan came next and was able to give me even more complex feedback and comment on the experience’s effectiveness at standing out as an educational tool that can extended the physical limits of current educational practices. She thought it accomplished what I set it out to do and that the graphics along with the interactions, make for an extremely immersive and complementing asset for education.

As a space enthusiast herself, she thoroughly enjoyed exploring all the assets and looking at them up close, as well as watching each lecture and absorbing its contents. She gave me a few notes:

• Overall, looks amazing and accomplishes its purpose
• Solar System interaction was her favourite but she wishes the planets would spin so she could explore them further
• Slower rover rotation in the Mars Scene
• She enjoyed the lectures and didn’t think they were too long

Herman

Finally, Herman with some of the most relevant feedback. I think he enjoyed the experience in general. But he had a lot of notes to give me, and details I could work on, especially for the exhibition.

But I think the one that stood out the most to me was to have some sort of interaction during each lecture since he focuses better if he has something to do while learning anything. I fully understand this as in my case I tend to draw in order to pay attention or fidget. So I seriously am going to consider this change, even before the submission.

Here is some more feedback:

Ambient Sound

Unfortunately, I haven’t received all the ambient sound files from Vlad. As a result, I’ve decided to go ahead and find some sound effects on my own. If he sends those files in time for me to switch with the ones I found, I will reference them in my blog.

Since this wasn’t originally going to be my task, I didn’t have too much time to think about each level in detail. I simply added the sound I though was absolutely essential to making the experience better, and that Herman had included in his feedback list.

Overall, for open space scenes, I mostly consist of white noise since there isn’t supposed to be sound in space. For Mars some wind recordings I found from NASA. For the UI, I just added a button-clicking sound to all the widgets. For the solar system planets animation, a swoosh approaching sound. Finally, for the Erath Scene, some nature noises with crickets and wind.

However, I plan to develop these further for the exhibition, and hopefully, I will have more of Vlad’s work in it.

I created a list of what sound effects I wanted to have in each level:

• Menu Scene: click for UI buttons, white noise

• Spaceship Scene: click for UI buttons, white noise

• Solar System Scene: click for UI buttons, white noise, swoosh for planets approaching player

• Mars Scene: click for UI buttons, Mars wind sound,

• Black hole Scene: click for UI buttons, white noise, toy gun shooting sound

• Earth Scene: Night field sounds with wind and crickets

Links:

Changes after feedback

Beside the sound, I made some other changes after I got feedback from Herman and my classmates. Minor changes were:

• Slow down Mars rover rotation,

• Include a separate controller page on the menu scene so the player learns how to click the UI buttons before using the menu

• Removed snap rotation

Major change

The biggest change I made after the feedback originated from both the feedback I got from Herman and from watching Abhay playtest my project.

Firstly, I noticed that when Abhay was watching the lectures, he couldn’t really sit still and just pay attention to the lecture, even though it was just 3 minutes long. It felt like something was missing, something that he could do while the lecture was happening. This was the first time I thought something could be added to make it more interactive and less like a normal lecture.

Afterwards, I got the exact same feedback from Herman. He told me that he’s the type of person who likes to fidget while he’s learning anything. Furthermore, he pointed out that a lot of people are like this as well. Especially when it comes to people with attention deficits and ADHD, taking this into account would be a must. Finally, he also mentioned the fact that VR is a great opportunity for people to get up close to virtual elements and get more hands-on experiences. And it is true that a project about space is a great opportunity to make this happen.

Herman said I could add some interactions for the exhibition period. However, I decided I wanted to add them before submission as I think it would make a significant difference in how people experience my project, and I completely agreed with the feedback he gave me. It just makes the experience that much better and more interactive during the lecture bits.

Consequently, I added one small interaction to each of the three lecture levels:

Solar System

For the first interaction, I decided to have a little challenge where the viewer needs to place the sun and planets in the correct order. They can grab them, look up close, play with them, throw them, etc.

Mars

For the Mars Scene, the viewer can loom at all of NASA’s rovers and hold them to see their intricacies and specifications as well as learn when they landed on the red planet.

Note: The rover assets can be found on the NASA website. Except for the first one (https://skfb.ly/6YVJx).

Black hole

Finally, for this interaction, I decided to have a mini black hole and a gun for the viewer to shoot with. I made it so that the black hole destroys the bullets on impact so it looks like they have been sucked into it.

I am really happy I added these to my final experience before submission. They really enhanced the viewer’s experience and it doesn’t limit them to sitting down and listening.

Things to Improve for the Exhibition

Despite almost everything having worked the way that I originally planned, there are a couple of things I could improve for the physical exhibition.

Number 1 would be the sound. Unfortunately, I couldn’t get all the soundtracks from my partner in time for the submission so I had to improvise it a bit. However, for the exhibition, I plan to have it all arranged and ready for viewers to experience in a much more immersive way than it is now. The sound is a crucial part of immersiveness so I want to make sure I nail it as well.

Number 2 would be the interactions themselves and the feedback the viewer gets from them. Such as the mini black hole and the way it draws the bullets in, or having particle systems in the mini rovers when they collide with each other. Also having the solar system mini interaction play a small victory song when the player gets the order of the planets right.

Finally, I would possibly fix the limb collision in the robot animation, even tho it doesn’t affect the final experience as much.

And there is also a small issue with the player pawn location sometimes not working correctly. I haven’t had issues with it in the built file, but I have to make sure it doesn’t break during the exhibition.

Note: This blog is organised chronologically.

Intro

For my final major project, I intend to create a VR experience within the realms of cinematography. I want to cover this topic while also tackling it within the context of educational environments and how it could prove beneficial or not for students, as a way of complementing their overall content retention and today’s incorporation of education with emerging technology.

Furthermore, it is true that cinematography tends to leave a huge impact on people, and some of the hobbies or interests we pursue are greatly impacted by media, more specifically movies and shows. Just imagine how this potential could be used in VR and how much more it could expand the way we experience that media.

But for now, I think I will focus more on this question in regards to my dissertation. Since the first submission date isn’t until November, that gives me plenty of time to figure out what I could do.

Concept

Through my FMP, I intend to explore the idea of cinematography as a valuable tool for educational settings and whether or not it has an impact on how students (and learners in general) retain information based on visual data, cues and psychology of film-making strategies.

Here are some examples of useful papers I found:

“VR has shown tremendous potential in educating and rehabilitating children (ages 18 and below) with
neurodevelopmental disorders. A study by Wang and Reid (2011) explores the use of this technology, examining
multiple case studies that have tested the efficacy of VR on participants ranging in age, and gender
disorder. These hypotheses were tested for Attention Deficit Hyperactivity Disorder, Autism and Cerebral Palsy.
As a goal, this comprehensive analysis intended to further explore the potential of VR in addressing
neurological impairments in children and contribute to a better understating of its future implications in VR
education.”

This paper was a valuable resource in evaluating education within different neurodevelopmental spectrums. As
a result, it provided me with a crucial perspective on VR education that accounts for other perspectives aside
from those of neurotypical folk. This could help me develop a more accessible experience across a spectrum of
students.

Links: https://karger.com/ned/article/36/1/2/226406/Virtual-Reality-in-Pediatric-Neurorehabilitation

“When addressing filmmaking, the impact of cinematographic practices can vary considerably based on the
medium used to guide the spectator’s gaze. For one, in VR, since the viewer has complete agency over where to
look throughout the experience, cinematographic principles cannot be applied the same way as in 2D screens.
Although these practices may remain the same in terms of taxonomy, the methods used to apply them as a
way of diverting attention within Cinematic Virtual Reality (CVR) change completely. This is due to the viewer’s
agency and the use of 3D immersive environments that enforce the existence of another spatial dimension,
making it a physical experience. This paired with the existence of multiple points of interest (POI) influences the
viewer’s ability to fully absorb the contents of immersive cinematic pieces. Therefore, the focus lies on finding
the methods that allow the creators to direct focus. Further research by Rothe, Buschek and Hußmann(2019),
demonstrates some of the methods proposed, with novel approaches that address the processes within CVR.”

Links: https://www.mdpi.com/2414-4088/3/1/19

For a better analysis of these ideas, please refer to my thesis posted near the end of this page, where I explore these ideas further and create a more supported basis for my approaches throughout this project.

Update 1

I have been reflecting on what kind of cinematic experience I want to have for my project. I think it is logical that it covers some type of school subject since my goal is to make it an educational experience for regular students (or anyone else who is curious and wants to give it a go).

However, I feel like lately I’ve been having somewhat of an artistic block and I can’t think of what I should make. I’ve been brainstorming and I’ve come up with some potential ideas:

• Language experience
• A historical trip through time eras
• Space exploration
• Geographical exploration (potentially using Google Maps in unreal could be cool)
• Anatomy experience

Sadly, I don’t feel strongly about any of those ideas yet and it seems like I don’t really know what I should do… I’ve talked to Eric about this, and he said I should take my time thinking about this and look for inspiration. Hopefully, at some point, I’ll start feeling more drawn towards something.

Side Story

One week ago, a few girls from 3D Animation came to our class looking for people to collaborate with, from BA VR. They presented their idea, which consisted of a cinematic experience in a non-sensical world inspired by Alice in Wonderland. I immediately became extremely excite with the thought of being able to collaborate with them to create something that was still more or less coherent to my plans. But honestly, I think I rushed it a little bit since I was panicking with the fact that I couldn’t find an idea that pleased me at the moment. So I ended up joining their group and changed my original plan.

However, we had a meeting two days ago. This made me instantly realise that it would be a mistake for 3 reasons:

• They had complete control of the project (because it was theirs to begin with)
• I felt like I was getting sidetracked from my original plan
• I started feeling like I did it out of despair and not really for the idea

So, unfortunately, I had to message them saying I wanted to leave the group. And now I am back to square 1. They were really nice and their ideas were great. I simply cannot afford to jeopardise mine, for the sake of feeling comfortable because of their people’s ideas. But I feel like I should still write about it to show that sometimes things don’t go according to plan and we need to stay confident in our ideas and convictions.

Update 2

I have been getting back on track and looking for inspiration, so I can start moving in one direction. And actually, there is one video that really caught my attention! Meta released it in 2021 and it shows audiences a hypothetical depiction of what MR, AR, XR and VR could look like and how immersive they could be when showing people the wonders of space. I thought it was a really smart choice because it’s nearly impossible for the average human being to experience any sort of space-related experiences, and this might be the only way (for now), that we can bring that reality a bit closer to people, with the help of immersive technologies.

When it comes to other subjects such as history or geography, I can visit physical sites and I’ll be “teleported” to that reality because there are still traces of it (not for everything though). What I mean is, it is possible to immerse myself in those subjects through the real world. But not for space. That’s what made the video so enticing!

I feel like this is finally the direction I want to pursue! A VR experience about space, with the goal of not only educating people but also bringing them closer to such a distant and exclusive experience such as exploring space. Now I need to start developing the concept more and figure what the details of the experience. But at least I am finally feeling good about my FMP idea.

Weeks 3 to 6 – Developing ideas and Prototyping

(Note: I am writing this on week 6 so I already have a very solid idea, drawings, prototypes, etc. So I will kind of just skip to the main idea that I arrived at in the end, the feedback from Eric and Herman and my material)

Throughout these 3 weeks, I started to develop my ideas in a more solid and structured manner. My goal was to captivate the viewer with spatial scenarios and bring them closer to it through mesmerising graphics and interesting facts. So I chose to do an experience with three different themes, to bring variety not only to the visuals but also to the content provided to the viewer.

The 3 scenes will be about:

• Mars
• The Solar System
• Blackholes

I thought these would be a great choice since at least two of them are familiar enough to most people that it’s still recognisable and easy to enjoy while still looking amazing. Whereas the black holes scene would be more of an open-ended question since there is still so much to know about them and how they influence our universe. It would end the experience in an open-ended way and leave the viewer expecting more, and hopefully, feeling inspired to want to learn. Like we sometimes feel after watching a great movie (hence the power of cinematography.

I would still want to have some sort of human connection throughout the experience, so I think having someone guiding the viewer throughout the scenes would be a great way to have them connect in a human way as well, which would further captivate them since they’d be able to establish a connection with a mere virtual character.

For the visuals that precede the 3 main scenes, I want to have two other levels. One that will serve as a level selection lobby, where the viewer can choose which lecture to see first. And another one before which takes place “in our actual world”. What I mean by this is a scene that happens in a regular realistic scenario that everyone experiences, school.

Therefore, the experience will start by subverting people’s expectations into thinking they will have a normal school lesson in a boring-looking classroom, with a normal teacher. Which kind of defeats the purpose of VR. Or at least that’s what I want them to think at first. After that scene, the viewer will be transported into the space lobby and the real experience can start.

Also, it is supposed to be a cinematic experience where I want the viewer to be impacted by the visuals, so I don’t want to sacrifice their ability to enjoy high graphics in VR by having too much movement. That would also jeopardise the experience since I’ll be using Unreal which will really push this project to its performance limits. (I’ll talk more about the software below). So the entire experience will have the viewer sitting down in front of a desk (which makes the transition from the classroom environment to space, a lot smoother, since they will both have desks).

Update – Robot teacher

Why stop at the transition between classroom and space? What if the human teacher became a humanoid robot? In my opinion, it would once again impact the viewer once they made the transition from earth to space. No one would expect it, and there would still be a character creating human connection, but in this context, it would shift to something more futuristic. And the fact that this robot teacher wouldn’t exist in real life (at least for now), makes it more exclusive and memorable to the viewer.

I talked about this idea with Eric and he approved it. However, I think he sees it more like a small robot or more peet-like. In my view, it should be humanoid so as to not lose the human side that I want it to have. Besides, it would be much easier to animate with motion capture than moving a robot around, which could give me an unrealistic load of work.

Concept drawing

Initial storyboard

Quick explanation – It starts with the viewer in an ordinary classroom being taught by a human teacher who introduces the theme of the class (astronomy). It then transitions to the space lobby that replaces the classroom and a robot that replaces the human teacher. They give the viewer 3 options to explore: the solar system, mars or black holes. Upon choosing one option, the player is transported to each respective lecture where they can feel mesmerised by the realistic and immersive environments that correspond to each theme. They have an audio lecture and a few interactions at each level that will further teach them about astronomy. At the end of each lecture, they must return to the lobby. Finally, once satisfied the viewer can click on the finish button in the lobby scene and they will be transported “back” to Earth, where the robot does an extremely sentimental and touching monologue about the wonders of space. They then invite the viewer to lie down on the grass with them while gazing at the stars, as it all fades to black.

Lectures

I want to emphasise this again. Besides being a cinematic experience, this project is directed towards education, so I want to incorporate great visuals with interesting, factual and thought-provoking information.

As a result, I want each level to have a small lecture (which will be given by the robot teacher obviously), where the viewer will get to enjoy the environment, as well as learn more about it.

But I feel that simply audio could be ignored so I want there to be visuals, images, signs, slide perhaps, alongside with the audio and the robot. I think that would complement it very well.

Initially, I thought that I could create these lectures with someone specialising in astronomy, but due to time constraints and overcomplicating my work process, I will be using AI to generate the lectures. This could also be useful when writing my script.

Update – Interaction and UI

I feel like this was always on the back of my head, but I think it’s important to fully commit to this idea, as I develop and explain my project. I think the incorporation of good UI design is crucial for this project, as I will be tackling a type of encounter that is mostly based on user experience and comfortability. Therefore, a clear and modern UI seems like a no-brainer.

This will certainly help the viewer smoothly transition between scenes and interactions, while also getting a sense of comfortable user experience through smooth and easy-to-navigate design.

Scenes and respective Viewer Interactions

Scene 1 – Classroom scene

For the classroom scene, I want to create a “typical” sort of boring environment, reminiscent of the ones we can remember and/or sometimes see portrayed in media and films. I want it to have a tremendous impact on the player upon its transition to the space lobby scene.

Here is an AI-generated image of the aesthetic I am aiming for.

Note: This scene has no physical interaction besides UI.

Scene 2 – Space lobby

For the second scene, I want to create a futuristic environment, where the viewer can already get a perspective from space and how wonderful it can look. This scene will serve as both the transition between Earth and space and as a lobby for the viewer to select which lecture they would like to experience. So it should be aesthetically pleasing without being overwhelming, but also comfortable and spacious.

Here are some AI-generated images I used as inspiration.

Note: This scene has no physical interaction besides UI.

Scene 3 – Solar System Scene

This will be the first lecture. In order to make it truly immersive, it will take place right in front of a 3D depiction of the solar system, which the viewers can admire as they listen to and watch the lecture playing in front of them.

Interaction: After this is done, the viewer will have the possibility to teleport in front of the sun or each planet and explore them up close as they get data about it.

Concept drawing

Scene 4 – Mars Scene

For the Mars scene, the viewer will be teleported to a Martian landscape, where they will also listen to and watch a lecture about the red planet.

Interaction: The viewer will get to choose between learning about the Mars Curiosity Rover and its intricacies, or learning about Mars colonization projects and its hypothetical future. For both elements, the viewer will get 3D models which they will be able to analyse and explore.

Scene 5 – Black hole

Finally comes the black hole scene. Similarly to the other two levels, the viewer will be transported to a depiction of the main theme of this level. Meaning they will be in front of a black hole, as they experience the lecture.

Interaction: This one was actually extremely difficult to figure out since I didn’t know what type of interaction the player could have that would be interesting. After talking to Herman and Eric about it, we arrived at the conclusion that being sucked into a black hole would be an entertaining experience to have at this level. It would also teach the player about the physical effects and implications of black holes in matter.

Scene 6 – Earth Scene

This will be the final scene when the viewer returns to planet Earth. It should seem familiar and peaceful. So I decided that a huge grassy field under the starry night sky would do the job. At the same time not deviating from the theme of space exploration and the wonders it has.

Something like this Image:

Note: This scene has no physical interaction.

Initial Game logic

Inspiration

I had quite a few sources that inspired me for this project. First, and probably the most relevant one would be the Meta video I mentioned above, which not only inspired my overall project direction but also the aesthetic I was trying to go for in terms of UI design. Along with Hyper-reality, which gave me a different, yet equally relevant image of UI design in a hypothetical future of XR.

Then for visuals, namely space-related movies, I had a lot of inspiration from both Interstellar and Gravity, which managed to wow the viewer with their overwhelming yet mesmerising depictions of space and its wonder as well as its mysteries.

Finally, Noun Town and Human Anatomy VR are two great examples of gamification of complex subjects and access to very specific and intricate resources through a merely virtual medium such as VR. These were great references for imagining what kind of interactions I would have in my scenes and how I would develop them so that they spark the interest of the player.

Software

Unreal Engine 5

The software I will use for this project is based on what I am trying to achieve (obviously). Since my goal is to get the best and most realistic graphics while getting great performance, the game engine I’ve selected for my FMP is Unreal Engine 5. This is my main tool for the FMP

Blender

The second software I will be using is Blender. This will be for any adjustments of 3D assets, or modelling of simple ones.

Rokoko

Rokoko will be used to record and export motion capture for the robot’s movements.

Adobe Premier Pro (added later)

This software was added later in the project but I decided to include it in this page.

It was used for trimming the lecture videos and removing the original soundtrack

Adobe Illustrator (added later)

This software was added later in the project but I decided to include it in this page.

This software was used for creating UI design

Collaborations

For this project, there will be at least two or three essential collaborations that will help me create a seamless experience while not overloading me with work that I can get done with the help of other fellow students. I will need:

• A Sound Arts Design Student (for overall experience of sound and dialogues)
• An acting student (for robot’s motion capture acting)
• A UI/UX design student (for the experience’s UI Design)

I already have a rough idea of who I can contact for both sound and acting. However, I will need to establish contact with someone from UI/UX design and hopefully get their help as well.

Week 8 – PowerPoint Presentation

Here is my final presentation. Here are things I will not repeat on my blog, such as my research (which is also incorporated in my thesis), I clear explanation of my solidified ideas and respective reasons, as well as prototype videos.

Note: This is also the point at which I chose the title for my project, “Astro Odyssey”. From Astro, relating to space, the planets, stars, or other objects in space, or to a structure in the shape of a star. and Odyssey for a long journey full of adventures.

Scripts

Below are the scripts for the dialogue of the robot teacher throughout the experience. They were created with a mix of my own inputs and also some AI.

Note: Refer to the Update about lectures at the end of the page.

Final Storyboard

There were a few changes that I made to the final storyboard.

• No more classroom scenes (refer to the last paragraph on this page)
• No robot lying down on the grass (to make the experience more coherent and less prone to player location issues)
• In the end, when fading to black, the experience restarts

Synopsis

Here is the synopsis for a clear summary of my project.

Prototypes for First Submission

Here are the videos for the first prototypes of the space lobby scene, using Unreal Engine 5.

I will not get into too much detail on my process here, as I will explain this further on the next page about “Project Development”.

Note: The VR one is a bit laggy, but later on this was addressed and is now smooth as butter.

Thesis and Research for FMP

As mentioned above, my idea for this project was corroborated and explored in my thesis, where I covered multiple aspects of cinematography and their potential impact on content retention for students. For more information, please refer to the document below.

Update about lectures

There have been a number of issues that prevented me from using originally created lectures. Number one was that the information given to me by chatgpt wasn’t completely correct and was omitting crucial points.

Issue number 2, would be creating UI and or videos that matched the speech and gave the viewer interesting and informative data. To do this, I would have to invest way too much time (that I don’t have) on something that isn’t my field of study and that wouldn’t allow me to focus as much on the quality of the overall project. I could have someone do it, but it seems like it would simply give me too many issues and who knows how long I would have to wait.

I needed to make a quick and solid decision to make sure that 1, the information was undoubtedly correct. And 2, the graphics shown along with the info are pleasing and easily comprehensible.

So I decided to switch the lectures to 3 videos by National Geographic, which I found on YouTube and use their script and the videos themselves as material for my own project. There is no better way to treasure the credibility of the information, than accessing that from big scientific sources.

Naturally, I give them credit by not only maintaining the National Geographic logo but by keeping the credits at the end of the video, incorporated within my project.

Update – No Classroom Scene

There was later another decision I had to make in order to save time and keep the focus on the main concept of this project. I talked to Eric about all the elements I would have in that scene, more specifically the met human character, which would have to have their face expressions animated and sewn with the body animation. And as beautiful as it could look, it was just too much work for something that didn’t really represent the main goal of this project.

So, Eric was the one to suggest I remove the first scene (aka the classroom scene) since it wasn’t essential nor completely relevant. He also pointed out that the people trying on this experience will already be inside an educational institution (LCC building), which is enough to make my point, which I agree with.

That leaves me with 5 scenes:

• Lobby
• Solar System
• Mars
• Black holes
• Earth (Final Scene)