Creating Scenes’ Layout
Lobby
For the first scene, my goal was to create a space station or lobby, where the player could admire the wonders of space while also serving as a transition scene between all the major levels. So it had to have some sort of impact on the player. So it served as a great starting point and was my first prototype.
For the basic layout and assets of this scene, I used two very important sources. A YouTube tutorial to create the planet and sky, and a ue5 package I found that had a space station I used in my scene.
Solar System
For the solar system, I wanted to create a standard solar system that looks like the one in my concept drawing. To achieve this, I once again found a very helpful tutorial on YouTube that guided me not only through the creation of each planet material, but it also gave me the real measurements, rotation and scale of all planets in unreal units. Of course, the distance between the planets and the sun is not at the same scale, otherwise it would be impossible to showcase them together.
Mars
For the Mars Scene, I also followed a tutorial and downloaded another package to use for the Mars terrain. I had to adapt it a bit though because the tutorial focuses more on a cinematic sequence. I mostly just used it to find out about the unreal package, which apparently had 90Gb… I just used the assets I needed and then deleted it.
For the sky, I adapted it to look like a Mars sunset.
Black hole
For the Black hole scene, I also followed an extremely complex tutorial in order to create a black hole texture that felt realistic and immersive.
Earth
Finally came the Earth scene. As the closing scene of the experience, I needed it to feel somewhat nostalgic and with an overwhelming sense of smallness when facing the huge infinity of our universe. So I created a huge grassy field under the night sky, where the viewer could wrap up the experience by having a more realistic perspective of the space, and once again feel “back on earth”, as if they took off the headset. To make it visible, I added a small lamp to serve as lighting for the scene.
Lamp asset link: https://sketchfab.com/3d-models/gas-lamp-0a5013605cc848e89ae5b2e3396504da
Grass: UE5 package called “City Environment Collection”
Desk and chair
As I’ve mentioned before, in the final experience the player will be sitting down at all times. And my goal was to have a desk reminiscent of school environments. To achieve this, I used the assets from the lobby package I’d downloaded. I simply used a pre-made blueprint, deleted all the elements I didn’t need, and just left the desk and the chair.
Afterwards, I added it to all the scenes along with the player start pawn, adjusting it to the view I wanted and the player’s position and proportion in accordance with the chair and desk’s height.
However, for the earth scene I used some quixel assets to create a more realistic desk and chair that matched our planet.
Robot
For my robot character, I wanted something that seemed somewhat human (humanoid robot), to really achieve that human-like motion and feeling I chose to portray. After a long time looking for the ideal asset, I came across the following character:
I really enjoyed the overall look this character has. It’s smooth and modern without looking aggressive, like most other models I’d found. The only change I made to it was the material which I changed to metallic, since it matched the darker colours better. I also didn’t include the shiny bit as it kept giving me problems by being a separate element in the animation sequences. So the robot simply became grey and metallic.
After downloading it, I rigged it using mixamo.
Collaborations
Collab 1 – Lecture narration and Sound (and Incorporation)
For my first collaboration, I worked alongside Vlad from BA Sound Arts year 3. He’s collaborated with our course before. Last year, he was with Miranda’s group and they produced some amazing work. This year Miranda recommended him to me so I reached out through whatsapp and our collaboration began.
I am so happy to say that he is a great person to work with! Very proactive and produces great material. Always very self-reflective and looking to improve in any way he can! I couldn’t have asked for a better work partner!
During our work, we shared files and ideas through both Whatsapp and OneDrive, where we uploaded all the videos, scripts and audio files. This made the entire process smooth and hassle-free.
In terms of robot content, he recorded all the scripts I’d sent him, including the voice-over for all the national geographic videos, which he recorded according to the timing of the original ones so that the robot’s acting could match them. It was perfect!
In the end, he produced the narration for the robot Teacher I used in my project. And once again I am delighted to say that his work was great and fit my project incredibly well.
Sound Incorporation
There isn’t much to say about the sound incorporation in Unreal. It’s a very easy process that doesn’t take more than 10 minutes. Overall very smooth step of this project.
Collab 2 – Motion Capture (and incorporation)
For my second collaboration, I worked alongside Kabir from BA Acting and Performance year 2. Kabir is also a great worker, very dedicated and talented! Once I mentioned to him we’d be doing motion capture, he became delighted and excited to work with such a relevant medium to today’s media production.
Our goal here was for him to perform as the robot teacher throughout all the scenes, by acting out the movements according to the audio Vlad had sent me. As a result, there wasn’t much preparation beforehand since Kabir didn’t have to memorise any lines. It was pure improvisation, with help from my directing and notes.
We ended up producing 5 takes, meaning one per scene. And I was extremely happy with the results as well. The movement was fluid and he was able to match the audio and body language to the audio seamlessly.
With the help of Kay (Digital Arts Specialist Technician), I was able to have the files exported by him through Rokoko. He sent me everything through teams and was willing to help me with any problems I might’ve had.
Animation Incorporation
Step 1 – Blender
The incorporation of all the animation clips was a long and complex process that took trial and error when it came to understanding how it worked. Step one was to add the animation to the rigged character in Blender by using the Rokoko plugin. Afterwards, I exported it as an FBX file. This was fairly easy and I didn’t encounter too many issues besides the exporting time and which exporting settings I should use.
Step 2 – Import Into Unreal
This is where I encountered my first issue. I will not get too much into it because it was honestly very stupid and I don’t know why it’s even a problem. Basically, every time I tried importing an FBX file, it gave me an error and I couldn’t do anything. I tried re-exporting many fbx files from Blender, checked the settings, and watched tons of videos and nothing… Until one video where the guy was exporting and creating a folder with the character skeleton and before importing an fbx file he clicked on the skeleton and only afterwards did he click on importing a file. I did it, and it worked… ._.
It was such a hassle at the time and I didn’t know what to do… I even contacted Kay about it. But eventually, I sorted everything out.
Step 3 – Level sequence
The next step was to create a level sequence and add the animated character to adjust the actions to the sound and to make sure the animation started and ended where I wanted it to. Afterwards, I exported it as an animation sequence file.
Step 4 – Animation Blueprints (start Issue)
The last step was to create an animation blueprint so that it would not only play but also loop the last sequence I told it to. This was so that after the dialogue, the robot would have a repetitive, NPC-like motion forever.
And this is where I also found a major issue. I had no idea how to tell it to only play the animation when I told it to, instead of playing it right away when I played the level. This was an especially big problem for the lobby scene because once the viewer put on the headset, there would be no warning or pause, nothing. It would play audio and animation right away. And I wanted to have the first menu panel before the robot teacher even appeared.
I’d had enough of Unreal blueprints and trying to figure out how to do even the most basic of things. And I did try to find solutions to this problem. I tried making it so the character only showed up after a button was pressed, which didn’t work because it was a different type of blueprint (animation blueprint).
So I gave up and came up with another solution. A new scene. I added a new scene before the lobby and called it “Menu Scene”, which takes place in outer space with the desk simply floating in space. This way, I could have the menu panel and a peaceful environment before the animation and narration started. This was a major change since it added an entirely new scene to my project. However, it was a basic layout that simply served as a transition to the experience.
Designing interactions and mechanics
Menu
Note: This Scene has no interactions besides the UI.
The Menu scene has no interaction or any mechanics. It is simply a transition scene that features the menu panel with controller instructions and an “About” page.
Lobby
Note: This Scene has no interactions besides the UI.
This scene is like the previous one. There are no special interactions or mechanics. It features the main panel of the experience, which lets the viewer pick which lecture they want to learn from or exit to the final scene.
This scene had initially an issue with lagging, which was fixed by removing the animation from the comet ring in the level.
UI
The UI was also very basic, simply serving to move from one level to the other
Solar System
This scene was the most complex to program and to set the correct timing. The goal was to have the viewer watch the lecture in a screen which then disappears, activating a menu which lets the player explore details about the sun and the planets by setting off animations that move the planets up close. The experience ends when the viewer presses the “Finish” button.
Mechanics
- Pop-off screen mechanic – a timer is set to deactivate the lecture screen once it’s over
- Play level sequence mechanic – Once a planet/sun button is pressed, the menu disappears, it plays the animation for the selected element and activates the “Go Back” button and information panels.
- Play in reverse mechanic – Once the “Go Back” button is pressed, the animation plays in reverse, the button disappears and the menu is reactivated
- Finish mechanic – Once the “Finish” button is pressed, the player goes back to the lobby
Issues
Issue 1 – I had a major issue initially because my plan was to have the player teleported to a location in front of each planet. But after many tries and prototypes, I realised it was more complicated than I needed it to be. So instead, I had the planets come to the player with simple animation sequences triggered by the UI.
Issue 2 – Initially instead of UI, I wanted to have a 3D menu with the planets/sun. Upon cooling with one of them and pressing the trigger button would trigger the animation just like the UI menu does. But again, this proved to be overly complicated so I replaced it with UI. Here is a video of that prototype:
Mars
This level was fairly easy to code. The goal was to have the viewer watch the lecture on a screen which then disappears, activating a menu which lets the player explore details about NASA’s Curiosity Rover or witness a hypothetical scenario of a human colony on Mars and learn about them up close. The experience ends when the viewer presses the “Finish” button.
Mechanics
- Pop-off screen mechanic – a timer is set to deactivate the lecture screen once it’s over and activates the Mars menu
- rover mechanic – on “rover” button pressed, the rover and rover’s UI appear (if the colony was activated first, it now disappears)
- Colony mechanic – on “colony” button pressed, the colony and colony’s UI appear (if the rover was activated first, it now disappears)
- Finish mechanic – Once the “Finish” button is pressed, the player goes back to the lobby
Issues
I had only one issue with this scene. And That was every time I assigned the hologram material to the colony mesh, it completely deformed it. The solution was to remove the transparency property from the material and have it be opaque instead.
Black hole
This level was again, more complex since it demanded visual distortion effects and assets moving towards the player. The goal was to have the viewer watch the lecture on a screen which then disappears, activating a button that once pressed lets the player experience being sucked into a black hole, and learn about its real-life effects. The experience ends when the viewer presses the “Finish” button.
Mechanics
- Pop-off screen mechanic – a timer is set to deactivate the lecture screen once it’s over and activates the Mars menu
- Go button mechanic – once “Go” button is pressed, a sequence animation of a post-process volume with distortion is played (changing its weight gradually), giving the illusion of physical distortion and movement towards the black holes. At the same time, black hole animation plays and the black hole moves closer to the player. Finally, it also triggers a fade-to-black sequence that reaches its max once the player is inside the black hole. The info UI is activated at the end.
- Finish mechanic – Once the “Finish” button is pressed, the player goes back to the lobby
Issues
The only issue here was similar to the one in the solar system scene. I wanted to move the player towards the blackhole, but it was easier to move the black hole towards the player along with other elements moving away from the player giving the illusion that the player is the one moving. So that’s what I did instead.
Earth
Note: This Scene has no interactions and no UI.
This level was the most basic one, with nothing but a timer and fading to black mechanics, which were set to match the end of the narration and mark the end of the experience.
UI/UX Design and Incorporation
Unfortunately, after looking for UI students who were willing to collaborate with me, I couldn’t find anyone that was willing to give me a hand. I even went with Abhay to their class but neither of us was able to find what we were looking for. And with not that much time left, I had to figure it out myself. So during break, I researched and designed all the UI by myself. To accomplish this, I used Adobe Illustrator to get the best resolution possible and to make it as precise as I could.
In terms of design, I’ve mentioned my inspiration in the previous page, but overall I was trying to go for a modern look reminiscent of Apple UI and the meta’s video UI as well. I worked with darker tones as I feel like it was easier on the eyes, making the entire experience more comfortable and less overwhelming. Plus it just looks more coherent with the darkness of the entire project.
As I started adding the UI to Unreal, I found two big issues.
1 – The resolution was bad, but it was an easy fix. I simply had to make the UI huge and scale it down on the scene.
2 – As I was working with pngs it always had a few resolution imperfections, so I ended up redoing all the UI within Unreal, based on my original illustrator designs.
After fixing these, I simply scaled them down as I saw fit, and transferred all the blueprint functions from the widget prototypes to the final designs. And then I did the same thing for the blueprints that governed each widget. It was a long repetitive process but it presented no issues whatsoever.
And I must say I am extremely happy with the final results that I got. Everything looks crisp and modern, just as I initially planned.
Playtests and feedback
Abhay
My first volunteer for the playtesting sessions was Abhay. Overall he really enjoyed the experience, especially the Mars scene and being able to look at the Mars rover up close. However, he had a few notes for me:
- Fixing a repeated title in one of the UI buttons in the Mars scene
- He thinks the screens should be in the middle rather than on the side
- the menu from the solar system scene should be more vertical instead of longitudinal
- Overall, “Very cool”
I think he felt a bit bored watching the lectures though, so I have to consider adding some sort of interaction for the viewer, while each lecture is playing.
Zedan
Zedan came next and was able to give me even more complex feedback and comment on the experience’s effectiveness at standing out as an educational tool that can extended the physical limits of current educational practices. She thought it accomplished what I set it out to do and that the graphics along with the interactions, make for an extremely immersive and complementing asset for education.
As a space enthusiast herself, she thoroughly enjoyed exploring all the assets and looking at them up close, as well as watching each lecture and absorbing its contents. She gave me a few notes:
- Overall, looks amazing and accomplishes its purpose
- Solar System interaction was her favourite but she wishes the planets would spin so she could explore them further
- Slower rover rotation in the Mars Scene
- She enjoyed the lectures and didn’t think they were too long
Herman
Finally, Herman with some of the most relevant feedback. I think he enjoyed the experience in general. But he had a lot of notes to give me, and details I could work on, especially for the exhibition.
But I think the one that stood out the most to me was to have some sort of interaction during each lecture since he focuses better if he has something to do while learning anything. I fully understand this as in my case I tend to draw in order to pay attention or fidget. So I seriously am going to consider this change, even before the submission.
Here is some more feedback:
Ambient Sound
Unfortunately, I haven’t received all the ambient sound files from Vlad. As a result, I’ve decided to go ahead and find some sound effects on my own. If he sends those files in time for me to switch with the ones I found, I will reference them in my blog.
Since this wasn’t originally going to be my task, I didn’t have too much time to think about each level in detail. I simply added the sound I though was absolutely essential to making the experience better, and that Herman had included in his feedback list.
Overall, for open space scenes, I mostly consist of white noise since there isn’t supposed to be sound in space. For Mars some wind recordings I found from NASA. For the UI, I just added a button-clicking sound to all the widgets. For the solar system planets animation, a swoosh approaching sound. Finally, for the Erath Scene, some nature noises with crickets and wind.
However, I plan to develop these further for the exhibition, and hopefully, I will have more of Vlad’s work in it.
I created a list of what sound effects I wanted to have in each level:
- Menu Scene: click for UI buttons, white noise
- Spaceship Scene: click for UI buttons, white noise
- Solar System Scene: click for UI buttons, white noise, swoosh for planets approaching player
- Mars Scene: click for UI buttons, Mars wind sound,
- Black hole Scene: click for UI buttons, white noise, toy gun shooting sound
- Earth Scene: Night field sounds with wind and crickets
Links:
Changes after feedback
Beside the sound, I made some other changes after I got feedback from Herman and my classmates. Minor changes were:
- Slow down Mars rover rotation,
- Include a separate controller page on the menu scene so the player learns how to click the UI buttons before using the menu
- Removed snap rotation
Major change
The biggest change I made after the feedback originated from both the feedback I got from Herman and from watching Abhay playtest my project.
Firstly, I noticed that when Abhay was watching the lectures, he couldn’t really sit still and just pay attention to the lecture, even though it was just 3 minutes long. It felt like something was missing, something that he could do while the lecture was happening. This was the first time I thought something could be added to make it more interactive and less like a normal lecture.
Afterwards, I got the exact same feedback from Herman. He told me that he’s the type of person who likes to fidget while he’s learning anything. Furthermore, he pointed out that a lot of people are like this as well. Especially when it comes to people with attention deficits and ADHD, taking this into account would be a must. Finally, he also mentioned the fact that VR is a great opportunity for people to get up close to virtual elements and get more hands-on experiences. And it is true that a project about space is a great opportunity to make this happen.
Herman said I could add some interactions for the exhibition period. However, I decided I wanted to add them before submission as I think it would make a significant difference in how people experience my project, and I completely agreed with the feedback he gave me. It just makes the experience that much better and more interactive during the lecture bits.
Consequently, I added one small interaction to each of the three lecture levels:
Solar System
For the first interaction, I decided to have a little challenge where the viewer needs to place the sun and planets in the correct order. They can grab them, look up close, play with them, throw them, etc.
Mars
For the Mars Scene, the viewer can loom at all of NASA’s rovers and hold them to see their intricacies and specifications as well as learn when they landed on the red planet.
Note: The rover assets can be found on the NASA website. Except for the first one (https://skfb.ly/6YVJx).
Black hole
Finally, for this interaction, I decided to have a mini black hole and a gun for the viewer to shoot with. I made it so that the black hole destroys the bullets on impact so it looks like they have been sucked into it.
I am really happy I added these to my final experience before submission. They really enhanced the viewer’s experience and it doesn’t limit them to sitting down and listening.
Things to Improve for the Exhibition
Despite almost everything having worked the way that I originally planned, there are a couple of things I could improve for the physical exhibition.
Number 1 would be the sound. Unfortunately, I couldn’t get all the soundtracks from my partner in time for the submission so I had to improvise it a bit. However, for the exhibition, I plan to have it all arranged and ready for viewers to experience in a much more immersive way than it is now. The sound is a crucial part of immersiveness so I want to make sure I nail it as well.
Number 2 would be the interactions themselves and the feedback the viewer gets from them. Such as the mini black hole and the way it draws the bullets in, or having particle systems in the mini rovers when they collide with each other. Also having the solar system mini interaction play a small victory song when the player gets the order of the planets right.
Finally, I would possibly fix the limb collision in the robot animation, even tho it doesn’t affect the final experience as much.
And there is also a small issue with the player pawn location sometimes not working correctly. I haven’t had issues with it in the built file, but I have to make sure it doesn’t break during the exhibition.