Here is the video with the Gameplay. Also, I forgot to mention this earlier but I decided to call my game “Keep Your Cool” since that is necessary to accomplish this game’s goal, which implies a peaceful resolution for all the levels.
Note: I would also recommend that the person who plays this game is sitting down to help prevent motion sickness.
Critical Appraisal
I am very proud of what I was able to accomplish over the past few weeks, especially in terms of coding with C#. As time went on, I felt myself get more and more comfortable with this programming language, and it gradually started making more sense in my head, to the point where I knew what type of logic I wanted to use even if I did not necessarily know how to code it. In terms of overall aesthetics and concept, I also feel content with the scenes I managed to build. I feel that I was able to achieve what I had initially set out to do, therefore I think I kept a fairly coherent path all throughout this project.
However, this is not to say that there were no challenges. My most significant issues were to do with things that were extremely specific and that I would not have been able to solve on my own, for example, the shooting mechanic or the soldier following the player on the last level. I think gaining more experience and acquiring new C# knowledge is the only way for me to progress in this aspect. Furthermore, initially, I had a tendency to complicate the coding a bit too much. For instance, after using an object, I set all of the other tags to the same tag as the object I had used, so that the other panels wouldn’t pop up. This was extremely time-consuming and made for huge lines of unnecessary coding. However, I later realised that I could simply turn off the character’s colliders. But as I have previously mentioned, I reckon these are things I will most likely fix with more time and experience.
If a could take this project further, I would definitely add more object feedback, such as sound as well as animations. Secondly, I would polish many elements and make the more professional, such as the pop-up menus and even the 3D models themselves. I would also want to make the levels longer and more complicated, in order to mimic the feeling of an escape room, where there are multiple steps to get to the solution.
This project was definitely a challenge because it was my first time building a game and using coding. However, I think it was an extremely enriching experience since it not only gave me and my colleagues more experience but also presented us with a new set of skills that could be quite useful in the future.
In conclusion, I would say I was successful in my attempt to bring my ideas to life and I adapted to the challenges with the tools I was given and the knowledge I had. Even so, there are a lot of things that could be improved and there is much yet for me to learn in this particular field.
Before building the actual scenes, I created a few scenes just to experiment a bit with the game layout and the interaction with the characters.
I also ended up experimenting a bit with coding just to get a better idea of the type of mechanics that I will be using throughout the game.
Level 1
In level one, the options are quite limited and the interactions themselves are apparently very simple. For each item, I need to add a tag that, on collision with the character “Karen” will activate the gamestate “lose” or “win”, and then the losing/winning panel will pop up
For the gun, Herman created a file that already includes the shooting mechanic.
The only issues that I currently have are the driving option and the radio button.
Level 2
This stage is quite similar to the first level in terms of mechanics. The only difference is that on this level, the player will be able to walk. Therefore, I need to add some sort of object that activates the gamestate “lose” if the user walks too far. Meaning they can’t run away from the game.
But once again, I don’t know how I will accomplish radio button mechanics. I have to watch a few tutorials…
Note: I have also decided that the song I will use for this level is “Don’t Stop Me Now” by Queen. I find it a bit ironic since the player literally plays that song to stop the player from attacking them. Furthermore, I really like this track and I think it’s groovy and relatable to many generations.
Level 3
The last level is ironically the hardest to play but the easiest to code. The player will not be able to pick up or use any of the objects on the floor. Instead, I just have to figure out a way to keep the player from lowering their arms and some sort of solution for the hug as well. And I will also add a walking barrier, just like in level 2. However, I don’t have anything in terms of coding for this stage, yet. I will figure that out later.
Designing and Prototyping
In terms of design, I did a few sketches for the characters, which are posted on my padlet presentation. For the environments, I will be using free 3D from sketch fab to get more realistic results and accomplish that GTA aesthetic I am aiming towards.
Level 1
For this level, I wanted a very simple city scenario with a road and a couple of buildings. I got all the models from Sketchfab.
For the Karen prefab, I used MakeHuman to create the character and downloaded some of the clothes from the MakeHuman community page, which offers a wide range of free prefabs to use on the characters.
I wanted her expression to be grumpy and unpleasant. Furthermore, I wanted to embody the typical “Karen look” that we often see on social media, so I gave her a short Karen haircut and gave her some middle age looking clothes.
Level 2
The Thug level has a darker atmosphere. It’s supposed to make the player feel trapped and isolated. I got a full-scale model of an alley that captured this exact feeling and will be using it for this level. The elements are all from Sketchfab as well. In my opinion, it feels a little bit less realistic compared to the first level(and some proportions are weird), but I think it still works, especially when the other objects are present in the scene.
The Thug prefab was created in MakeHuman. I created a young-looking fellow that has this edgy expression and the mouth covered. I mainly focused on the character’s readability to make sure that he stood out in the scene, while still making sure that he looked like a Thug. For his tank top, I got a jpeg with the logo from the band Queen(Clue) and added it in photoshop to the character’s clothes textures images.
Level 3
The last level is supposed to resemble a city destroyed by war. To accomplish this look, I once again use a few models from Sketchfab a created a city that is slowly falling apart due to the bobbing and destruction caused by violence. I think this level was really supposed to embody the message that was originally intended for this game, “Reflecting on violent acts and their casualties, in a serious manner”. Therefore, I wanted to make sure it was easy to perceive by making it a fairly big open space with very readable elements.
For the soldier prefab, I tried to make sure it was very easily readable as well. I also made him a bit scrawny and scared to show that his intentions due not solely rely on violence and that he himself is scared and does not want to hurt the player, and that he is also a submissive meek character. I used MakeHuman for the Soldier, as well.
Game Mechanic and Coding
Hitting character mechanic (filtered by tags) – Level 1 and Level 2
This mechanism consists of creating tags that define which objects are “wrong” and which one is “right” when in contact with the character in the level (Karen/Thug). If they are wrong, it triggers the “Lose” Gamestate and the losing panels pop up. If it’s the right object the Gamestate “Win” is activated and the winning panels pop up. Also to make sure that after winning or losing, I wouldn’t be able to change the Gamstetae, I started by changing the objects tags after using an object, but later on, found out it was easier to turn off the character’s colliders.
Shooting mechanic – L1
Initially, I tried using the shooting mechanic uploaded by Herman on Moodle, but for some reason that was not working so, I used an extremely helpful tutorial from YouTube, which I will reference below.
Driving mechanic – L1
I struggled a bit with this mechanic… I was trying to create a smooth-driving car when the button was pressed, but for some reason, it wasn’t working. So I just created a very simple “if you press this button, this object’s position will be added +3”, kind of statement. And then attached a cube to the car with the tag “WrongItem1” to trigger the “Lose” Gamestate. I then attached the script to the main camera, created a button, attached the object (camera) and referred to the statement “Drive” for the car to move when I clicked the button.
Radio mechanic button – L1
This was pretty much like the driving script but instead of changing the car’s position, it played a song from an audio source, and then set the Gamestate to “Lose”.
Feedback systems – sound, animation and panels – L1, L2 and L3
For the feedback systems, I just added the required feedback responses to the already written “if statements”. For example, if tag = wrong item1, set gamestate to lose1 and karenmad.bool = true(this would trigger the animation). I just did this for all the sounds and animations for each character’s main script and attached the corresponding elements to them. Same thing with the Gamestate panels, but for these, I also added the option to follow the player’s head movement for levels 2 and 3. Also, I added coroutine timers to get more time to play the character’s animations/sounds, before the panels popped up.
Radio mechanic object – L2
This script was very similar to the first radio script, the only difference being that to play the song on this one, the player has to touch an object (collide with it), instead of pressing a button. And then Gamestate = lose, etc… I added a tag to the player’s hand and made it so if that tag touched the radio button object, the song would start.
Cross barrier mechanic – L2 and L3
This logic was also very straightforward. Basically, if the player’s hand (added specific tag) touched the barrier I created, the Gamestate would be set to “Lose”, etc. then after that, I added another physical barrier (no script) so that the player isn’t able to walk any further.
Parent mechanic – L2
For this one, I just followed Herman’s tutorial which would set the XR rig as the parent of each object I picked up and let me walk with them. This one took me a while to figure out. I did follow the tutorial, but for a while, I didn’t know why it wasn’t working. And then it hit me… I had set the parent to be any elements with the tag from the player’s hand and forgot to add the tag from the XR origin as well. And then it worked!
Hands-up mechanic – L3
This mechanic is the basis of the final level. It makes the player raise their arms by working as a collider that when in contact with a specific tag on the hand’s prefab, activates the Gamestate “Lose” and the level has to be restarted. This basically means that the player cannot lower their arms at any point through this last stage.
Five-second screen mechanic – L3
For this panel, I just created a coroutine timer so that the player gets a few seconds to read and adapt to this level’s rules. I also added a physical barrier so that they cannot move during these 5 seconds. Also, this timer works for the Hands-up cube by only activating it after the initial panel is gone.
Hug Mechanic – L3
This one was fun to think about. Basically, I wanted the player to have to use both hands to trigger the hug and get the “win” response. To do this, I tried a couple of approaches like “only if y & x” statements but for some reason, once again it didn’t work. So I thought it would be interesting if I had two cubes (one for the left hand and one for the right), when for example, my left hand collides with the left hand cube, it would set.active(true) a hug trigger on the right side and only by touching it with my right hand, will it activate the gamestate “win”. Same logic for the right hand. But to summarise, it means that you always need to hug with both arms, one to activate the trigger and one to touch it on the opposite side to get the “win” gamestate.
Follow player mechanic – L3
I used this mechanic to have the soldier follow (as in rotate)the player as they walk. I tried to do this for a while but it was not working. I followed countless tutorials and even tried doing it during class by also adding a rotation restraint and nothing. And then, Herman sent me a working script and it was much more simple than I’d previously thought. So yeah… now it works fine. Thanks, Herman!
Menu mechanic – L1, L2 and L3
For the pop-up menu, I followed a Youtube tutorial that taught me how to make it so that when the player moved it would follow them. However, later on, I had to change the trigger button because the button I was using was restricted and when I exported the game, it would not work. I ended up using a different button, and now it’s fine. Also, to back to the main menu, I created a script for each scene that pretty much subtracted a specific number of scenes to get to the main menu. For example, for the first level, it subtracted 1, for the second level it subtracted 2 scenes, etc. You get the logic. I was trying to do it in one script only but I did not get it to work. This logic was fine too so I used it instead. Finally, for the next level option, I used the same logic but added +1, and for the “restart” option, I just told the script to reload the current scene.
Update: Adding a Main Menu Scene
I forgot to mention this decision earlier but I decided to add a main menu before the actual game starts so that the player has time to get familiarised with the rules beforehand. Also added the buttons tutorial, which I thought would be pretty useful since I got some feedback on that.
Gameplay
Level 1
The player looks around and tries to find a clue.
In the first level, the player has to either pick an object to use or a button to push.
If they press any button, they lose and have to restart the level.
If they pick an object they will either pick the “right” one (solution) or the “wrong object”.
If they pick and use a “wrong object”, they lose and have to restart the level.
If they pick and use the “right object”, they win and can proceed to the next level.
First Playtest (Colleagues) – I let a few of my colleagues try the game to get a better understanding of how I could improve the gameplay. Actually, there wasn’t much. It was mostly feedback on the first level and second.
For the Karen level, I was told to make the Karen more obvious, since their first reaction wasn’t to look in the right direction.
For the Thug level, The radio object was too obvious and the player would accidentally press the button without even knowing it. To solve this, I put the radio in a less obvious place and set the pivot/grabbing point to a much further distance from the player’s hands, so that they wouldn’t trigger it accidentally and would instead try to find how to play it.
Second Playtest (Herman) – This playtest was crucial for me to understand some of the points I really had to improve for my final version of the game.
The first thing that was pointed out to me is that I should create some kind of particle system to make the interactable objects more apparent instead of making the player figure out which ones are usable. At first, I thought this would be necessary and that the challenge of finding the objects that worked made more sense and was part of the difficulty. However, I think it left the players confused as to what they could do on the level. Therefore, I ended up adding particle systems to the intractable objects.
The other advice I got was to make the Soldier from the last level, follow the player as they walked. I’d already been trying to do this for a while but to no avail. In the end, Herman ended up emailing me with a script that worked!
Third Playtest (Brother) – I thought I would just be showing my brother the game and everything would go smoothly but I actually found one problem that really annoyed me. On the Karen level, when the player drove by clicking the drive button, it wouldn’t be disabled and my brother ended up clicking more than once, which made him drive further and no longer be able to see the “lose” screen to restart the level.
It was an easy fix by disabling the object after the gamestate was set to “lose”, but it certainly reminded me that I had to carefully evaluate all the components in my game, to make sure nothing was missing or incorrect. I’m glad I made him play it…
Other Playtests (myself) – These were the final Playtests I did by myself. And I actually ended up tweaking a few things, such as the camera/XR rig position and for the last level, I perfected the hug triggers and their position relative to players’ hands.
Overall, easy fixes but very monotonous since every time I exported the game and thought I’d be happy with it, I ended up finding something to adjust and had to save and export everything all over again.
Update: Actually, I found a relatively big problem. When I built the game, the button that triggered the menu was not working. I spent hours trying to fix this and look for a solution when I finally found a thread online with someone that had the same problem. To summarise everything, the button I was using could not acquire other functions besides its own. so the solution was simple. I had to pick another button from the VR controller.
Update 2: I ended up doing a final playtest with a couple of friends from a different course, and got some feedback that was extremely useful. First, they asked for a controller tutorial so I added that in the main menu scene. Then they said the last level’s panel’s phrase was too long and they did not have time to raise their arms, so I made it shorter. Finally, they said the winning/losing panels for the second level could follow the character instead and I agreed. Also, I found out that one of my friends was able to jump the physical barrier on the second level somehow, and fixed it.
I used ChatGPT to help me write lines of code that I wanted/understood how they would work but didn’t know how to write the language.
I cannot stress how useful this was! Because if I were to have googled some of these, the results might have been somewhat adapted to other games and would not be able to explain as well as ChatGPT did.
My problem wasn’t not knowing what mechanics I needed to use, it was how to write them. This was extremely helpful and I highly recommend it to anyone (beginner or pro) this software, cause it saves a lot of time and gives really in-depth explanations of how to apply the code to the game.
We’ve grown accustomed to witnessing human suffering like it is our daily bread. Not that we hear about it all the time, but we see it so often through images and movies and animations to the point where we don’t really feel much about it anymore. It’s so natural nowadays to go into a store and have the ability to buy a bunch of games that show violence, suffering and human killings, that at a certain point, it doesn’t even affect us. This is not to say that I’m entirely against these kinds of games, but I think it is crucial for people to not become disconnected from the ideas represented in these entertainment platforms so that we are able to separate reality from fiction.
My idea for this game is to put violence in perspective. With everything that has been happening in the world right now, I think we could make people reflect a bit more on this issue, even if it’s through a silly game.
To put it simply, I want my game to be a series of challenges that present the player with stressful and/or annoying scenarios, from which they have to escape, by picking the correct solution without hurting the other characters or getting hurt by them.
Game type – a puzzle game where the player needs to find the correct solutions to solve each level and keep progressing. It would be like a simplified escape room. (Note: Add a clue to each level)
As I mentioned in the previous heading, there is a link to my class presentation that explains some of my ideas for the levels as well as their characters and the elements used to solve each level. However, I think it would be helpful to explain each stage a bit more in-depth.
Level 1
For the first level, the player will face an annoying situation. There will be a Karen complaining about something nonstop. The player will be inside a car (so they cannot simply run away), and there will be a bunch of actions/objects to use on Karen.
Clue – for this level will be a valentine’s day card.
Solution – giving a rose to Karen.
Twist – There isn’t one. This first stage is the easiest one and I want to make it very straightforward to give the user a very clear idea of what the goal of the game is.
Other options: Baseball bat, gun, driving the car, playing the radio, punching her(I removed this one). These are all wrong options that will lead to the player’s loss and a funny panel will pop up, telling the player what went wrong.
Level 2
For level 2, the difficulty will increase. And all of a sudden, the situation isn’t just annoying anymore, it’s dangerous.
There will be a criminal/thug threatening the player while they are completely cornered in a shady alley. The player won’t be able to walk very far (or they lose). Therefore, they will have to use the objects scattered around the floor to try to calm down the Thug.
Clue – There will be a logo of a band on the Thug’s clothes
Solution – Play the song on the radio.
Twist – instead of using this item on the Thug, the player will have to pick up the radio and figure out how to use it (so that it plays the song).
Other options – Burger, Wooden Plank, Trash Bag, Phone. These are all wrong options that will lead to the player’s loss and a funny panel will pop up, telling the player what went wrong.
Level 3
In level 3, there will be a considerable increase in difficulty. This time, the player will face a soldier pointing a gun at them, in a war-damaged city.
Clue – there will be a picture somewhere that will indirectly (or directly) tell the player what they have to do.
Solution – Hug the soldier (without lowering the arms too much…)
Twist – The player must keep their hands up all throughout this level, or they will die. They also can’t pick up any objects, but they will not know that… hehehe
Other options – Walk away, Try to pick up objects, Lower arms. These are all wrong options that will lead to the player’s loss and a funny panel will pop up, telling the player what went wrong.
Storyboard
This is also on the padlet page, posted above. But here is the general flow of the game and what happens at each stage.
Inspiration
Aesthetically, I got inspiration from the game Grand Theft Auto. I think there is also a bit of irony in picking a game that encourages pretty much just violence, as inspiration for a game that discourages it. However, I think that only makes the message stronger. By graphically communicating to the player that this is a similar game, they will be tempted to resort to violence. But they will quickly realise that this will not work.
I also looked into a game called “Toilet Chronicles”, which was shown to me by Herman. I found this game extremely funny, but what attracted me to it the most was its mechanic. It is an escape room kind of game, where the player has to gather multiple clues and solve various puzzles throughout many stages to solve all the challenges a finish the game.
There was something else that got my attention. In this game, when the player dies, a panel pops up, explaining the type of death and for each type, there is a funny little drawing of the cause that lead to the player’s demise. I think I will do something similar to this, as I reckon it gives some comedic relief to the gameplay.
Why VR?
I think this question is especially relevant considering my concept. I want the player to have to make the decisions from a first-person perspective. However, if it were to be played on a computer or console, the only way to interact with it, would be through joysticks and/or buttons. By making it a VR game, the player has to perform the actions physically and interact with the situation “first hand”. All of the responsibility of hurting other characters will feel much more present and real. Meaning the player will be able to choose to physically hurt another character “with their own hands” and see the physical consequences of their actions.
The animation has been by far the smoothest task up until now. I only had to animate the camera moving from the house and around the scene. Firstly, I had to create a motion path and then attach the camera to it. finally, I just had to fix the camera aim to the Japanese temple and define the number of frames I wanted my animation to have.
However, the rendering process comes with many issues. If I want to render using Arnold, The image takes about 3 minutes to render, and since I have 960 images, that is not the best option, when Maya Hardware 2.0 is only a second. Of course, there was a problem with the lights (mentioned on the previous page) but since I have enough lights for what is relevant, I will have to stick with Maya 2.0. Also, Arnold can’t process the ocean shader and it deforms the Japanese temple, for some reason.
Update: Maya 2.0 took about 15 minutes to render the whole sequence. After retouching some stuff and re-rendering the animation. I finally have a sequence to put together.
I found a free app that is apparently very easy to use and the process to import the sequence is very intuitive. It’s called Shotcut.
Update 2: The video is complete and I also added some sound to it. I got a soundtrack from YouTube with jungle/alien-like voices. I think it gives the scene more realistic and mysterious.
Most of the texturing process was smooth but it didn’t come without its difficulties. For the most part, I only had to choose textures I liked from the internet, but others required a little more fine-tunning.
First of all the trees. For some reason Maya wasn’t recognising the leaf texture, making the branches look like flat images. But this problem was quickly solved when Kelvin told me that I needed a transparency shader. I just had to use photoshop and it finally worked!
Secondly, had to find a way to create a type of soil that would look as if it were from another planet. Once again, YouTube tutorials were vital sources of information to solve this. I found out that I could blend two images and multiply the number of times they are repeated. This was extremely useful and it gave the terrain a unique and exotic look.
Also, I had to switch from a skydome to a sphere due to some complications with lighting (I will mention it below). But this did not compromise anything.
Finally, the biggest problem I had with the textures was with the base for the temple. I originally wanted it to be made out of stone but since I did not understand how to do UV mapping, I had to find a quicker solution and opt for wood, since it looks good even if it’s stripy. I really tried to understand UV mapping but there was a lot to unravel and I only have a couple of days left, so I think it was a pretty successful alternative.
Lights
As for the lights, there is not much to say. I wanted to have a few panels illuminating the path where there were no trees, in order to shift the focus towards the temple and the cave, as well as the paths that led to them. I chose light pink for the lighting to make it more unnatural and somewhat cosmic. I also added some pink ambient lights near the planets to create a similar effect. And finally, I added one directional light to brighten up the space.
I only had one issue throughout this process. When I rendered it with Maya Hardware 2.0, the panels would literally show up uninvited. But I found out how to solve this after being here for almost one hour. I had enabled the lights to show on the Maya 2.0 engine because I wanted the skydome to show the image. I just had to replace it with a sphere, disable the lights on the render settings, and done.
It’s a little bit shaky since it was recorded in Unreal’s VR viewport. It showcases the entire experience nonetheless.
WARNING
ATTENTION: When playing the project, sometimes, the player’s height is a little off, transporting the player to a lower or higher spot than where it should be. In case this happens, please refer to the project file and adjust player height as necessary. I am still trying to figure out why it’s happening. It hasn’t happened with the built file, so hopefully, no one will encounter this issue during the assessment period.
For the first scene, my goal was to create a space station or lobby, where the player could admire the wonders of space while also serving as a transition scene between all the major levels. So it had to have some sort of impact on the player. So it served as a great starting point and was my first prototype.
For the basic layout and assets of this scene, I used two very important sources. A YouTube tutorial to create the planet and sky, and a ue5 package I found that had a space station I used in my scene.
Solar System
For the solar system, I wanted to create a standard solar system that looks like the one in my concept drawing. To achieve this, I once again found a very helpful tutorial on YouTube that guided me not only through the creation of each planet material, but it also gave me the real measurements, rotation and scale of all planets in unreal units. Of course, the distance between the planets and the sun is not at the same scale, otherwise it would be impossible to showcase them together.
Mars
For the Mars Scene, I also followed a tutorial and downloaded another package to use for the Mars terrain. I had to adapt it a bit though because the tutorial focuses more on a cinematic sequence. I mostly just used it to find out about the unreal package, which apparently had 90Gb… I just used the assets I needed and then deleted it.
For the sky, I adapted it to look like a Mars sunset.
Black hole
For the Black hole scene, I also followed an extremely complex tutorial in order to create a black hole texture that felt realistic and immersive.
Earth
Finally came the Earth scene. As the closing scene of the experience, I needed it to feel somewhat nostalgic and with an overwhelming sense of smallness when facing the huge infinity of our universe. So I created a huge grassy field under the night sky, where the viewer could wrap up the experience by having a more realistic perspective of the space, and once again feel “back on earth”, as if they took off the headset. To make it visible, I added a small lamp to serve as lighting for the scene.
Grass: UE5 package called “City Environment Collection”
Desk and chair
As I’ve mentioned before, in the final experience the player will be sitting down at all times. And my goal was to have a desk reminiscent of school environments. To achieve this, I used the assets from the lobby package I’d downloaded. I simply used a pre-made blueprint, deleted all the elements I didn’t need, and just left the desk and the chair.
Afterwards, I added it to all the scenes along with the player start pawn, adjusting it to the view I wanted and the player’s position and proportion in accordance with the chair and desk’s height.
However, for the earth scene I used some quixel assets to create a more realistic desk and chair that matched our planet.
Robot
For my robot character, I wanted something that seemed somewhat human (humanoid robot), to really achieve that human-like motion and feeling I chose to portray. After a long time looking for the ideal asset, I came across the following character:
I really enjoyed the overall look this character has. It’s smooth and modern without looking aggressive, like most other models I’d found. The only change I made to it was the material which I changed to metallic, since it matched the darker colours better. I also didn’t include the shiny bit as it kept giving me problems by being a separate element in the animation sequences. So the robot simply became grey and metallic.
After downloading it, I rigged it using mixamo.
Collaborations
Collab 1 – Lecture narration and Sound (and Incorporation)
For my first collaboration, I worked alongside Vlad from BA Sound Arts year 3. He’s collaborated with our course before. Last year, he was with Miranda’s group and they produced some amazing work. This year Miranda recommended him to me so I reached out through whatsapp and our collaboration began.
I am so happy to say that he is a great person to work with! Very proactive and produces great material. Always very self-reflective and looking to improve in any way he can! I couldn’t have asked for a better work partner!
During our work, we shared files and ideas through both Whatsapp and OneDrive, where we uploaded all the videos, scripts and audio files. This made the entire process smooth and hassle-free.
In terms of robot content, he recorded all the scripts I’d sent him, including the voice-over for all the national geographic videos, which he recorded according to the timing of the original ones so that the robot’s acting could match them. It was perfect!
In the end, he produced the narration for the robot Teacher I used in my project. And once again I am delighted to say that his work was great and fit my project incredibly well.
Sound Incorporation
There isn’t much to say about the sound incorporation in Unreal. It’s a very easy process that doesn’t take more than 10 minutes. Overall very smooth step of this project.
Collab 2 – Motion Capture (and incorporation)
For my second collaboration, I worked alongside Kabir from BA Acting and Performance year 2. Kabir is also a great worker, very dedicated and talented! Once I mentioned to him we’d be doing motion capture, he became delighted and excited to work with such a relevant medium to today’s media production.
Our goal here was for him to perform as the robot teacher throughout all the scenes, by acting out the movements according to the audio Vlad had sent me. As a result, there wasn’t much preparation beforehand since Kabir didn’t have to memorise any lines. It was pure improvisation, with help from my directing and notes.
We ended up producing 5 takes, meaning one per scene. And I was extremely happy with the results as well. The movement was fluid and he was able to match the audio and body language to the audio seamlessly.
With the help of Kay (Digital Arts Specialist Technician), I was able to have the files exported by him through Rokoko. He sent me everything through teams and was willing to help me with any problems I might’ve had.
Animation Incorporation
Step 1 – Blender
The incorporation of all the animation clips was a long and complex process that took trial and error when it came to understanding how it worked. Step one was to add the animation to the rigged character in Blender by using the Rokoko plugin. Afterwards, I exported it as an FBX file. This was fairly easy and I didn’t encounter too many issues besides the exporting time and which exporting settings I should use.
Step 2 – Import Into Unreal
This is where I encountered my first issue. I will not get too much into it because it was honestly very stupid and I don’t know why it’s even a problem. Basically, every time I tried importing an FBX file, it gave me an error and I couldn’t do anything. I tried re-exporting many fbx files from Blender, checked the settings, and watched tons of videos and nothing… Until one video where the guy was exporting and creating a folder with the character skeleton and before importing an fbx file he clicked on the skeleton and only afterwards did he click on importing a file. I did it, and it worked… ._.
It was such a hassle at the time and I didn’t know what to do… I even contacted Kay about it. But eventually, I sorted everything out.
Step 3 – Level sequence
The next step was to create a level sequence and add the animated character to adjust the actions to the sound and to make sure the animation started and ended where I wanted it to. Afterwards, I exported it as an animation sequence file.
Step 4 – Animation Blueprints (start Issue)
The last step was to create an animation blueprint so that it would not only play but also loop the last sequence I told it to. This was so that after the dialogue, the robot would have a repetitive, NPC-like motion forever.
And this is where I also found a major issue. I had no idea how to tell it to only play the animation when I told it to, instead of playing it right away when I played the level. This was an especially big problem for the lobby scene because once the viewer put on the headset, there would be no warning or pause, nothing. It would play audio and animation right away. And I wanted to have the first menu panel before the robot teacher even appeared.
I’d had enough of Unreal blueprints and trying to figure out how to do even the most basic of things. And I did try to find solutions to this problem. I tried making it so the character only showed up after a button was pressed, which didn’t work because it was a different type of blueprint (animation blueprint).
So I gave up and came up with another solution. A new scene. I added a new scene before the lobby and called it “Menu Scene”, which takes place in outer space with the desk simply floating in space. This way, I could have the menu panel and a peaceful environment before the animation and narration started. This was a major change since it added an entirely new scene to my project. However, it was a basic layout that simply served as a transition to the experience.
Designing interactions and mechanics
Menu
Note: This Scene has no interactions besides the UI.
The Menu scene has no interaction or any mechanics. It is simply a transition scene that features the menu panel with controller instructions and an “About” page.
Lobby
Note: This Scene has no interactions besides the UI.
This scene is like the previous one. There are no special interactions or mechanics. It features the main panel of the experience, which lets the viewer pick which lecture they want to learn from or exit to the final scene.
This scene had initially an issue with lagging, which was fixed by removing the animation from the comet ring in the level.
UI
The UI was also very basic, simply serving to move from one level to the other
Solar System
This scene was the most complex to program and to set the correct timing. The goal was to have the viewer watch the lecture in a screen which then disappears, activating a menu which lets the player explore details about the sun and the planets by setting off animations that move the planets up close. The experience ends when the viewer presses the “Finish” button.
Mechanics
Pop-off screen mechanic – a timer is set to deactivate the lecture screen once it’s over
Play level sequence mechanic – Once a planet/sun button is pressed, the menu disappears, it plays the animation for the selected element and activates the “Go Back” button and information panels.
Play in reverse mechanic – Once the “Go Back” button is pressed, the animation plays in reverse, the button disappears and the menu is reactivated
Finish mechanic – Once the “Finish” button is pressed, the player goes back to the lobby
Issues
Issue 1 – I had a major issue initially because my plan was to have the player teleported to a location in front of each planet. But after many tries and prototypes, I realised it was more complicated than I needed it to be. So instead, I had the planets come to the player with simple animation sequences triggered by the UI.
Issue 2 – Initially instead of UI, I wanted to have a 3D menu with the planets/sun. Upon cooling with one of them and pressing the trigger button would trigger the animation just like the UI menu does. But again, this proved to be overly complicated so I replaced it with UI. Here is a video of that prototype:
Mars
This level was fairly easy to code. The goal was to have the viewer watch the lecture on a screen which then disappears, activating a menu which lets the player explore details about NASA’s Curiosity Rover or witness a hypothetical scenario of a human colony on Mars and learn about them up close. The experience ends when the viewer presses the “Finish” button.
Mechanics
Pop-off screen mechanic – a timer is set to deactivate the lecture screen once it’s over and activates the Mars menu
rover mechanic – on “rover” button pressed, the rover and rover’s UI appear (if the colony was activated first, it now disappears)
Colony mechanic – on “colony” button pressed, the colony and colony’s UI appear (if the rover was activated first, it now disappears)
Finish mechanic – Once the “Finish” button is pressed, the player goes back to the lobby
Issues
I had only one issue with this scene. And That was every time I assigned the hologram material to the colony mesh, it completely deformed it. The solution was to remove the transparency property from the material and have it be opaque instead.
Black hole
This level was again, more complex since it demanded visual distortion effects and assets moving towards the player. The goal was to have the viewer watch the lecture on a screen which then disappears, activating a button that once pressed lets the player experience being sucked into a black hole, and learn about its real-life effects. The experience ends when the viewer presses the “Finish” button.
Mechanics
Pop-off screen mechanic – a timer is set to deactivate the lecture screen once it’s over and activates the Mars menu
Go button mechanic – once “Go” button is pressed, a sequence animation of a post-process volume with distortion is played (changing its weight gradually), giving the illusion of physical distortion and movement towards the black holes. At the same time, black hole animation plays and the black hole moves closer to the player. Finally, it also triggers a fade-to-black sequence that reaches its max once the player is inside the black hole. The info UI is activated at the end.
Finish mechanic – Once the “Finish” button is pressed, the player goes back to the lobby
Issues
The only issue here was similar to the one in the solar system scene. I wanted to move the player towards the blackhole, but it was easier to move the black hole towards the player along with other elements moving away from the player giving the illusion that the player is the one moving. So that’s what I did instead.
Earth
Note: This Scene has no interactions and no UI.
This level was the most basic one, with nothing but a timer and fading to black mechanics, which were set to match the end of the narration and mark the end of the experience.
UI/UX Design and Incorporation
Unfortunately, after looking for UI students who were willing to collaborate with me, I couldn’t find anyone that was willing to give me a hand. I even went with Abhay to their class but neither of us was able to find what we were looking for. And with not that much time left, I had to figure it out myself. So during break, I researched and designed all the UI by myself. To accomplish this, I used Adobe Illustrator to get the best resolution possible and to make it as precise as I could.
In terms of design, I’ve mentioned my inspiration in the previous page, but overall I was trying to go for a modern look reminiscent of Apple UI and the meta’s video UI as well. I worked with darker tones as I feel like it was easier on the eyes, making the entire experience more comfortable and less overwhelming. Plus it just looks more coherent with the darkness of the entire project.
As I started adding the UI to Unreal, I found two big issues.
1 – The resolution was bad, but it was an easy fix. I simply had to make the UI huge and scale it down on the scene.
2 – As I was working with pngs it always had a few resolution imperfections, so I ended up redoing all the UI within Unreal, based on my original illustrator designs.
After fixing these, I simply scaled them down as I saw fit, and transferred all the blueprint functions from the widget prototypes to the final designs. And then I did the same thing for the blueprints that governed each widget. It was a long repetitive process but it presented no issues whatsoever.
And I must say I am extremely happy with the final results that I got. Everything looks crisp and modern, just as I initially planned.
Playtests and feedback
Abhay
My first volunteer for the playtesting sessions was Abhay. Overall he really enjoyed the experience, especially the Mars scene and being able to look at the Mars rover up close. However, he had a few notes for me:
Fixing a repeated title in one of the UI buttons in the Mars scene
He thinks the screens should be in the middle rather than on the side
the menu from the solar system scene should be more vertical instead of longitudinal
Overall, “Very cool”
I think he felt a bit bored watching the lectures though, so I have to consider adding some sort of interaction for the viewer, while each lecture is playing.
Zedan
Zedan came next and was able to give me even more complex feedback and comment on the experience’s effectiveness at standing out as an educational tool that can extended the physical limits of current educational practices. She thought it accomplished what I set it out to do and that the graphics along with the interactions, make for an extremely immersive and complementing asset for education.
As a space enthusiast herself, she thoroughly enjoyed exploring all the assets and looking at them up close, as well as watching each lecture and absorbing its contents. She gave me a few notes:
Overall, looks amazing and accomplishes its purpose
Solar System interaction was her favourite but she wishes the planets would spin so she could explore them further
Slower rover rotation in the Mars Scene
She enjoyed the lectures and didn’t think they were too long
Herman
Finally, Herman with some of the most relevant feedback. I think he enjoyed the experience in general. But he had a lot of notes to give me, and details I could work on, especially for the exhibition.
But I think the one that stood out the most to me was to have some sort of interaction during each lecture since he focuses better if he has something to do while learning anything. I fully understand this as in my case I tend to draw in order to pay attention or fidget. So I seriously am going to consider this change, even before the submission.
Here is some more feedback:
Ambient Sound
Unfortunately, I haven’t received all the ambient sound files from Vlad. As a result, I’ve decided to go ahead and find some sound effects on my own. If he sends those files in time for me to switch with the ones I found, I will reference them in my blog.
Since this wasn’t originally going to be my task, I didn’t have too much time to think about each level in detail. I simply added the sound I though was absolutely essential to making the experience better, and that Herman had included in his feedback list.
Overall, for open space scenes, I mostly consist of white noise since there isn’t supposed to be sound in space. For Mars some wind recordings I found from NASA. For the UI, I just added a button-clicking sound to all the widgets. For the solar system planets animation, a swoosh approaching sound. Finally, for the Erath Scene, some nature noises with crickets and wind.
However, I plan to develop these further for the exhibition, and hopefully, I will have more of Vlad’s work in it.
I created a list of what sound effects I wanted to have in each level:
Menu Scene: click for UI buttons, white noise
Spaceship Scene: click for UI buttons, white noise
Solar System Scene: click for UI buttons, white noise, swoosh for planets approaching player
Mars Scene: click for UI buttons, Mars wind sound,
Black hole Scene: click for UI buttons, white noise, toy gun shooting sound
Earth Scene: Night field sounds with wind and crickets
Links:
Changes after feedback
Beside the sound, I made some other changes after I got feedback from Herman and my classmates. Minor changes were:
Slow down Mars rover rotation,
Include a separate controller page on the menu scene so the player learns how to click the UI buttons before using the menu
Removed snap rotation
Major change
The biggest change I made after the feedback originated from both the feedback I got from Herman and from watching Abhay playtest my project.
Firstly, I noticed that when Abhay was watching the lectures, he couldn’t really sit still and just pay attention to the lecture, even though it was just 3 minutes long. It felt like something was missing, something that he could do while the lecture was happening. This was the first time I thought something could be added to make it more interactive and less like a normal lecture.
Afterwards, I got the exact same feedback from Herman. He told me that he’s the type of person who likes to fidget while he’s learning anything. Furthermore, he pointed out that a lot of people are like this as well. Especially when it comes to people with attention deficits and ADHD, taking this into account would be a must. Finally, he also mentioned the fact that VR is a great opportunity for people to get up close to virtual elements and get more hands-on experiences. And it is true that a project about space is a great opportunity to make this happen.
Herman said I could add some interactions for the exhibition period. However, I decided I wanted to add them before submission as I think it would make a significant difference in how people experience my project, and I completely agreed with the feedback he gave me. It just makes the experience that much better and more interactive during the lecture bits.
Consequently, I added one small interaction to each of the three lecture levels:
Solar System
For the first interaction, I decided to have a little challenge where the viewer needs to place the sun and planets in the correct order. They can grab them, look up close, play with them, throw them, etc.
Mars
For the Mars Scene, the viewer can loom at all of NASA’s rovers and hold them to see their intricacies and specifications as well as learn when they landed on the red planet.
Note: The rover assets can be found on the NASA website. Except for the first one (https://skfb.ly/6YVJx).
Black hole
Finally, for this interaction, I decided to have a mini black hole and a gun for the viewer to shoot with. I made it so that the black hole destroys the bullets on impact so it looks like they have been sucked into it.
I am really happy I added these to my final experience before submission. They really enhanced the viewer’s experience and it doesn’t limit them to sitting down and listening.
Things to Improve for the Exhibition
Despite almost everything having worked the way that I originally planned, there are a couple of things I could improve for the physical exhibition.
Number 1 would be the sound. Unfortunately, I couldn’t get all the soundtracks from my partner in time for the submission so I had to improvise it a bit. However, for the exhibition, I plan to have it all arranged and ready for viewers to experience in a much more immersive way than it is now. The sound is a crucial part of immersiveness so I want to make sure I nail it as well.
Number 2 would be the interactions themselves and the feedback the viewer gets from them. Such as the mini black hole and the way it draws the bullets in, or having particle systems in the mini rovers when they collide with each other. Also having the solar system mini interaction play a small victory song when the player gets the order of the planets right.
Finally, I would possibly fix the limb collision in the robot animation, even tho it doesn’t affect the final experience as much.
And there is also a small issue with the player pawn location sometimes not working correctly. I haven’t had issues with it in the built file, but I have to make sure it doesn’t break during the exhibition.
For my final major project, I intend to create a VR experience within the realms of cinematography. I want to cover this topic while also tackling it within the context of educational environments and how it could prove beneficial or not for students, as a way of complementing their overall content retention and today’s incorporation of education with emerging technology.
Furthermore, it is true that cinematography tends to leave a huge impact on people, and some of the hobbies or interests we pursue are greatly impacted by media, more specifically movies and shows. Just imagine how this potential could be used in VR and how much more it could expand the way we experience that media.
But for now, I think I will focus more on this question in regards to my dissertation. Since the first submission date isn’t until November, that gives me plenty of time to figure out what I could do.
Concept
Through my FMP, I intend to explore the idea of cinematography as a valuable tool for educational settings and whether or not it has an impact on how students (and learners in general) retain information based on visual data, cues and psychology of film-making strategies.
Here are some examples of useful papers I found:
“VR has shown tremendous potential in educating and rehabilitating children (ages 18 and below) with neurodevelopmental disorders. A study by Wang and Reid (2011) explores the use of this technology, examining multiple case studies that have tested the efficacy of VR on participants ranging in age, and gender disorder. These hypotheses were tested for Attention Deficit Hyperactivity Disorder, Autism and Cerebral Palsy. As a goal, this comprehensive analysis intended to further explore the potential of VR in addressing neurological impairments in children and contribute to a better understating of its future implications in VR education.”
This paper was a valuable resource in evaluating education within different neurodevelopmental spectrums. As a result, it provided me with a crucial perspective on VR education that accounts for other perspectives aside from those of neurotypical folk. This could help me develop a more accessible experience across a spectrum of students.
“When addressing filmmaking, the impact of cinematographic practices can vary considerably based on the medium used to guide the spectator’s gaze. For one, in VR, since the viewer has complete agency over where to look throughout the experience, cinematographic principles cannot be applied the same way as in 2D screens. Although these practices may remain the same in terms of taxonomy, the methods used to apply them as a way of diverting attention within Cinematic Virtual Reality (CVR) change completely. This is due to the viewer’s agency and the use of 3D immersive environments that enforce the existence of another spatial dimension, making it a physical experience. This paired with the existence of multiple points of interest (POI) influences the viewer’s ability to fully absorb the contents of immersive cinematic pieces. Therefore, the focus lies on finding the methods that allow the creators to direct focus. Further research by Rothe, Buschek and Hußmann(2019), demonstrates some of the methods proposed, with novel approaches that address the processes within CVR.”
Links: https://www.mdpi.com/2414-4088/3/1/19
For a better analysis of these ideas, please refer to my thesis posted near the end of this page, where I explore these ideas further and create a more supported basis for my approaches throughout this project.
Update 1
I have been reflecting on what kind of cinematic experience I want to have for my project. I think it is logical that it covers some type of school subject since my goal is to make it an educational experience for regular students (or anyone else who is curious and wants to give it a go).
However, I feel like lately I’ve been having somewhat of an artistic block and I can’t think of what I should make. I’ve been brainstorming and I’ve come up with some potential ideas:
Language experience
A historical trip through time eras
Space exploration
Geographical exploration (potentially using Google Maps in unreal could be cool)
Anatomy experience
Sadly, I don’t feel strongly about any of those ideas yet and it seems like I don’t really know what I should do… I’ve talked to Eric about this, and he said I should take my time thinking about this and look for inspiration. Hopefully, at some point, I’ll start feeling more drawn towards something.
Side Story
One week ago, a few girls from 3D Animation came to our class looking for people to collaborate with, from BA VR. They presented their idea, which consisted of a cinematic experience in a non-sensical world inspired by Alice in Wonderland. I immediately became extremely excite with the thought of being able to collaborate with them to create something that was still more or less coherent to my plans. But honestly, I think I rushed it a little bit since I was panicking with the fact that I couldn’t find an idea that pleased me at the moment. So I ended up joining their group and changed my original plan.
However, we had a meeting two days ago. This made me instantly realise that it would be a mistake for 3 reasons:
They had complete control of the project (because it was theirs to begin with)
I felt like I was getting sidetracked from my original plan
I started feeling like I did it out of despair and not really for the idea
So, unfortunately, I had to message them saying I wanted to leave the group. And now I am back to square 1. They were really nice and their ideas were great. I simply cannot afford to jeopardise mine, for the sake of feeling comfortable because of their people’s ideas. But I feel like I should still write about it to show that sometimes things don’t go according to plan and we need to stay confident in our ideas and convictions.
Update 2
I have been getting back on track and looking for inspiration, so I can start moving in one direction. And actually, there is one video that really caught my attention! Meta released it in 2021 and it shows audiences a hypothetical depiction of what MR, AR, XR and VR could look like and how immersive they could be when showing people the wonders of space. I thought it was a really smart choice because it’s nearly impossible for the average human being to experience any sort of space-related experiences, and this might be the only way (for now), that we can bring that reality a bit closer to people, with the help of immersive technologies.
When it comes to other subjects such as history or geography, I can visit physical sites and I’ll be “teleported” to that reality because there are still traces of it (not for everything though). What I mean is, it is possible to immerse myself in those subjects through the real world. But not for space. That’s what made the video so enticing!
I feel like this is finally the direction I want to pursue! A VR experience about space, with the goal of not only educating people but also bringing them closer to such a distant and exclusive experience such as exploring space. Now I need to start developing the concept more and figure what the details of the experience. But at least I am finally feeling good about my FMP idea.
Weeks 3 to 6 – Developing ideas and Prototyping
(Note: I am writing this on week 6 so I already have a very solid idea, drawings, prototypes, etc. So I will kind of just skip to the main idea that I arrived at in the end, the feedback from Eric and Herman and my material)
Throughout these 3 weeks, I started to develop my ideas in a more solid and structured manner. My goal was to captivate the viewer with spatial scenarios and bring them closer to it through mesmerising graphics and interesting facts. So I chose to do an experience with three different themes, to bring variety not only to the visuals but also to the content provided to the viewer.
The 3 scenes will be about:
Mars
The Solar System
Blackholes
I thought these would be a great choice since at least two of them are familiar enough to most people that it’s still recognisable and easy to enjoy while still looking amazing. Whereas the black holes scene would be more of an open-ended question since there is still so much to know about them and how they influence our universe. It would end the experience in an open-ended way and leave the viewer expecting more, and hopefully, feeling inspired to want to learn. Like we sometimes feel after watching a great movie (hence the power of cinematography.
I would still want to have some sort of human connection throughout the experience, so I think having someone guiding the viewer throughout the scenes would be a great way to have them connect in a human way as well, which would further captivate them since they’d be able to establish a connection with a mere virtual character.
For the visuals that precede the 3 main scenes, I want to have two other levels. One that will serve as a level selection lobby, where the viewer can choose which lecture to see first. And another one before which takes place “in our actual world”. What I mean by this is a scene that happens in a regular realistic scenario that everyone experiences, school.
Therefore, the experience will start by subverting people’s expectations into thinking they will have a normal school lesson in a boring-looking classroom, with a normal teacher. Which kind of defeats the purpose of VR. Or at least that’s what I want them to think at first. After that scene, the viewer will be transported into the space lobby and the real experience can start.
Also, it is supposed to be a cinematic experience where I want the viewer to be impacted by the visuals, so I don’t want to sacrifice their ability to enjoy high graphics in VR by having too much movement. That would also jeopardise the experience since I’ll be using Unreal which will really push this project to its performance limits. (I’ll talk more about the software below). So the entire experience will have the viewer sitting down in front of a desk (which makes the transition from the classroom environment to space, a lot smoother, since they will both have desks).
Update – Robot teacher
Why stop at the transition between classroom and space? What if the human teacher became a humanoid robot? In my opinion, it would once again impact the viewer once they made the transition from earth to space. No one would expect it, and there would still be a character creating human connection, but in this context, it would shift to something more futuristic. And the fact that this robot teacher wouldn’t exist in real life (at least for now), makes it more exclusive and memorable to the viewer.
I talked about this idea with Eric and he approved it. However, I think he sees it more like a small robot or more peet-like. In my view, it should be humanoid so as to not lose the human side that I want it to have. Besides, it would be much easier to animate with motion capture than moving a robot around, which could give me an unrealistic load of work.
Concept drawing
Initial storyboard
Quick explanation – It starts with the viewer in an ordinary classroom being taught by a human teacher who introduces the theme of the class (astronomy). It then transitions to the space lobby that replaces the classroom and a robot that replaces the human teacher. They give the viewer 3 options to explore: the solar system, mars or black holes. Upon choosing one option, the player is transported to each respective lecture where they can feel mesmerised by the realistic and immersive environments that correspond to each theme. They have an audio lecture and a few interactions at each level that will further teach them about astronomy. At the end of each lecture, they must return to the lobby. Finally, once satisfied the viewer can click on the finish button in the lobby scene and they will be transported “back” to Earth, where the robot does an extremely sentimental and touching monologue about the wonders of space. They then invite the viewer to lie down on the grass with them while gazing at the stars, as it all fades to black.
Lectures
I want to emphasise this again. Besides being a cinematic experience, this project is directed towards education, so I want to incorporate great visuals with interesting, factual and thought-provoking information.
As a result, I want each level to have a small lecture (which will be given by the robot teacher obviously), where the viewer will get to enjoy the environment, as well as learn more about it.
But I feel that simply audio could be ignored so I want there to be visuals, images, signs, slide perhaps, alongside with the audio and the robot. I think that would complement it very well.
Initially, I thought that I could create these lectures with someone specialising in astronomy, but due to time constraints and overcomplicating my work process, I will be using AI to generate the lectures. This could also be useful when writing my script.
Update – Interaction and UI
I feel like this was always on the back of my head, but I think it’s important to fully commit to this idea, as I develop and explain my project. I think the incorporation of good UI design is crucial for this project, as I will be tackling a type of encounter that is mostly based on user experience and comfortability. Therefore, a clear and modern UI seems like a no-brainer.
This will certainly help the viewer smoothly transition between scenes and interactions, while also getting a sense of comfortable user experience through smooth and easy-to-navigate design.
Scenes and respective Viewer Interactions
Scene 1 – Classroom scene
For the classroom scene, I want to create a “typical” sort of boring environment, reminiscent of the ones we can remember and/or sometimes see portrayed in media and films. I want it to have a tremendous impact on the player upon its transition to the space lobby scene.
Here is an AI-generated image of the aesthetic I am aiming for.
Note: This scene has no physical interaction besides UI.
Scene 2 – Space lobby
For the second scene, I want to create a futuristic environment, where the viewer can already get a perspective from space and how wonderful it can look. This scene will serve as both the transition between Earth and space and as a lobby for the viewer to select which lecture they would like to experience. So it should be aesthetically pleasing without being overwhelming, but also comfortable and spacious.
Here are some AI-generated images I used as inspiration.
Note: This scene has no physical interaction besides UI.
Scene 3 – Solar System Scene
This will be the first lecture. In order to make it truly immersive, it will take place right in front of a 3D depiction of the solar system, which the viewers can admire as they listen to and watch the lecture playing in front of them.
Interaction: After this is done, the viewer will have the possibility to teleport in front of the sun or each planet and explore them up close as they get data about it.
Concept drawing
Scene 4 – Mars Scene
For the Mars scene, the viewer will be teleported to a Martian landscape, where they will also listen to and watch a lecture about the red planet.
Interaction: The viewer will get to choose between learning about the Mars Curiosity Rover and its intricacies, or learning about Mars colonization projects and its hypothetical future. For both elements, the viewer will get 3D models which they will be able to analyse and explore.
Scene 5 – Black hole
Finally comes the black hole scene. Similarly to the other two levels, the viewer will be transported to a depiction of the main theme of this level. Meaning they will be in front of a black hole, as they experience the lecture.
Interaction: This one was actually extremely difficult to figure out since I didn’t know what type of interaction the player could have that would be interesting. After talking to Herman and Eric about it, we arrived at the conclusion that being sucked into a black hole would be an entertaining experience to have at this level. It would also teach the player about the physical effects and implications of black holes in matter.
Scene 6 – Earth Scene
This will be the final scene when the viewer returns to planet Earth. It should seem familiar and peaceful. So I decided that a huge grassy field under the starry night sky would do the job. At the same time not deviating from the theme of space exploration and the wonders it has.
Something like this Image:
Note: This scene has no physical interaction.
Initial Game logic
Inspiration
I had quite a few sources that inspired me for this project. First, and probably the most relevant one would be the Meta video I mentioned above, which not only inspired my overall project direction but also the aesthetic I was trying to go for in terms of UI design. Along with Hyper-reality, which gave me a different, yet equally relevant image of UI design in a hypothetical future of XR.
Then for visuals, namely space-related movies, I had a lot of inspiration from both Interstellar and Gravity, which managed to wow the viewer with their overwhelming yet mesmerising depictions of space and its wonder as well as its mysteries.
Finally, Noun Town and Human Anatomy VR are two great examples of gamification of complex subjects and access to very specific and intricate resources through a merely virtual medium such as VR. These were great references for imagining what kind of interactions I would have in my scenes and how I would develop them so that they spark the interest of the player.
Software
Unreal Engine 5
The software I will use for this project is based on what I am trying to achieve (obviously). Since my goal is to get the best and most realistic graphics while getting great performance, the game engine I’ve selected for my FMP is Unreal Engine 5. This is my main tool for the FMP
Blender
The second software I will be using is Blender. This will be for any adjustments of 3D assets, or modelling of simple ones.
Rokoko
Rokoko will be used to record and export motion capture for the robot’s movements.
Adobe Premier Pro (added later)
This software was added later in the project but I decided to include it in this page.
It was used for trimming the lecture videos and removing the original soundtrack
Adobe Illustrator (added later)
This software was added later in the project but I decided to include it in this page.
This software was used for creating UI design
Collaborations
For this project, there will be at least two or three essential collaborations that will help me create a seamless experience while not overloading me with work that I can get done with the help of other fellow students. I will need:
A Sound Arts Design Student (for overall experience of sound and dialogues)
An acting student (for robot’s motion capture acting)
A UI/UX design student (for the experience’s UI Design)
I already have a rough idea of who I can contact for both sound and acting. However, I will need to establish contact with someone from UI/UX design and hopefully get their help as well.
Week 8 – PowerPoint Presentation
Here is my final presentation. Here are things I will not repeat on my blog, such as my research (which is also incorporated in my thesis), I clear explanation of my solidified ideas and respective reasons, as well as prototype videos.
Note: This is also the point at which I chose the title for my project, “Astro Odyssey”. From Astro, relating to space, the planets, stars, or other objects in space, or to a structure in the shape of a star. and Odyssey for a long journey full of adventures.
Here are the videos for the first prototypes of the space lobby scene, using Unreal Engine 5.
I will not get into too much detail on my process here, as I will explain this further on the next page about “Project Development”.
Note: The VR one is a bit laggy, but later on this was addressed and is now smooth as butter.
Thesis and Research for FMP
As mentioned above, my idea for this project was corroborated and explored in my thesis, where I covered multiple aspects of cinematography and their potential impact on content retention for students. For more information, please refer to the document below.
There have been a number of issues that prevented me from using originally created lectures. Number one was that the information given to me by chatgpt wasn’t completely correct and was omitting crucial points.
Issue number 2, would be creating UI and or videos that matched the speech and gave the viewer interesting and informative data. To do this, I would have to invest way too much time (that I don’t have) on something that isn’t my field of study and that wouldn’t allow me to focus as much on the quality of the overall project. I could have someone do it, but it seems like it would simply give me too many issues and who knows how long I would have to wait.
I needed to make a quick and solid decision to make sure that 1, the information was undoubtedly correct. And 2, the graphics shown along with the info are pleasing and easily comprehensible.
So I decided to switch the lectures to 3 videos by National Geographic, which I found on YouTube and use their script and the videos themselves as material for my own project. There is no better way to treasure the credibility of the information, than accessing that from big scientific sources.
Naturally, I give them credit by not only maintaining the National Geographic logo but by keeping the credits at the end of the video, incorporated within my project.
Update – No Classroom Scene
There was later another decision I had to make in order to save time and keep the focus on the main concept of this project. I talked to Eric about all the elements I would have in that scene, more specifically the met human character, which would have to have their face expressions animated and sewn with the body animation. And as beautiful as it could look, it was just too much work for something that didn’t really represent the main goal of this project.
So, Eric was the one to suggest I remove the first scene (aka the classroom scene) since it wasn’t essential nor completely relevant. He also pointed out that the people trying on this experience will already be inside an educational institution (LCC building), which is enough to make my point, which I agree with.
This is the video I recorded for our presentation. I thought of redoing it but the changes were so minor, I don’t think it is worth recording the entire gameplay once again.
To be honest, I am not much of a prototype person in general. I normally tend to build the scene first and then work on them directly because it saves me a lot of time. And that was something crucial in this project. Nonetheless, I did make 1 major prototype outside the clown scene, since Jason was working on it. Actually, I am glad I did it outside that scene because it gave me a lot more visibility and space to work with.
I also did some prototyping for the skyscraper scene. Mainly in the teleportation system for when the player falls and spawns on top of the buildings. But again, this was done directly in the main scene.
I feel like my prototyping would be talking about all the mechanics I use directly in each scene since I don’t do too much experimenting by blocking out stuff beforehand. So I’ll just explain all of those in the mechanic’s section. And meanwhile, I’ll mention here what could be considered a prototype. ¯\_(ツ)_/¯
Clown Scene Doors
This idea hurt my brain in more ways than I could imagine. :,) It was so difficult to make it work properly and I swear to God a cried a tear when I stopped finding bugs every time I playtested it.
The goal was to create a three-door choice for the player. I explained it in my sketch on the previous page, but basically, I needed two doors with jumpscares and one that was correct. Every time the player lost, they’d go back to the spawn point in front of the three doors and everything would reset (animations, positions lighting, etc). And the elements for each scenario would appear based on which door was opened and the triggers the player collided with. Also when colliding with the triggers, all the other triggers would disappear, the door behind would close and a timer would set off the lights. And for the jumpscares, I used a ray cast system so that when the player looked at the mirror/clown behind, it would set a coroutine delay system that would turn the lights on and off at specific times and enable the clowns in front of the player’s face. And finally teleport them to the spawn point where there’d be a cube that on trigger enter would reset everything. You can see how much could go wrong… It was horrible to code and an absolute nightmare to make it work perfectly. But I did it so a moment of silence for my remaining brain cells.
Skyscraper teleportation and crane
Another mechanic to hunt my dreams… I wanted the player to respawn on a specific building each time they fell and hit the coloured cubes corresponding to each stage of the level. But for some reason, I couldn’t make it work. After hours of experimenting with the first-person controller’s settings and reading threads online, I finally found someone that had a solution. I had to disable the player first and then change its position. And also had to make all the cubes within the player kinematic which also solved the crane problem.
Regarding this last issue, I couldn’t make the player a child of the crane and therefore wasn’t able to take the fps to the next building. But as I mentioned in the previous paragraph, it had something to do with the cubes having to be kinematic, otherwise, it wouldn’t work. It also took me a while to figure this one out.
Designing and Building the scenes
Normal Room Scene
“This is the first scene the player encounters. A nice room where everything is seemingly normal. There is a warm Jazz track in the background and the birds sing happily. That is until the player drinks the water.”
This room was fairly straightforward. Since it was more of a transitional scene I didn’t have to focus as much on the mechanics and simply had to create a cozy-looking atmosphere. To do that, I used slightly warmer tones and replace the asset’s floor with a wooden texture. For the outside environment, I simply used a skybox image. However, this went through a couple of iterations because when I played the scene, the image looked blurry and had some weird marks. In the end, I added a couple of trees to make it more isolated but still comfortable enough that the player wouldn’t be alarmed by it.
Finally, I added a water bottle asset, attached a particle system to make it obvious to the player, and then added a teleportation script. I’ll talk more about this in the game’s mechanics section.
Weird Room Scene
“After passing out, the player is teleported to the second scene. It’s an iteration of the first room, and it marks the beginning of the horror game by setting a dark, unsettling atmosphere. Once the player walks out the door, the game starts.”
This room was simply a different iteration of the normal room. I made a few changes to the lighting and global volume, and that was pretty much it.
I decided to go with a darker blue for the overall aesthetic and to give it a mysterious, offputting look. Then, I used contrasting colours such as green and red to really give the scene an ominous horrific vibe.
The red light obviously had to be used for the door to symbolise danger, but also alert the player to its existence and give them no choice but to investigate it.
Skyscraper Scene
“The first level is acrophobia (fear of heights). An extreme mental and physical challenge where the player not only needs to find courage but also physically move in order to escape this terrible parkour-like challenge. It is set in a city with mostly skyscrapers and surrounded by a misty fog that only intensifies the vertiginous feeling.”
This scene was probably my favourite to design. Considering the fact that it happened during the day, it was much easier for me to visualise all the elements and come up with something pleasing to the eye.
Again, I used assets from sketchfab for the building used for the obstacle course and all its elements. On the other hand, I used a unity asset of New York I found for the background buildings. Then I added a fair amount of fog to give a more vertiginous sensation to the player. I made it blue for the exact same reason I mentioned above. It gives this sort of fear of the unknown kind of feeling to it.
This scene is not as dark or overly sensory as the other levels, because the focus is not making it look scary. If someone has acrophobia, they don’t need to be exposed to heights at night to be absolutely petrified of it. Furthermore, I liked the fact that it gave the player a false sense of security by thinking “Oh… it doesn’t look that bad…”. Obviously, it gets much worse quickly and that slow but steady progression is also what makes the game more intriguing.
Clown Scene – Night
“Once the player tries to grab the balloon, they officially start level 2, Colrouphbia (fear of clowns). A terrifying experience set in the same abandoned amusement park, and it shatters the player’s trust by surprising them constantly with jumpscares and horrifying soundtracks. Probably the most relatable scary scene, as it mimics the sort of feeling we get when watching horror movies. The goal is to get to the end. And as simple as that sounds, it is definitely not an easy task.”
Even though it wasn’t my responsibility to do it, I still helped design and retouch the clown scene. Overall, Jason did a very good job at adjusting the lighting. However, I did make a couple of changes. First of all, I adjusted the terrain a bit by making the hills a bit lower. Then, I replaced the lamps with something more cohesive and modern and adjusted some of the lighting for the main interactions and assets. Afterwards, I added the models Jason got from Taobao. This was a whole process… I had to use Unreal engine to open the files and export each one + all the textures and then add them to the main scene. Finally, I took care of the final stage by adding the 3 doors and all the mechanics involved.
We ended up with a very dark scene, which was definitely great because there was a lot of empty space. But the player can’t tell…
The lighting was again a matter of contrast between the colours green and red to unsettle the player throughout the level. Moreover, the models for this scene were crucial to creating an abandoned park scenario and Tabao was definitely a saviour because it gave us a very complete package of assets for dirt cheap.
And even though I would like to make some major changes to this scene I feel like was very effective, overall.
Clown Scene – Day
“When the player finishes level 1, they are teleported to this intermediate scene. It’s quite simplified and sort of bizarre. It doesn’t seem real, even within the nightmare scenario. It’s weird and it looks like the player can see the building blocks of the game as if it were a dream. It throws them off for a second and nothing prepares them for what comes next.”
There isn’t much to say about this scene. It is a very brief transitional scene. I simply changed the skybox and removed all the lights. Jason took care of the asset changes and created the scene.
Overall it just had to look somewhat normal and not scary to give the player a false sense of safety and a somewhat confused feeling.
Hallway Scene
“Finally, level 3 is Claustrophobia (fear of enclosed spaces). Taking place in a never-ending corridor, the player is not only challenged with puzzles but also emotionally pressured by the time constraint and a growing but steady realisation that the walls are closing in. It is necessary to find all the solutions to the challenges in order to get out. Otherwise, the player is stuck in a perpetual cycle of death. Once they open the correct door, they find a water bottle and are suddenly reminded of how it all started. But they have no other option. They must drink it.”
For the last level, I used a very fitting corridor model from sketchfab. It fits the aesthetics perfectly, and with the appropriate lighting and some global volume effects, it was looking sharp. I did have a problem later on related to the walls moving and animating that. The mesh was all fused so naturally thought “I’ll fix that in blender”. Oh boy, was I wrong… The mesh was a mess! it was so bad that each time I tried to separate it and import it, it had holes everywhere and was ruined. So the solution was to shrink the entire corridor instead which worked fine but wasn’t ideal. Nonetheless, it worked.
For the stars and the map, I used Illustrator to create the shapes and the sprite images. The reaming models were also from sketchfab.
Main Menu Scene
The main menu scene was very straightforward. Jason create the main menu panel with all the elements necessary, and I just added a black box around the play to create a neutral dark environment. This way the user doesn’t know what to expect but is still unsettled and intrigued by it. And to create contrast, I made the letters white and red, once again to communicate danger. Then I simply disabled the walking mechanic and poof, the main menu is done.
Update: later on I also added the title in bright red letters to mimic those bright lamp signs that make that annoying buzzing sound. This way, it stood out.
Credits Scene
“It all ends in this room. Where it all started. It all looks like it is back to normal but not quite. This final iteration of the first two rooms introduces elements that were seen throughout the game. The city outside, a picture of a clown, the playing cards from the hallway, and even a TV. It makes the player question whether everything they went through was real. Maybe it wasn’t a dream? Or is it still a dream? It’s up to you to decide.”
And last but not least, came the credits scene. And another iteration of the first “Normal Room”. Only this time, it needed to have elements from all the levels to make the player question whether it had all been real or not. The first two elements were easy to come up with. Cards spread around the room and a picture of a clown. But I didn’t know what to do for the skyscraper scene. Add a crane? Nope, don’t make sense… And then It hit me. What if when the player looks outside, they see the same city as the skyscraper scene? Chef’s kiss. It worked out perfectly! Then all that was left was for Jason to add the TV and make the credits video e Voila, the last scene was finally complete. Also, there is a little surprise for the player as well muahaha.
Game mechanics
This section is really difficult to summarise because there were so many scripts but a lot of them used similar/the same mechanics so I’ll do my best to summarise all of them and explain where they were included. I think in the past I was a bit confused by what game mechanics are but I think now I understand it a little better. I understand now that I am not to mention every script individually but instead, the mechanics that are within them and that govern the game and the player’s options.
Walking mechanic – all scenes except MM
This mechanic was automatically included in all levels. However, it was deactivated in the Main Menu because it wasn’t necessary.
Parent mechanic – NR, L1, L2N, L3, CS
The parent mechanic was mainly used for any interactable with the scenes, for example, the bottles, playing cards, pieces of paper or the key. However, it was also applied to the crane’s cart which became the parent of the fps when both collided.
Grab object mechanic – NR, L1, L2N, CS
This mechanic goes along with the parent mechanic for all interactables in this game. It allows the player to hold any objects that have the interactable script.
Pop Menu mechanic – all scenes except MM
This mechanic was activated by the Y button in the left-hand controller, and it can change the flow of the game completely by taking the player to other levels or the main menu.
Pass out mechanic – L1 and L3
This mechanic firstly was activated in the normal room with the water bottle. And then twice (if the player dies) in Level 3, by either getting trapped between the walls or once again drinking from the water bottle that marks the end of the game.
Level progression/ regression mechanic – all scenes
This progressive mechanic was activated at the end of each level, and it allows for a continuous progression from challenge to challenge. The only exception is the Main Menu, where the mechanic is activated right at the beginning of the scene. The second exception is the pop-up menu where this mechanic can be applied and the only one that has a regressive mechanic by allowing the player to, for example, go from level 3 back to level 1.
Reload level mechanic – L1,2D and 3
This mechanic is true for all the scenes that are included in the pop-up menu, meaning if the player chooses to level 2 while in level 2, they will reset the level and start right at the begging which in this case would mean the clown scene (day version).
It is a very generalised way of including any collision interactions that resulted in animations. These include anything from the doors to the crane, the clowns moving, the carousel speeding up or the hallway shrinking. They were all tag-based conditions that resulted in the activation of the necessary bools to trigger animations.
Activate lights on collision mechanic – L2N
This mechanic was only used for the clown scene (night version) to activate deactivate/activate the lights on collision with a trigger. It was used for the glitch light in the path and the room with the 3 doors. Much like the previous mechanic it worked by using the “Player” tag and then turning on or off the desired lights.
Activate/Interrupt sound on collision mechanic – all scenes except MM
Much like the previous two mechanics, this tag-based logic worked for all the sounds that were not activated at the begging of the scenes and needed to be played or interrupted at specific times. For example, riding the crane cart, playing the sounds when clowns showed up or stopping all sounds when the player opened the door in level 3.
Activate object on collision mechanic – L1, 2N and CS
Same logic as the previous three mechanics, but instead activated objects. For example the clowns in the clown scene (night)and Credits scene or the window cleaning platform in level 1.
Teleport mechanic – L1, 2N and 3
The teleportation mechanic was crucial for all levels. But this is especially true for the skyscraper scene, where the player would benefit from having spawn points instead of restarting the entire level. It was also used for the clown scene(night) to take the player back to the spawn point in front of the final stage. And finally, it added a sort of “cheating mechanic” to the last level by allowing the player to move from one end of the corridor to the other in a blink of an eye, at least for the players who notice it anyway.
Activate gravity mechanic – L1
Only employed in the last bit of the first level, this was a tag-based collision that activated the gravity component of the wooden planks, which made them fall in the correct timing and let the player land in the precise spot to move to level 2.
Climbing mechanic – L1
This mechanic was used solely for the ladder climbing stage in level 1. It was extremely complex to code and it required me to remove major scripts from the fps in that scene for it to work. In the end, I think it was very successful and never gave me any big headaches. It was simply a strenuous process.
Raycast mechanic – L2N
Finally, there was the Raycast mechanic, used only in the last stage of the clown scene(night). It was activated by a tag-based collision and it allowed the player to activate objects(clowns) by looking at specific elements in the room.
MM – main menu
NR – normal room
WR – weird room
L1 – Level 1
L2N – Level 2 night
L2D – Level 2 day
L3 – Level 3
CS – credits scene
Gameplay
Playtests and feedback
First Playtest – Miranda
Miranda was officially the first outside person to try out our game. At the time we only had the hallway scene and the skyscraper scene for her to play around with. She really enjoyed the experience and suggested that we add birds flying or people screaming at the skyscraper level.
Good thing she tested it because there was in fact a problem with the wooden planks in the first level. But overall, very helpful playtest.
Second Playtest – Serra
This playtest was hilarious! It was the first time we got a reaction from someone to the clown jumpscares and to the entire game in one go. Moreover, we received very useful feedback from Serra.
She was the first person to get motion sickness from our game. We were not expecting it but she said it had probably been the lights from the hallway level. This was very important for us, because one of my main focuses throughout the entire process of making this game, was to create something interactive and playful, while not sacrificing movement and action too much, and still providing a comfortable experience for the player. I took it into consideration a tweaked the lighting in that scene.
There was something else that I noticed during this playtest. When it the time to climb the ladder from level 1 came, she couldn’t reach it, due to her height (hahaha). But I was very intrigued by this because she could jump to reach it, which added a layer of realism to the game. So decided to keep it! The same thing when riding the crane cart, sometimes I had to lower my head to get in. But it makes sense! In real life, we have different heights and sizes and therefore experience the physical world differently.
Third Playtest – Myself
This isn’t really a fourth or first playtest. I can’t really count the number of times I had to play each level to make sure it was working or to fix any bugs. But during each playtest I’d always find something to be tweaked or an unexpected bug. This was especially true for the last section of the Clown Scene. As I wasn’t teleporting the player to different scenes each time they chose a door, resetting all the positions, lights and animations was an absolute nightmare and there were always some things not working correctly.
Sometimes, when the clowns appeared right in front of my face, they would glitch with the lights. At first, I thought it would be cool to leave it like that and embrace the glitch. But I was too afraid to incorporate it into the project due to unforeseeable outcomes. Maybe after I’m done submitting this project, I’ll play around with that…
But this is all to say that this process was truly complex and I think it’s also got something to do with the way I code. Compared to the last project, I feel like I am way more concise and organised in my coding. However, there is still a lot for me to learn.
Forth Playtest – Abhay
Abhay was the playtester for our presentation (Thank you!). He was very enthusiastic and was able to grab the crowd’s attention. Unfortunately, he knew about some of the jumpscares so he saw them coming sometimes. But still, very successful overall. Also, he couldn’t reach the ladder either hahahahahha.
And I noticed one small problem. Because this stage has to haunt me until I submit this project… In the 3 doors stage, when he went to the correct room, it didn’t activate the lights or something was off but he couldn’t see the image I put there. I don’t know if he was just looking at the wall or if there was a bug but I’ll have to look at it. It’s weird because it worked for me fine…
The Sound
Sadly I shouldn’t have to write about this section as much as I’m going to but I have no choice since I ended up having to create/find online most of the sounds. I know it wasn’t my job and I wouldn’t have been penalised for it, but I take pride in the work that I do, and I knew the sound was a crucial part of our game. So I took matters into my own hands and created something I could use for the game.
I used various sources such as YouTube and Zappslat. And software such as Adobe Audition and Bandlab.
I created things like the voices and beats for the hallway scene and edited the creaking sounds as well. I also did the clown voice saying “bye-bye”. And I edited the background track and the doll’s pitch as well. Everything else was either from the very few sounds the Sound Art students gave me or mostly from YouTube and Zapslat.
As for each scene, I mainly just created Audio Sources and also added Audio reverb zones to make the audio more realistic and coherent to the environment. Finally, I occasionally used an audio mixer to make some clips a bit louder.
The presentation was a resounding success. From the commitment from everyone to dress up to the final projects, everything was really inspiring and made me feel proud of our class!
Furthermore, the feedback was very positive and extremely useful as well. We got so many different opinions, but in the end, we felt very satisfied with all the responses. Thank you to everyone!
I hope all the other groups were equally satisfied. Everyone should be proud of what they did (background clapping).
I made a few changes to the game after the presentation. Thankfully minor ones, because the feedback was mostly positive and because I didn’t have a lot of time to change everything. But I”l list all the things a tweaked:
Add a particle system to the final water bottle;
Added a carpet on top of the floor in the Hallway Scene. this way the player can get the illusion that the walls are moving by looking at the carpet, instead of shrinking which was much less noticeable;
I made the creaking sound of the walls progressively louder to add a 3D effect and make the shrinking more obvious to the player;
CHANGE DOOL TO GHOST AND PUT IT ON THE SWING!!!!!!
I didn’t change everything though. Serramentioned changing the lights for some of the scenes and the aesthetic for the skyscraper model as well. But I think it’s too much for the time I have to finish all the projects, and honestly, I am pretty happy with the aesthetics. Changing the entire look of the skyscraper scene would be too much.
Final Work Distribution Table – who did what
Scene
Layout
Coding
– Main Menu
Jason and Margarida
Margarida
– Normal Room
Margarida and Jason
Margarida
– Weird Room
Margarida
Margarida
– Skyscraper Scene
Margarida
Margarida
– Clown Scene (night)
Jason and Margarida
Jason and Margarida
– Hallway Scene
Margarida
Margarida
– Credits Scene
Jason and Margarida
Margarida
– Clown Scene (day)
Jason and Margarida
Margarida
– Adding Sound
Margarida
Margarida
Sources
Youtube
As always, YouTube is a crucial source for building our projects. It’s basically our second teacher when we don’t have class. There are tons of tutorials that helped us through the making of this game. However, this is not to say that all of them worked and there are always issues and bugs along the way. But it is part of the process to mess up as well. But overall, it was the most important source that we used to make this game.
https://youtu.be/qKDzZd0sjxQ
ChatGPT
It’s funny how normal it feels to use chatGPT nowadays. As always, it helps me turn my thoughts and mechanics in my head into code. Once again, it doesn’t do all the work for me. But when I don’t know how to write something specific in C#, it’s a great source that gives me the appropriate code for specific actions within a mechanic. Way better than having to find the solutions in online threads or on youtube.
Moodle – Herman
Once again, Herman, thank you for all the tutorials and help you gave us throughout this module! The videos, files and were extremely useful and effective! The feedback was also very important and helped us make some crucial changes in our game before and after the presentation. Thank you so much for your dedication!
Working as a group – thoughts
I don’t want to mention this too much in my critical appraisal but I still want to express my opinion in regards to this collaboration and my overall experience.
At first, I thought everyone was excited and committed to making something great! It felt awesome to know that I was surrounded by people with such diverse knowledge and willingness to work hard. We had meetings, and even almost had one of the sound art students involved in the 3D process…
But then it all shifted very quickly. I gave them a sound list and videos, I organise all the meetings, and they started bailing on me. It was so frustrating!
The thing is, I love group projects because we not only get to share and distribute work equally but we also expose ourselves to other types of knowledge and people that we will find in professional scenarios as well. However, when a group doesn’t do anything, it makes it hard not only to do work properly but it’s almost impossible to be on good terms with them afterwards. It’s not about the grades or the project. It’s the attitude and extreme lack of respect for us as individuals. At least try!
Some of them did send us some sound to work with, but they posted them weeks ago and it wasn’t even for the main scenes… A few days before the presentation, they sent me a couple of audio clips as well, but again not enough to cover the entire project.
And as if it wasn’t bad enough already, after the presentation some wanted me to desperately add their sounds to the game as if they suddenly woke up to reality. Besides having other submissions(which I told them one week before the presentation), I also had a lot to do in this one week. There is no way that I’ll be replacing all the sound clips now… It’s simply too much… And one of them even complained… Only of them showed up to the presentation… Honestly, I could write a critical appraisal just on this subject but I think this gets the message across perfectly.
Overall, I found this project way more stressful than it had to be since I had to take over almost everything. I need a break.
For this collaboration project, Jason and I were paired to work alongside Sound Art students. The goal is to create a VR experience that is at least, 10 minutes long, and focuses on the user’s interaction and the sound feedback. Therefore, it should be a balanced experience where both the VR interaction and the sound experience shine.
Our group is composed of 7 people:
Jason Cui (BA Virtual Reality)
Margarida Belo (BA Virtual Reality)
Hanrui Zhang (BA Sound Arts)
Si ting Wang (BA Sound Arts)
Ties Linssen(BA Sound Arts)
Harvey Young (BA Sound Arts)
Bohang Zhang (BA Sound Arts)
First Ideas
After the first collaborative sessions, my group and I met up to brainstorm some ideas to start thinking about how we will manage the project regarding spring break, online/in-person meetings, sharing platforms, communication platforms, etc.
First of all, we were immediately drawn to the idea of a horror experience. When it comes to terrifying experiences, such as scary games/movies, the sound is one of the most if not the most relevant factors throughout the entire product. So we thought this would be a brilliant way to create a balance between the visual components and the audio ones.
There was also this ongoing idea of making users doubt themselves throughout the entire experience. Like we want them to go slightly crazy by giving them jumpscares and playing with their perceptions of the narrative and how it’s all connected.
In terms of aesthetics, we’re still not quite sure how to go about it. One person suggested a futuristic theme. Others were leaning more towards a “classical” modern horror experience. Honestly, I prefer the latter because I feel that the focus should be on the phobias themselves and not on the fact that it happens in space or something like that. We’ll figure something out.
In the end, we unanimously agreed that our VR Artefact would be a game. All the ideas and suggestions pretty much lead us to conclude that what we were describing, would involve the player’s agility and interaction in order to finish the entire experience and face their fears by solving each level.
Update – developing our concept
We have been developing the idea further and decided to create something to do with phobias. We thought it would be interesting to get people’s reactions and make them face their worst fears by overwhelming them with a highly sensory experience. We did some research on the most common phobias and took note of the ones we reckoned were most universal/impactful.
And there was also this really funny idea of having a dog playing Jazz music throughout the experience and having it guide the player/playing with their emotions, and ultimately just be a normal dog once the challenges/main experience is over.
Even though I think it’s premature to start discussing this, we talked about having the ending be a giant version of the house where the player is spawned at the begging of the game and is stepped on by a giant Jazz Dog and then the game is over. Personally, I think this shouldn’t be set in stone, since the levels should be the main focus and will require a lot more work. So I wouldn’t want us to spend too much time working on unnecessary plots. But the idea sounds pretty fun, I’ll give them that.
Jason and I pretty much had to refuse some of their ideas straight away. There was the idea of a mirror maze with a delay to the player’s reflection and then stuff would show up but not in the mirrors… There was also an idea with rising water and then having to dive and the dog would be swimming there and guiding the player. Just confusing and too complicated for VR. Besides, we also warned them about motion sickness and the fact that this is not a regular “2D” computer game. We have limitations in terms of knowledge, skill and most importantly the platform on which the game will be played.
Just writing about it makes me anxious. However, I think it was good that we were able to establish our limits and what we thought would be feasible, while also still pleasing our group and compromising on other factors. I think because they don’t know what creating a game implies, they just started to throw ideas in hopes we would just do it. But we explained everything thoroughly and made sure everyone was clarified and happy with our suggestions.
Update 2 – Final Phobias and other ideas
We ended up selecting three phobias for our final game. Those would be Acrophobia (fear of heights), Coulrophobia (fear of clowns) and Claustrophobia (fear of tight spaces). We figured these were relatable enough and would give us more options to interpret fear and how to go about scaring the player, without using jumpscares all the time.
We don’t have a logic for each level yet, but we have a pretty clear idea of how we want the game to flow/how the levels will be connected.
Final Idea and Storyboard
For our final idea, we just polished it a bit and ended up going with the phobias idea. We also created a very rough game flow idea, as in which levels would go first and how the player would transition from one level to the other. the order we ended up going with was Acrophobia, Coulrophobia and Claustrophobia. The reason is that we want the player to get a fake sense of familiarity and security due to the bright somewhat relatable skyscraper scene, and from then it gets worse. Also, the clown scene goes in a second so that we can later add a surprise jumpscare when the player has already forgotten about this scene. So, that leaves the hallway scene for last. We are still deciding how the transition from level three to the credits scene is going to play out but for now, this should be solid enough of an idea.
We presented this in class and the feedback was very helpful. We even got some suggestions for movies to use as inspiration.
Also, we officially removed the idea of the dog because it was going to take too much time to create something that wasn’t essential/the focus of the experience. Instead, when the player finishes the game, they will watch the credit (maybe scare them one more time… hehe heh) and then they go back to the main menu and the entire game starts over.
We started to refine our idea by making sure the genre was clear and the concept/game logic was easy to understand by the audience. We also distributed tasks amongst each other equally and made sure everyone had something to work on. I will post a link below to our PowerPoint presentation that demonstrates this a bit further.
For the first two rooms and the credits, we used a model we found on sketchfab. The only thing that changed throughout the game is the lighting and the interaction the player can have within the room. Meaning its purpose is to be a transitional scene.
Normal and Weird room. These are the first two scenes introduced to the player. They mark the transition between normal and nightmare. Reality vs abnormality. Once the player walks out of the Weird room, the game starts.
The goal is to create a fake sense of security and trust in the normal room, which is immediately shattered once the player drinks the water. The weird room is the first scare the player faces in the game. But it is just the begging…
Level 1 – Skyscraper Scene
For level 1, we had to come up with ways the player would interact with a parkour scenario. There is walking and climbing. Unfortunately jumping wasn’t a viable option. But to achieve this, we decide to create a city scene where the player needs to walk and climb their way through an obstacle course to get to the end. While doing this they must face the sight and constant pressure that comes with not falling to their “deaths”.
These are the elements we included in this level, by order:
Glass as a way to make the player look down;
Climbing the ladder as a physical challenge
Riding the crane was a bit of a jumpscare,
Crossing the parkour platforms as a coordination challenge;
Crossing the unstable wooden planks (and then falling hehe);
The ideas and layout changed throughout the process of building the level, so we ended up with something closer to the first sketch below. But to summarise, the player must go through the aforementioned trail of obstacles to get to the final door and move on to the next level (clown scene).
Level 2 – Clown Scene
For level 2, it took a while to design a good plot. But in the end, we chose to do an abandoned park where the player is followed/terrorised by clowns that keep showing up announcing and terrorising them in various ways. Since it was an open scene and there was more visibility, we had to come up with diverse interactions to keep the player engaged (scared) throughout the experience. Furthermore, the visuals and interactions were also a bit tough to imagine due to this scene taking place in such a huge environment. But in the end, we were able to think of quite a few obstacles to include in this level.
Firstly, as the player walks, they encounter a clown sitting on the bench, after walking a bit more forward, the clown says “bye-bye” and disappears. Then, the player will encounter a crazy-looking clown crawling across the path (jumpscare). Then the carrousel will be triggered and its animation will go faster and a clown will pop up there, laughing. Afterwards, the player sees a lamp glitching for a second and as this happens, a clown appears and disappears. Finally, before the last stage, the users will see a creepy swing with a doll singing and murmuring.
At the end of the path, after all the interactions I mentioned there will be three final doors, where the player has to pick the correct one to move on to the final level. There will be no clues (hehe). I will not explain all the options because they are literally written in the second image below.
Sorry that summary was probably as easy to understand as the drawing below but it is a very concise description of everything that happens on this level, by order (which actually changed as proved below).
Note: Initially someone from our group suggested we do the entire scene in a circus tent, but we felt like it was a bit restrictive. Furthermore, we like the idea of the player being in an open space and exposed to a huge environment because it makes them feel more vulnerable to any threats.
Level 3 – Hallway Scene
Finally, for level 3 the idea was to have a long corridor with the walls closing in on the player to really simulate that claustrophobic feeling. We had to think of an escape room type of logic to challenge the player’s ability to solve this level while being slowly antagonised by the time constraint.
In order to get out, the player needs to first find the correct door by analysing the map and finding the symbol corresponding to the correct door. Then, by paying close attention to the sound, they must choose the correct cards and place them on the scanners next to the correct door. And finally, they must get the key to open the door. While all of this is happening, the walls are shrinking and the player has 3 minutes to complete the level before they are squished to death, and the level restarts.
After they finish this, they will find a room with a bottle of water, just like in the beginning, which they must drink to go to the credits scene.
Regarding the sound, the clue to the cards will be in the number of beats the player can hear.
Game Flow Diagram
In the end, we ended up with 8 scenes. After a couple of changes to the plot and suggestions from tutors, we decided to add an extra clown scene just to add some mystery and confusion. It won’t be anything fancy since it’s not the main level, but just something to further take the player into a state of despair.
The flow of the game is pretty linear and straightforward. The player has to finish a level to go to the other in order to finish the game. However, there is one exception that changes this a bit. The pop-up menu will allow the player to go from one level to another by simply selecting it. So the only mandatory level is really just the last. Ideally, we want the players to go through each level, but for presentation/ experience purposes, Jason and I decided to have this option so users can swiftly go from one level to another and take a look around.
Also, I guess we realised this at some point but the goal of this game slightly shifted from getting people to overcome their phobias to instead traumatising them as much as possible, which in my opinion, is a whole lot more fun!
Inspiration
In terms of inspiration, there were 3 games that inspired our ideas and 1 movie.
Firstly, the obvious choice to go with when it comes to clowns, we took inspiration from the movie “It”. Having watched the entire sequel I was immediately drawn to that creepy atmosphere where a clown is going after you but not only acts scary but also has the ability to make everything around you silly, but in an extremely creepy and unsettling way.
Therefore, conceptually and plot-wise, it was a crucial reference when it came to depicting scary clowns and creating successful jumpscares.
On the other hand, we used the VR game “Richie’s plank experience” as a pivot to our skyscraper/acrophobia level. The goal was to capture that same exact feeling of walking and desperately making sure not to look down. As a result, this game was not only conceptually necessary but also a big visual inspiration for our first stage.
Planks, fog, skyscrapers, immersive sound. All of these elements were essential to create something similar to this game. The only difference is that in Richies Plank’s experience, the user has to actually walk in real life, which results in more immersion and a heightened sense of fear. The fact that we will be using joysticks to move the first-person controller, means that will get lost. It is still possible to do. We would just need a huge field for the users to experience our game properly.
Then, we used the game FNAF (five Nights at Freddy’s) as inspiration for the gameplay and aesthetics.
Fnaf is a game made up of rooms(stages) where things start to show up as the player progresses. This was a good reference for the jumpscares in the clown scene, where things aren’t there but suddenly appear.
Additionally, the bright and dim lighting was an excellent visual reference for the clown and hallway scenes, where we really wanted to create that same contrast in order to place the player’s focus on the clowns or any frightening elements.
Finally, we took inspiration from the game Slenderman visually and gameplay-wise. The lack of light and the feeling of disorientation were very interesting approaches that we could play around with. Again, this was a major inspiration for the clown scene and not only in terms of lighting.
Even though we ended up not doing it, it would have been interesting to have the player hold a lantern while they are walking and have them get that same lost feeling.
Communication and Group Dynamic
Throughout the entire project, we kept in contact with each other and made sure everyone knew what was going on or what tasks they’d be responsible for. There were weekly meetings to keep everyone updated and talk about tasks and goals.
To do this, we used 3 major platforms to communicate and share our progress. Those were whatsapp, onedrive and Teams. We also occasionally used Miro, but mostly in the beginning to create a calendar and brainstorm ideas.
When using onedrive, we kept two major folders. One for sound and one for assets and anything related to game development. Each group (sound/VR) would be responsible for sharing files, and keeping their folder up to date and organised.
For our last game project, we had to briefly justify the use of VR in our games. I think it is equally relevant to do it now as well since we could’ve simply created a computer horror game instead. So what makes VR necessary in this project then?
Our goal here is to create an immersive experience by taking “2D” horror games to the next level. A simple computer game lacks real-life interactivity, the feeling of presence and physical sensation/movement. In order to get closer to simulating these, virtual reality becomes an essential component. By doing this, we get closer to real life and therefore, real fear, which is the main goal of all horror games.
For my portfolio, I chose to create a showreel. I recorded the work I thought would be relevant using Blender a screen recording. Then, used premier pro to stitch it all together. I included everything from 3D modelling to game development and motion capture. I think the selection was overall satisfactory, but I would like to have even more work there in the future. But I guess that is the natural progression of things. Overall, I am very happy with the showreel I created and I think it is a good reflection of my work and me as an individual.
Secondly, I created a website through Cargo and posted my work there. Meaning, this would be the platform I use to share my portfolio. I also included links to my Instagram, Youtube channel and Linkedin. My CV is also there, as well as my old projects that despite them not being the main focus of the website, I thought were still relevant to show my diverse set of skills. However, I didn’t include everything. I did a lot of graphic design that I would love to post as well.
Aesthetics-wise, I love the way the website looks. It is this sort of play on a website being a desktop and the work is accessible through icons that simulate those from computers. I thought this was not only unique but also a smart way to show how my work is related to technology and the fact that I’ve embraced digital mediums.
We were very happy with the final result! There are things that can be improved in terms of quality and overall polishing. However, it ended up looking very crisp and the switch from Blender to premier pro had little to no impact on the 3D effect of the characters in space. This was definitely our biggest concern, which actually became our saviour. Otherwise, having a rendered video would have been impossible. I also really appreciated the fact that we included a 2D smartphone experience for the viewers during our presentation. Even though it wasn’t with VR headsets we felt everyone was equally immersed in the experience and without a doubt added a layer of one-to-one experience that was more personal and interactive.
Final Corrected Version
After talking with Manos, I made a few changes to the video, mainly to the speed of the air character and the position of its video so that it wouldn’t leave such a big space between itself and the fire character. For the particle character, I tried to change its colour in After Effects but it wasn’t working properly because the character was too dark to be recognised as a separate mesh. I did follow a couple of tutorials but in the end, I couldn’t do much to change it.
Regarding the remaining characters, we didn’t change anything. The rendering would take too long and both Ana and I knew this wouldn’t be a realistic/necessary approach.
During our first group sessions, we went to LCDS and drafted ideas and started to record videos. We then experimented a bit with the AI Rokoko tool and created the very first clips that we used as a reference for this project.
Ai Rokoko software and limitations
We noticed right away that Rokoko Studio has some clear limitations. The animation looks very glitchy and the movement doesn’t look very natural or fluid. However, our plan is to use the mocap suits to get the best possible results.
Communication and Role Distribution
Our chosen method of communication was Instagram. We created a group chat and added everyone. This way, we were able to share our ideas, development and insights during break and whenever we didn’t have sessions together but still required each other’s opinions.
As for role distribution, we assigned the characters/elements to each of our group members as follows. The dance students will also be responsible for creating their own choreographies and we will leave all interpretation to them. Our role will mostly be the interpretation through aesthetics.
Between me and Ana, I would do the water, earth and post-processing, while she does air, fire and the human. Each one will be responsible for the rendering and after each model is complete, I will add all of them to the same file and render a 360 video.
The Characters and their Movement
For each character in our project, we wanted to depict a human interpretation of how each element would move. To achieve this, we let each dancer decide what element they would be based on personality. For example, Vee, the most extroverted and energetic person in the group became fire, which totally fits her overall behaviour.
Afterwards, we let them explore the movement for each element, as long as it made sense and was cohesive to its physical properties. Water should flow smoothly. Air should look free and light. Earth is solid and robust. Fire is wild and unpredictable. The human character would be up to the dancer because it could be a representation of themselves as a person and therefore, the representation of a human being.
Software and Tools
As I’ve previously mentioned, we are using Rokoko Studio to create some rough character animations. After that, we will export those to Blender and retarget the animation to a Mixamo character and we will then add all the effects and render everything. Finally, we will use Premier Pro to add the sound effects and any other post-processing elements we deem necessary.
Rokoko plug-in for Blender – Video Tutorial
Update
Another change of plans. I explain it in more detail below but we will also be using the Mocap Suits since they are finally working! The movement looks so much smoother and we can finally produce something that looks more professional.
Experiments
I started experimenting with my characters a bit. I followed some Youtube tutorials and adapted them a bit to what I thought looked better. For the Water element, the baking took half an hour and the rendering for a 15-second video took almost 2… I will not be able to do this many times so I have to figure out a way to cut down on the rendering time. Ana’s also had similar problems, unfortunately.
We have been discussing this and we think it would be better to render everything at LCC and split the work among 10 desktops. This way, the rendering will be way faster and hopefully, the desktops are more powerful than our laptops.
For the Earth character, the experimentation process was much smoother. For the body, I added some rock models I got from Sketchfab. As for the grass, I used the hair particle system, which created some really interesting-looking renders. I used another Youtube tutorial to help with this process which I will link below.
I’d also like to mention that after having watched and experimented with all these methods and tutorials, I realised I can pretty much attach anything I want. The particle systems are very intuitive and not as complicated as they look. I look forward to experimenting a bit more with these tools in future projects.
Youtube tutorial 1
Youtube tutorial 2
Green Room Recording
Thank God the Green Room was open and the Mocap Suits are finally working! We were able to record all the choreographies with the mocap suits and the movements look so much better! It was also super fun to see LCDS students enjoying themselves and experimenting with the mocap suits!
Final Models
Modelling the characters again was very easy and after having done all those experiments, everything felt intuitive and natural. However, I made some major changes to the Earth character. the previous model looked a bit weird and I wasn’t very happy with the result. I think it looks a bit silly… Therefore, I remade everything and instead of using the hair particle system for the grass, I used a grass 3D model from Sketchfab and used it as a particle spread throughout the character’s body. I also added a couple of flowers to make it a bit cutter and really get that feeling of nature emanating from him. Finally, I decided to use Mixamo’s character mesh and imported a rock/dirt-like material that worked very well aesthetically and really brought it all together.
The Water element pretty much remains the same. I changed the colour slightly and finally found a way to parent the liquid mesh to the character so that it moves as the character dances, without changing the height, meaning only x and y moved, or side to side.
Water Model Update
When adding the lights, I noticed I couldn’t really see anything because the background was black and the water, was transparent. After playing around a bit, I found a really cool solution. I attached two orbs of light to the character’s mesh by parenting them (just like I did for the mesh’s cube) and it created this really cool effect as if was part of the character all along.
Rendering Issues
Where to start… We finally finished all the character’s meshes and models, and after hours of work, we realised that we can’t render everything in one blender file. As soon as we opened it, the desktop crashed. We had to toss that idea and find a solution. At least if we wanted to keep the 360 video’s original idea.
I suggested we rendered each character individually, in their own respective file. And then, I’d stick all the videos together in Premier Pro by creating a 360 video and importing all the choreographies and 2d videos. By turning them to sphere objects, adapted for VR, this would technically be possible. And honestly, we don’t have any options left so this will have to do. Fingers crossed!
Time/Aesthetics update
After some discussion, we’re going to be doing 45 seconds of video. This is already a lot but we think it should be more than enough to showcase our work and still be manageable within the limited timeframe that we have. This is equivalent to 1350 frames at 30 fps.
Secondly, we had to settle on the same settings for all the characters. This means, the same camera angle, background and render settings. This was actually not a problem. Ana created the background, all black with a reflective floor just like we had planned. And I created the camera angle and format that became the standard for all 5 videos.
Water Rendering
This model gave me so many issues. For some reason, once I started rendering, the water would just fall. As in the water, the character would dance and all of a sudden the water decided to behave like water and become a puddle, even though I had baked the mesh multiple times. It was actually very funny but frustrating nonetheless. But this was due to me trying to render it in the same file as the other characters. We later figured that file was probably corrupted since it kept giving us weird renders and bugs.
Furthermore, the rendering was going to take too long, to the point LCC would have closed by the time it was done creating all those PNGs. I had to find a solution. After watching countless Youtube tutorials on how to reduce rendering time, I came across one that finally worked! I changed I couple of settings and took notes so Ana could do the same to her files.
Afterwards, I tried distributing the file between 2 desktops to cut down the rendering time. However, this was not possible. Due to the baked mesh and the physical properties I couldn’t simply render from one frame to a different one, since what result looked completely different and in the end I wouldn’t be able to stitch the sequence together. This meant I had to render everything in one go, sadly.
I gave it a go on my laptop and to my surprise, it was so much faster than the uni’s desktops! I am so glad I switched back to my laptop. It literally took 40 minutes to render the water animation. I was pleasantly surprised.
And with that, the first video was complete…
Earth Rendering
Since we had fixed almost all issues regarding the rendering process, this character was, once again, very easy to work with. I simply left it to render throughout the night, and once I woke up, it had finished processing my second character.
The second video was complete!
Human Character Issues and Rendering
Even though this was not my original task, we were having a lot of problems with rendering time still so Ana sent me the Human file so I could render it on my laptop. However, when I opened the file, once I started the animation, the character’s mesh would not move. Even rebaking it didn’t work. Then I noticed that the character was in the shared file (aka corrupt file) and since it had originally been modelled there, I had to redo it. Ana was rendering Fire and Air, which was already taking ages so I had to fix this issue because I’d finished everything else and my laptop was free to work.
I had to redo the entire particle system and added the same definitions Ana had picked. This was actually quite simple since the particles were just spheres with a randomized flowing path. Afterwards, I rendered it, Which took almost 6 hours…
But with that, the third video was complete.
Post Processing – Premier Pro
The final step was to stitch all the videos together. This was hands down the easiest step in the entire project and there is honestly not much to it. For the background, I simply rendered a 360 image of our blender background. Then, I added all the 2D videos and changed them from plane to sphere. Finally, we added the soundtrack and exported it as a VR 4k video and uploaded it to YouTube.
Sound
For the song, we wanted something more conceptual and abstract that was able to capture the fluidity as well as the robotic and somewhat technological aspects of the piece. We came across an extremely interesting track that, in my opinion, fit the video perfectly. There is a natural feeling that you get. However, the beeping in the background creates a mechanical/hospital-machine sound.
Presentation
I think our representation went quite well. All the group members that were present, were able to engage and talk about their ideas and roles through the project. I also think the fact that we had a final product to present, helped gain everyone’s attention. It felt as if we accomplished what we set out to do and we are really proud of that.
