Investigation Simulation - Interactive video & 360 tour

Forums: 

Incident Investigation is a simulation experience where OHS students investigate a workplace accident at a medium sized fictional brewery, with the intended learning outcome of producing a written report.  It was built by a digital designer for an academic teaching OHS at Federation University, Australia.  

The sim uses a combination of H5P interactive video and 360 Tour tools, actors on greenscreen, an animated interface and related documents and artifacts for the student to explore in a 30min experience. 

On launch, the user is presented with a phone and a big accept call button. The interaction required is obvious, and the caller introduces the scenario while also giving the player a call to action. 

To further guide the user, a screen of instructions is provided to explain additional controls:

The core game loop is to watch a video interview with a character, pose a series of questions to gain new knowledge, then write and review notes in the journal. Then onto the next interview and repeat the process.

It’s a simple, somewhat repetitive interaction but still manages to be more engaging than a passive video, and additionally reinforces the investigation process to the learner as they navigate through the experience. 

As more people from the workplace are interviewed throughout the experience and you learn more about the workplace, more tasks appear as your journal progressively fills itself in as new information is gained -  so a range of systemic issues become apparent, from HR and staffing issues to maintenance and process failures as you complete the story.  These notes will also then be used as evidence to formulate an activity at the end.  

Across the top of the interface is a slide out tab that gives access to useful documents that are handed to you from different characters you interview.  These can be accessed any time you’re in the main map between character interviews. 

 

 Here's a short video of the H5P IV in action : 

https://fedflix.federation.edu.au/media/H5P-Edited/1_537zgba5

I learned a lot about H5P when building this, discovered lots of workarounds and ways to approach different problems and present information clearly and more logically for the player.  Stringing multiple H5P content types together to load into the same iFrame was tricky at first but it worked fairly seamlessly.  IV is a very powerful set of tools and I was constantly impressed with what we could achieve. Lumi was used to quickly build and test the H5P content and Avidemux player was crucial to discover, and copy paste precise timecodes back into the IV.  Kaltura video server was used to serve MP4 video content directly as YouTubes "recommendations" when paused was too messy.  I also forced the iFrame to present to almost full screen in our LMS - it all works well and looks lovely as it's pushing a 1080p stream directly into player.      

A couple of features i'd love to see in Interactive video would be Muting (or pausing the video) of the H5P iFrame audio when window visibility changes (open new window or tab etc), as well as more target options when linking (blank, self etc) with the URL function, not just the text hyperlink option.  Being able to make hotspots that open in the same iFrame opens up a lot of game design options.  Also Skip / Jump in timecode without user input would also open up design options instead of requiring pausing and a button input.  

Thanks for taking a look at this project and happy to share more details if anyone has questions :) 

Eammon Jones

Senior Digital Designer | Federation University Australia

 

BV52's picture

Hi Eammon,

Thank you for sharing the great looking content. Also we will make to take into consideration your suggestions. You are also welcome to post a feature request, this way developers may take interest on your suggestion and develop them.

-BV

Thanks BV!  I'll make sure to search through the feature requests as others using these content types have probably come to similar conclusions after stress testing the tools :D 

-Eammon

Very impressive content indeed ! It seems very engaging and well built. Amazed to see that it is powered up by H5P that reveals its full potential !

Just out of curiosity : how long did it take to create the complete simulation ? Can you explain the different parts of your creation process ? Which part was the most time consuming (my guess would be designing screens, artifacts and making the animations ?) ?

Anyway, thanks for sharing this stunning content with the community ; this is very inspiring !

Best,

Isabelle

Hi Isabelle,

Thank you for the kind words, i'm glad you see it's potential!  It was roughly 8 weeks build time, but at least 2-3 weeks of that time was spent in testing and R&D to make sure it all would work. R&D could be mostly discounted on future projects as we now have a strong idea how to approach it, we've overcome tech issues and developed best practice for video presentation in this medium.     

The sim effectively had a zero budget (beyond my time at my desk) so that in itself posed a number of challenges. Almost all production work was completed by myself, and we provided guidance and assistance to the subject matter expert to write the 12 page scenario/script with 6 characters.   

Creation process

The initial meeting with the academic was to produce and film a very typical passive video scenario of the process involved to perform an accident investigation.  Maybe a 10-15 min video with a setup, investigation and end.  During this discussion it occured to me this type of process was a lot like a detective investigation : That is > interview a person > ask question > get response > repeat with next person.  This didn't feel very compelling content  as the interviewee does most of the talking and it's less conversational.  Bit boring really to watch.  So I suggested that a branching approach might be way to go.  The academic also really wanted to include a 360 tour of the accident scene so I had to work that into the design as well.  

Game and Story: 

Articulating game mechanics to a 'non-gamer' Academic who was writing the story was quite tricky without having example past projects to demonstrate how it all worked. So I sketched out a rough mind-map in a Teams whiteboard to help convey how I think the player might interact and play through this interactive. 


While this intial sketch is a lot less linear than the final product, quite a few of those initial ideas made it to the final product such as the phone as an interface for more information.  The notes on the pad that develop over the duration of the sim was probably the single key in the design to really scaffolding the investigation process to the students. That's not present in this initial mindmap and evolved iteratively over the early build process.    

Script:

Once we'd managed to agree how the interactive would function a template script was developed for the Academic and they created characters and wrote the story.  The Academic had a strong understanding of using scenarios in the classroom so it was fairly straightforward to translate into this new format. She also didn't have any screenwriting skills so with some guidance and a template to get started she was fine after that.   

Production:

In past digital narrative productions at my institution, we've had budgets to pay professional actors (and a screenwriter) as well as ramp up a larger film crew when required. There was no money available so we had to convince a bunch of awesome co-workers to perform in front of camera.  The lack of acting chops is usually an issue when pages of conversational dialog need to be learned and recounted on screen.  However...

 

...since we'd designed the player to be a "silent protagonist" with no dialogue only on-screen text questions (using crossroads in IV H5P), this meant our amatuer actors only needed to learn very short paragraphs of lines, directed at the camera lens and on greenscreen.  Very achievable for most people, even if a number of takes were required to achieve a resonable performance. 

The greenscreen videos were composited in Premiere and the locked off camera was given a bit more motion and life using the free Jarles camera shake presets that were made for the movie Deadpool. Interestingly during testing I found that adding a very brief “blur and 50% darkness” transition on the characters video just before the pause for the branching question did help smooth over the jump as time moves forward or backwards during navigation.  It felt jarring without this transition - could be a great future H5P feature to add actually! 

Animation:

This small item might have been the only cost-item : a few credits for the Brewery stock image from shutterstock, though free ones do exist on pixabay and pexels. I went with an simple isometric vector design that functions as a kind of map of the brewery site, and also serves the navigation and other interface items. Alternatively this could just as easily been a aerial drone photograph of a factory site, or sketched map.

 

Animations for the titles and instructions, map interface, phone conversations and notepad were all made in Adobe After Effects. While animation adds production value, the use of still images for these items instead could easily perform a similar job on a cheaper time budgets. 

Time allocations:

Post-production on the video component and the H5P navigation were probably the most time consuming part of the project.  Roughly 25mins of composited character videos to polish up, and adding precise time-code for all the navigation with the Crossroads function did take some effort.  Lumi and AVIdemux helped with this as building directly into the LMS was a bit cumbersome.  

There’s lot’s more to discuss but I’m waffling now.  Thanks for reading and happy to show more specific detail if anyone is still interested.

Cheers,

Eammon

(sorry i broke this lengthy response into two parts with an edit so the secpnd half is waiting to be approved by mods.  Thanks mods! :)