Introduction

Got a basic version of pizza recipe work.

Progresss

(Xuan) Paired with Greg to implement timer. It is triggered when the current step finishes. It shows the count down. When time is out, it will make a spatial sound and show what need to be done next. Fix aspect ratio on video. Got chess team's plugin running in our project as backup for Holovideo. Spatial audio alerts. Put things together and make the final steps with the help of whole team. Got the videos, images and sounds to create the final steps. StepManager is modified a little bit to ease the process of creating the steps.

(Greg) Paired with Xuan to implement timer. Minor polishing on recipe content. Video shows preview image. Helped others debug code and integrate/merge code. Found more readable font; partially implemented. Study timing and framing for final demo.

(JJ) Worked on making the Pizza Tossing Holovideo work on HoloLens. Made experiments to fix the problem of jittering of the video on one eyes. Lowered the speech recognition confidence level to make interaction smoother. Generalized the step class so that it can be inherited. Study timing and framing for final demo. Ordered training dough.

(Griffin) Added next step indicator. Added new pointer that uses a standard model and correctly sets the stabilization plane. Fixed rotation for content in spatial registration and all steps. Study timing and framing for final demo.

(Team) Integrating our features. Ideating for final demo. Trying to debug HoloVideo (mainly JJ). Logisitics for final demo. General discussion of design and features. Architecting features.

Challenges

The HoloVideo contents, when played with the given SVF plugin from Microsoft, jitters all around and is, overall, unreliable. We spent too much time on experiments trying to debug what we seemed could be mistakes on our end, only to find out that the plugin is buggy. We confirmed this with the Microsoft team, and they are currently trying to fix the bug (a race condition in the renderer). As a backup plan, we have started testing and using the Chess team's renderer on our machines. Switching to that will involve extra effort. We hope to get the plugin fixed before the demo.

Plan for Next Week

Adding polish and running the demo with other people to validate it.

(JJ) Keep working on getting HoloVideo working. Work on polishing recipe content and layout.

(Greg) Prepare logistics for demo (get ingredients, furniture, detailed sketch of demo). Add music. Work on polishing recipe content and layout.

(Griffin) Next step navigation improvements (walk to next step). Video clip timings. Work on polishing recipe content and layout.

(Xuan) Pick up Holovideo assets from Microsoft in person. Work on polishing recipe content and layout.

Demo Plan

  • Person from team gives short verbal framing of the application and what player will do. They will be instructed on following the pizza tosser and moving around the videos playing at different speeds as they feel comfortable.
  • Player puts on the hololens
  • Italian music starts playing, fading in gently
  • Player sees 3 Holovideos looping in sequence. They will be put in different parts of the space. They will be play at different rates to make it easier to follow.
  • Player is instructed to try doing it with the training dough. Player does this for about 20-30 seconds. Then italian music fades out and new music starts playing at the next step.
  • Player goes to the Thai summer roll making area. They see the overview step saying "Let's make some Thai summer rolls! Walk to the next step to begin." with a picture of a summer roll.
  • Players sees a step to use some hand sanitizer and "Let's get our hands ready for cooking."
  • Player takes the ricepaper wrap out and dips it in warm water. Player sees a video and is instructed to follow the timing of the video. The video loops.
  • Player picks up the ricepaper and moves to the next step. They place the wrapper down on a wooden surface. The ingredients (noodles, carrots, cilantro, lime) are already there.
  • Player sees a short video placed on the counter with the same size as the real items. It shows a person putting the ingredients on. There is also an example picture of what not to do, not to overstuff the rolls. There is also an image on the wooden board that shows boxes for about how much of each ingredient to add. Player also sees text telling them to say "next step" when they are ready to fold.
  • After the voice command, the player sees a video showing how to fold the roll. The player sees text saying "When folded, place the wrap on the plate and you're done!"
  • When they get to the plate the text "Enjoy or share with a friend!" shows up and that ends the experience.
  • A team member prompts them to take off the Hololens. Then a team member prompts them to try it, asks them how it tastes and what they thought.