Setup Layout: (possible)

My idea for implantation for the Kinect in my project:

• Audience / viewer approaches or walks by
• Kinect detects and pulls data into Unity
• Unity scripts will separate video and audio data
  • Video data will be face, color, depth, skeleton and # id
  • Audio data will listen for keywords and direction
• Data will be push to a preset list of options
• Options will be prioritized and sorted then picked upon
• The combined option data will be passed to an animation to play that animation/audio
• Once animation is completed return to the begin or have the new segment cached waiting until animation is done to play the new animation sequence

“Kinect is the eyes and ears of the piece.”

                  These features of the Kinect are important because it the eyes and ears of the piece. Without the Kinect, the piece will be static and non-interactive. It won’t have the life and emotion that I am striving for. For example, facial recognition will help the piece because I plan to use it to receive important data points.

One way facial recognition data would be useful is by determine if the audience or viewer is focused on the piece. Another would be by determining what facial expression they are making. I could use that data to point to and select the best animation sequence. Another way I could use facial recognition would be to determine when to start listening for audio input. I could use facial recognition to help determine gender if I needed too.

Color data is another useful input for the piece because I could try to determine the color of the viewers clothing, eye colors, and so on. I could even capture the image to save it and display it on the display if I wanted to add another feature. (< possible addon)  Another data capture would be interesting is by using the skeleton tracking.

Skeleton tracking could be used as a confirmation if the user is interacting with the piece or not. It may also be used to determine how far the audience is from the piece. I could try to call over the audience if it is determined they are too far from the piece. If it works, it will help the facial recognition feature by moving the viewer closer in. That way, I can receive better and more accurate results to process. I may have to combine it with id tracking but, from my understanding, I could also determine how many people are standing in front of the piece. I think the total can only be 6.

With all these features, I believe the piece will come alive. These fine details, when combined, should “wow” the audience. Either, they will be excited or creeped out by the piece. Either way, they will have an experience! The next step in the journey is testing the Kinect and getting the plugin to work to within Unity as the plugin is quite old. Nevertheless, I am excited to work the Kinect.