Tag: kinect

Blog Entry – Research and Project Updates 3

Posted on by Matthew

This week of research has been held up and on delay because of a delivery issue. I purchased a Kinect v2 that has all the accessories required to run on the PC. Unfortunately, it was shipped across to the other side of the country. As I write this, I should be receiving the items tomorrow, 9/19/18.

Nevertheless, I decided to create a Unity project and started importing the required plugins. I also installed the Kinect SDK for Windows 10. I got it all imported and installed without a hitch.

I did see an issue about a script called joints as Unity has a script the same name. It shouldn’t be an issue to update the scripts and the calls. Unity did want to update the plugin scripts using deprecated calls. I let Unity do its thing. Hopefully, I don’t regret doing that. I opened an example scene and ran it. Of course, it ran but it didn’t throw an error! Be that good or bad news, who knows until I plug the device in.

FYI – anyone that has Kinect running on a computer or laptop that switches gpus, by default when rebooting/starting Windows 10, it will always switch on the dedicated GPU. Without the NVIDIA notification icon, I would be wondering why my fans are constantly running and battery life that is trash. Easy fix is to end the tasks in the task manager. I haven’t found a way to stop it from being started by default. I will have to look at it some more but I fear if I mess with, I will cause more harm then good at this point. So I am going to hold off until I can test the Kinect.

I am excited to play around with the Kinect. Not only because of the project but also to expand the use of the device. I am curious if I can create animations using it as well as scan objects. If so, I want to use some of the features to help build the art around my project to help speed up the production.

Barber Shop Video

I remember something like this video at Disney World Hollywood Studios. It was called “Sounds Dangerous! Starring Drew Carey”. It was closed in 2012 and replaced by numerous different shows but the audio show had the same idea about the binaural audio. It used Dolby as the audio driver. Essentially, you watch a video while having a headset on. The lights would turn off and the audio show begins. There is a part where Drew Carey goes to the barber and he gets a haircut. The video felt and sounded the same as I remember the show did.

In the video where they put a bag over the microphones, I felt like there was a bag on my head. I started to panic and opened my eyes. I am surprised how well the illusion works. The buzz cutters sound sounds life like as well. I am curious how many games and movies use this audio technique. I feel like it would create a better immersion into their worlds. Would it work with a surround sound system or only with headphones?

 

Meow Wolf

Meow Wolf is an interesting place. At first, it looked underground and run down but now they have expanded so much! It is quite impressive. I like the idea where they expand to other cities, and have local artist work on pieces of art to display in their “museum”. I use “museum” loosely.

It looks more like a take on a modern “museum”, where the rooms are the pieces with pieces within the room’s world. One piece that was shown was the black and white room. I would have liked to see more of that piece. It reminded me of some comic book or something Tim Burton would make. Another thing that caught my eye was the “doors” to the other rooms. One was a fireplace where you would walk through. Another was a hole in the wall where you would crawl through. I am assuming there are regular doors but experiencing these “gateways” would help create the illusion. The illusion that you are in the art piece. Someday, I would like to visit one of these places. Not only for the experience but to be inspired by these “rooms”.

Blog Entry – Research and Project Updates 2

Posted on by Matthew

Setup Layout: (possible)

My idea for implantation for the Kinect in my project:

  • Audience / viewer approaches or walks by
  • Kinect detects and pulls data into Unity
  • Unity scripts will separate video and audio data
    • Video data will be face, color, depth, skeleton and # id
    • Audio data will listen for keywords and direction
  • Data will be push to a preset list of options
  • Options will be prioritized and sorted then picked upon
  • The combined option data will be passed to an animation to play that animation/audio
  • Once animation is completed return to the begin or have the new segment cached waiting until animation is done to play the new animation sequence

“Kinect is the eyes and ears of the piece.”

                  These features of the Kinect are important because it the eyes and ears of the piece. Without the Kinect, the piece will be static and non-interactive. It won’t have the life and emotion that I am striving for. For example, facial recognition will help the piece because I plan to use it to receive important data points.

One way facial recognition data would be useful is by determine if the audience or viewer is focused on the piece. Another would be by determining what facial expression they are making. I could use that data to point to and select the best animation sequence. Another way I could use facial recognition would be to determine when to start listening for audio input. I could use facial recognition to help determine gender if I needed too.

Color data is another useful input for the piece because I could try to determine the color of the viewers clothing, eye colors, and so on. I could even capture the image to save it and display it on the display if I wanted to add another feature. (< possible addon)  Another data capture would be interesting is by using the skeleton tracking.

Skeleton tracking could be used as a confirmation if the user is interacting with the piece or not. It may also be used to determine how far the audience is from the piece. I could try to call over the audience if it is determined they are too far from the piece. If it works, it will help the facial recognition feature by moving the viewer closer in. That way, I can receive better and more accurate results to process. I may have to combine it with id tracking but, from my understanding, I could also determine how many people are standing in front of the piece. I think the total can only be 6.

With all these features, I believe the piece will come alive. These fine details, when combined, should “wow” the audience. Either, they will be excited or creeped out by the piece. Either way, they will have an experience! The next step in the journey is testing the Kinect and getting the plugin to work to within Unity as the plugin is quite old. Nevertheless, I am excited to work the Kinect.