This week’s sprint required me to detect if a person is present in the scene, and to detect/track a face and the face features the Kinect sensor registers. I completed it using the Kinect v2 Examples with MS-SDK and Nuitrack SDK plugin for unity. There are two main necessary scripts to use. One is KinectManager.
KinectManager script is required for the Kinect to initialize and run. It also contains numerous classes that are important for body tracking and could be useful later in the project. The other script is called FaceTrackingManager. This script is like the KinectManager but only manages the face and head of the user. With these scripts, it is easy to debug any issues. For instance, I have it set up to create a small window with a color map video. In that video, right now, displays a rectangle box around my face. It enables me to see if the face tracking is working. With those two scripts working, I edited a script that was a demo. That script was only to detect a smile. I added in all the face properties Microsoft allows. I should be able to call the variables once I start creating my database. These properties should give a direction to what script/animation needs to be played at a given time.
Next week’s sprint:
Get speech recognition to work with some basic phrases. Then create a script to call all the input data and display them.
Also, as I mentioned above, the KinectManager has some useful classes so I wrote them down. Later, I discovered a website that has it all indexed: