Thesis I – Project Blog 10

Lipsync and eye controller added. All blendshapes are done. I wish to add wrinkle maps but currently that has been put on the back burner. Head turns, eyes blink, and lipsync shapes is about 50% done


Working on a procedural car generator. Although I will be reworking the concept to incorporate multiple curves and use sweep or rails.


Background

                        Theme parks are places where families go to let loose and be entertained. They are full of creative art pieces from the overall themes to the attractions themselves. Many artists, designers, and engineers has spent years working on designs to immerse audiences. A great an example is at Universal Orlando Islands of Adventure. There is an area that is dedicated to the Harry Potter universe. As soon as you walk through the entrance, you are immediately immersed. Everything from the ground, the buildings, the shops, sounds and the rides are designed in a way to mimic the books and movies. Places like this can ignite and spark creativity. For me, was the place that inspired my project.

Anyone who has visited any amusement park or resort will agree waiting in line is part of the experience. Knowing this, creatives who build the attractions, design sets that try to distract you from the wait time. One environment that stood out to me was in the Harry Potter and the Forbidden Journey ride at Universal Orlando. As you make your way through Hogwarts Castle, you come to a room with a tall ceiling. Within this room are paintings that are assorted on walls above the visitors. If one stood and observed, they would notice these portraits are not ordinary paintings but are animated. They move and talk among themselves. Fooling the visitors into believe they are alive. Seeing this I was amazed. Not only by the presentation and immersion but how people, including myself, were reacting. Visitors were standing around watching and interested in the character, but as they returned, they already knew what to expect. They have lived the experienced. The immersion fell apart because of the repetitions and limited interactions. This problem led me down a tunnel of questions. What would take this concept to another level? What if the paintings saw a person? Or if they responded back to the visitors? What if they would drove a story so each person would have a unique experience every time they visited? How would the audience react? Would they feel for the character? These questions brought my project to life.

Designing my project, I knew I desired to have a painting that could interact with the audience. I wanted a character that would be able to seek and determine the presentence of a person and try to communicate with that person. My curiously lead me to see if I could develop an interaction that could allow individuals to emotionally connect with an artificial being. This was a challenge considering the scope of the programming requirements needed and the limitations of the hardware, but I knew it was possible. As I progressed with the project, I started to see something interesting that happened. Each element of the character became more like me. Visually, the character is an old man, but his personality, intentions, expressions and message are mirror of myself. This discovery encouraged me. It gave me the freedom to communicate a message that could directly affect a person emotionally. As well as giving an individual a unique experience with this character. Although I was startled how much of me is within the piece, I have accepted the result. It brought my own mentality and personality to my attention. In turn, allowing myself to design an experience were the audience can create their own thoughts about the character without knowing they actually met a part of myself.

Thesis I – Project Blog 5

This week, I worked on my studio project. My task was to start the blendshapes and get most done. I got most created and have about 12 left. I plan on working this week on wrapping it up and importing it into Unity and attaching LipSync. Also reminder to me: both top & bottom teeth are not UVed. Copy UVs from the combined game mesh for the final piece.

Here are some tests:


ROUGH ROUGH DRAFT of defense 2: Social cognition

Defense 2: Social Cognition

            Anthropomorphism and theory of mind intertwine with one another. As noted earlier, they trigger the temporoparietal junction in our brains highlighting the connection between, but anthropomorphism goes further than theory of mind. The phenomena can affect our cognitive function that processes how we perceive others, and how we view the surrounding world. (Cherry, Social Cognition in Psychology) Psychologists who study this process called it social cognition.

Social cognition is the “mental processes that (are) involved in perceiving, remembering, thinking about, and attending to the other people in our social world.” (Cherry, Social Cognition in Psychology) These processes influence how people behave, form impressions, and react to others. Simply put, it is the way we store, process, and interact with other living and non-living entities. Many things can encourage the develop of our social abilities such as culture, ideologies, experiences and so on. With anthropomorphism in mind, this cognitive ability grants humans an efficient way for our minds to process social stimulus that engulfs us every day. Predicting, and comprehending is important to us hence why we anthropomorphize the unknown. It puts the current unknown entity into context reducing uncertainty and allows for a simple way to communicate and socially reason to others about them. (Atherton) Therefore, anthropomorphism allows for an efficient way to socially predict and comprehend. Unfortunately, this mental efficiency can come at a cost.

Visually, we are highly sensitive to humanlike characteristics. (Agnieszka Wykowska) Our awareness of the difference between living and inanimate is impressive. As earlier, I explain how we naturally favor smooth human-like movements, called biological motions and how we are born with facial recognition but even with those traits, we make assumptions on what we observe. (Susan R. Fussell) When we interact with entities, we assign categories based on stimuli such as facial features and shapes, body shapes, gender, etc. When humans cannot understand the actor, we unknowingly rely on stereotypes we have learned to process their behaviors, and traits. (Susan R. Fussell) As the actor becomes more similar in appearance to oneself, the more we are likely to reference ourselves and anthropomorphize the non-human entity. (Atherton) Either anthropomorphizing or not, we judge because it allows us to create make assumptions. This assumption allows for forecast of behaviors of the entity and the environment that we are surrounded in. Another important aspect of social cognition is the need for connection that we draw from when we interaction with others.

Human beings desire and crave for some type of interaction that has the potential to produce a connection, emotionally and/or physically. From infants to adults, the social connection between others is a survival instinct. Plucking a person out of their social life, or rather, banish an individual would be a “death sentence for one’s genetic inheritance”. (Atherton) This type of punishment can have an enormous effect to an individual. Lacking or being withdrawn from social connections can influence and trigger the same neural functions that can induce physical pain. (Atherton) This pain may lead the individual to seek for a connection elsewhere. Requiring the person to seek a social connection and will start to anthropomorphize nonhuman entities to satisfy their lust. This basic motivation has been researched and found out that the lonelier a person is, the more they have connected with anthropomorphize objects. (Adam Waytz) Satisfying our need for a connection is important whether its fulfilled by living or nonliving entities. We need “someone” to survive.

Anthropomorphizing has its benefits. It allows us to be able to predict, to comprehend, and to make assumptions about the uncertainty. Lonely individuals that are deprived from social connection and interaction will anthropomorphize and feel at ease. Anthropomorphizing, to me, is about efficiency. This efficiency enables us to expect future actions from others. Removing the uncertainty that we as humans try so hard to unveil and learn what lies within.

 

Works Cited

Adam Waytz, Nicholas Epley, and John T. Cacioppo. "Social Cognition Unbound: Insights Into Anthropomorphism and Dehumanization." Psychological Science. 19 February 2010.

Agnieszka Wykowska, Thierry Chaminade, and Gordon Cheng. "Embodied Artificial Agents for Understanding Human Social Cognition." Philosophical Transacations B, 19 February 2016.

Airenti, Gabriella. "The Development of Anthropomorphism in Interaction: Intersubjectivity, Imagination, and Theory of Mind." (2018). <https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6231421/>.

Atherton, Gray and Cross, Liam. "Seeing More Than Human: Autism and Anthropomorphic Theory of Mind." (2018). <https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5932358/>.

Cherry, Kendra. How the Theory of Mind Helps Us Understand Others. 26 July 2019. <https://www.verywellmind.com/theory-of-mind-4176826>.

—. "Social Cognition in Psychology." Very Well Mind, 13 August 2019. <www.verywellmind.com/social-cognition-2795912>.

Drubach, Daniel A. "The Purpose and Neurobiology of Theory of Mind Functions." Blanton-Peale Institute, 18 December 2007. Online.

Karolina Zawieska, Brian R. Duffy, and Agnieszka Spronska. "Understanding Anthropomorphisation in Social Robots." Pomiary Automatyka Robotyka. November 2012.

Leslie, A.M. "Theory of Mind." International Encyclopedia of the Social & Behavioral Sciences. Elsevier Ltd, 2001. <https://www.sciencedirect.com/topics/neuroscience/theory-of-mind>.

Nauert, Dr. Rick. PsychCentral. 15 June 2019.

Nicholas Epley, Adam Waytz, and John T. Cacioppo. "On Seeing Human: A Three-Factor Theory of Anthropomorphism." Psychological Review. The American Psychological Association, 2007.

Susan R. Fussell, Sara Kiesler, Leslie D. Setlock, and Victoria Yew. "How People Anthropomorphize Robots." Carnegie Mellon University, 12 March 2008.

Thompson, Brittany N. Psychology Today. 03 July 2017. Website. <https://www.psychologytoday.com/us/blog/socioemotional-success/201707/theory-mind-understanding-others-in-social-world>.


 

Thesis I – Project Blog 4

Below are some videos of the final particle fx for the Procedural Art course. I decided to work on some effects referencing Battlefield 5. The main requirement was to create sprite sheets from Houdini and then make the effect in Unreal 4 engine using cascade. There was some difficulties with crashing and render times but even with those issues, I believe they came out well.



 

I added finish skin weighting the character and he is now imported into the screen. Prior to this, I created a zbrush file for creating blendshapes. Which is my plan to start work on this week.


Defense 1: Theory of Mind

To anthropomorphize something, we humanize that entity. We give it human characteristics, be it emotions, intentions, or thoughts. During that transformation, our brains begin to process and comprehend those distinct human traits. This cognitive process is called Theory of Mind. First reported in an article by psychologist David Premack. (Leslie) It later became a term psychologist use to describe the cognitive process that gives an individual the ability to comprehend another’s emotional and mental statues. This cerebral function starts to develop around age of 3-5. (Cherry) However other research has shown it could develop earlier or could be delayed depending on the individual. This ability to theorize or predict another’s state of mind – thought, emotion, and behaviors, is the most important function for survival for human beings. (Drubach) Being social individuals, we commutate in such a way to comprehend other’s intentions, thoughts, and feelings. (Nauert) As we do this, areas of our brains such as the temporoparietal junction activate. Additionally, anthropomorphizing activates the same area and the more a person anthropomorphizes, the larger the areas of the brain are for Theory of Mind processing. (Atherton) Predicting and theorizing, our brains never rest. Humans continuously try to make sense of everything around them, especially with motions.

Instinctually, we humanize non-human actors to predict their behavior, but we also anthropomorphize motions. Research has shown that processing and recognizing biological motion contributes to the awareness of animacy. (Atherton) Recognizing motion is an instant flow of information which allows humans to predict and identify an actor’s behavior. Humans expect to observe smooth human-like movements as oppose to erratic motions. Researcher also found that early in development, infants prefer biological movements over artificial and by the age of 2, prefer human motion. (Atherton) Example of biological movements would be objects moving in a coherent manner with respect to one another. (Airenti) Using movements alone, we can start to understand actor’s intentions. We interpret the two objects as interacting with one another. As both objects could understand one another. Instantly we begin to anthropomorphize the objects by assigning unique roles to each one. These anthropomorphized motions allow for easier recognition and predictions. It allows humans to theorize about what might the objects do next. As important as biological movement is to Theory of Mind, there is another process we anthropomorphiz to help interpret. This clever process is called facial recognition.

Face processing is part of the cognitive process in which humans can determine what a person is thinking when observing a facial expression. As infants, we naturally develop skills to determine faces and mimic facial expressions. Whether a face is familiar or not, we can immediately tell one face apart from another. As humans develop, this constant stimulus will train a person to specific facial shapes and emotions (Atherton) which enables us to see key differences in faces. An example of anthropomorphizing a face is when we humanize our pets. For this instance, humanizing the face of a pet. They would express anthropomorphizing terms to explain the facial expressions of it. Such as the pet is smiling or happy. We assign them behaviors and emotions. Using such vocabulary helps us instantly recognize the other entity’s status without ever needing to see it in person. This grants us the ability to simulate and mimic the experience using our imagination.

We often use our creativity to dream up anything that our hearts desires and anthropomorphism is a unique part of this. Inadvertently, we trick our brain into believing these non-human entities are other persons. This creates a sense of familiarity and predictability that we know and need. This feeling of ease allows us to use everyday strategies to determine what the other’s motivations are and enables us to predict future behaviors. Humanizing non-human actors can go a long way as anthropomorphism and Theory of Mind triggers the same part of ours brains. Human need predictable communication and environments. We strive to make sense of the purpose of other’s goals. If we are unable to comprehend any situation, we will start to anthropomorphize. As soon as this happens, the humanized situation becomes easier to accept, comprehend and predict.

 

Works Cited

Airenti, Gabriella. "The Development of Anthropomorphism in Interaction: Intersubjectivity, Imagination, and Theory of Mind." (2018). <https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6231421/>.

Atherton, Gray and Cross, Liam. "Seeing More Than Human: Autism and Anthropomorphic Theory of Mind." (2018). <https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5932358/>.

Cherry, Kendra. How the Theory of Mind Helps Us Understand Others. 26 July 2019. <https://www.verywellmind.com/theory-of-mind-4176826>.

Drubach, Daniel A. "The Purpose and Neurobiology of Theory of Mind Functions." Blanton-Peale Institute, 18 December 2007. Online.

Leslie, A.M. "Theory of Mind." International Encyclopedia of the Social & Behavioral Sciences. Elsevier Ltd, 2001. <https://www.sciencedirect.com/topics/neuroscience/theory-of-mind>.

Nauert, Dr. Rick. PsychCentral. 15 June 2019.

Thompson, Brittany N. Psychology Today. 03 July 2017. Website. <https://www.psychologytoday.com/us/blog/socioemotional-success/201707/theory-mind-understanding-others-in-social-world>.


 

Thesis I – Project Blog 3

Thesis Paper Outline – Matthew Hopkins

  • Quote – “We keep moving forward, opening new doors, and doing new things, because we’re curious and curiosity keeps leading us down new paths.” – Walt Disney
  • Intro (Abstract) – summary of what the paper is about
    1. Anthropomorphism
      1. Where it started
      2. Where it used
      3. Where it is going
    2.  Project
      1. Relates to combing anthropomorphism and A.I.
      2. How artist can easily communicate using the combination
  • Thesis Statement
    1. Interacting with anthropomorphic A.Is can create and intensify participators’ connections, emotionally and physically, towards synthetic beings. Giving an artist the flexibility to communicate directly to their audience by manipulating the human psychology.
  • Background (Intro) – reason why I made my project
    1. Experience
      1. My background & skills
      2. Opportunity to create an interactive installation piece
      3. Learning in skills and tools
    2. Inspired from Disney and other productions
      1. Universal Studios – Harry potter ride attraction with talking walls
    3. Being able to watch participators and visitors reactions
  • Tech Background – describe hardware & software works if they did not understand
    1. Unity, zBrush, Maya, Blender, substance painter
    2. Lipsync – facial animations
    3. Excel – story progression datasheet
    4. Kinect v2 – input device to capture data
  • Defense
    1. Social Cognition
      1. *
    2.  Technopomorphism/Personification
      1. *
    3. Theory of Mind
      1. *
    4. Uncanny Valley
      1. *
  • Description of Project
    1. Overall/Summary
      1. “Project Frame” is a dynamic interactive experience featuring a hanging oil painting that responses to visual and audio inputs. It will detect the presence of the viewers, follow faces and body movements, and listen for speech responses. All of which, will advance down a fixed “script” that the A.I. continuously references. The final presentation will consist of an environment that will be build around the frame for maximum immersion.
      2. Story
        • Pen name* loved his work more than anything, so much so, that his soul was bound to his greatest piece of work, his portrait. Participators can interact and learn from this broken soul. If they are lucky enough, then they can discover his own lives story
    2. Design
      1.  Character
        1. The true meaning behind the man in the painting
        2. Technical story map *picture*
      2. Art style
        1. References
          1. Victorian era
          2. Dishonors and other video games
          3. Realistic stylization
        2. Installation
          1. Layout
          2. Setup
          3. Design
        3. Behind the scenes – Code
          1. Special functions
            1. Interaction data
            2. Interaction mood level
        4. Response from people
          1. Hypothesis
            1. Creeped out, Curious, entertained
            2. Create emotional connection
          2. Results
      3. Reason I choice the decisions I made
        1. How I started on the project
          • Hardware testing
        2. Everything, besides hardware, is designed around the main character
          • Story, background art, interactions
    3. Conclusion

 

 


Procedural Art – FX project WIPs

https://drive.google.com/file/d/1pexePetINLsZPUIOR1SMy9eQnfIoIPrS/view?usp=sharing

https://drive.google.com/file/d/1xvwiY5N2ti35Y7dean3LbYtoScr452yK/view?usp=sharing

Character progress

Done this week:

  • the UVs combined
  • character 50% rigged – needs to accessories and hands weighted
  • blendshape readys to be sculpted – zbrush file specially made for the character’s head
  • Throw textures together to get a base maps

Thesis I – Project Blog 2

Character background meeting

  • Alone does not like people
  • Needs people to survive and thrive
  • Grumpy / Scroogey
  • He knows he is in the painting
  • Has regrets – like missing out on his family because of work, past experiences
  • Likes to talk about his perspective of what he sees from as a painting
  • He is an Artist! Who loves his work, the painting, so much his soul is attached to it
  • Goes by his pen name

 

  • Take away message of the painting – Take a step back from work every once awhile and make time for yourself & your family.
    • Like a warning to visitors through the interact
  • Based locally in a distance Victorian era

Background Art

  • Canvas painting in BG
    • Happy – painting of family
    • Sad – distorted self portrait
    • Uses the same canvas but distorts and blends between the two

 

Thesis Topic:

Interacting with anthropomorphic A.Is can create and intensify participators’ connections, emotionally and physically, towards synthetic beings. Giving an artist the flexibility to communicate straight to their audience.


RESEARCH NOTES

Anthropomorphism research

 

https://psychcentral.com/news/2018/03/01/why-do-we-anthropomorphize/11766.html

Giving human characteristics to animals, inanimate objects or natural phenomena is a human trait called “to anthropomorphize.”

The term anthropomorphism was coined by the Greek philosopher Xenophanes when describing the similarity between religious believers and their gods

Neuroscience research has shown that similar brain regions are involved when we think about the behavior of both humans and of nonhuman entities, suggesting that anthropomorphism may be using similar processes as those used for thinking about other people.

Human brains are tuned to try to understand other human’s intentions, thoughts and feelings. This concept is called Theory of Mind.

Specific regions of the brain contain populations of ‘mirror’ neurons are the same regions of the brain that are active when a person is anthropomorphizing.

Predicting the actions of animals and inanimate objects employs the same brain regions as predicting the behavior of another human. Though we can consciously differentiate between human and non-human, the same mechanisms in our brain are activated when we are observing actions of both.”

What accounts for this selectivity? One factor is similarity. An entity is more likely to be anthropomorphized if it appears to have many traits similar to those of humans (for example, through humanlike movements or physical features such as a face).

Various motivations may also influence anthropomorphism. For example, lacking social connections with other people might motivate lonely individuals to seek out connections from nonhuman items. Anthropomorphism helps us to simplify and make more sense of complicated entities.

https://www.theguardian.com/science/2016/jan/15/anthropomorphism-danger-humans-animals-science

Patricia Ganea, a psychologist at Toronto University, ran a series of experiments on three- to five-year-olds in which they were given information about animals in straight factual form and then in a more fantastical anthropomorphized way.

children were likely to attribute human characteristics to other animals and were less likely to retain factual information about them when told they lived their lives as furry humans.

attributing human-like intentions and beliefs is a “very natural way to explain certain animal behaviors”

lead to inappropriate behaviors

https://www.sciencedirect.com/science/article/pii/S0003347215003085

Proposed to be a result of a cognitive default state. The main idea behind this hypothesis is that the human brain evolved to efficiently process social information.

automatic response to any human-like behaviour (Caporael & Heyes, 1997) or human-like feature (Guthrie, 1997) that requires a swift identification or interpretation, which cannot be accounted for using the knowledge at hand.

Epley, Waytz, and Cacioppo (2007) proposes that anthropomorphizing has strong motivational triggers, particularly effectance and sociality.

The first is described as the need to make sense of the actions of other agents to reduce uncertainty concerning their behavior

he second refers to the need of people to maintain social connections

increased tendency to anthropomorphize in situations of high cognitive load (e.g. situations in which a lot of information needs to be processed at the same time) and in social isolation (Waytz, Gray, Epley, & Wegner, 2010).

rooted in social cognition,

Automatic processes such as motor matching mechanisms will probably be engaged as a result of observing animals displaying behaviours that are familiar to humans, especially if their anatomy and general configuration resemble those of a person (Buccino et al., 2004Kupferberg et al., 2012)

Anthropomorphic interpretations of nonhuman entities, especially animals, are supported by a set of cognitive mechanisms. Some of these processes, including motor matching mechanisms, evolved schemata and empathy for pain from the social cognition domain, are probably engaged in anthropomorphizing and mind attribution in an automatic way.

https://www.edge.org/response-detail/27219

anthropomorphism provides an alternative “model” to help us to interpret behavior.

 

https://www.cs.cmu.edu/~kiesler/anthropomorphism-org/psychology2.html

https://prowritingaid.com/art/812/anthropomorphism-%26-personification%3A-what-s-the-difference.aspx

  • Personification: The attribution of a personal nature or human characteristics to something non-human, or the representation of an abstract quality in human form.
  • Anthropomorphism is literal. Speaking animals or objects who talk, think, or behave like humans are all anthropomorphic. Think of Thomas the Tank Engine, Winnie the Pooh, or Peter Rabbit. They literally act as if human.
  • Personification is figurative. Do you sometimes feel like your computer hates you, especially when it’s not working right? Well, it can’t literally hate you because it’s not human.

https://medium.com/emergent-future/technical-human-problems-with-anthropomorphism-technopomorphism-13c50e5e3f36

Technopomorphism is the attribution of technological characteristics to human traits, emotions, intentions, or biological functions.

This brings to mind a third major problem with anthropomorphism: the uncanny valley. While adding humanlike interactions can contribute to good UX, too much (but not quite enough) similarity to a human can result in frustration, discomfort, and even revulsion.

http://becker.idm.oclc.org/login?url=https://search-proquest-com.becker.idm.oclc.org/docview/1422064525?accountid=35619.

Animism (i.e., perceiving life in nonliving entities)

WHEN WE NEED A HUMAN: MOTIVATIONAL DETERMINANTS OF ANTHROPOPMORPHISM

Recipe for supreme happiness was other people

People need other humans in daily life for reasons ranging from the practical to the existential

Need is so strong that people sometimes create humans out of non-humans

Humanlike characteristics

Physical appearance

Emotional states perceived to be uniquely human

Inner mental states and motivations

Anthropomorphism does not include behavioral descriptions of observable actions

Requires going beyond what is directly observable to make inferences about unobservant humanlike characteristics

Does not merely entail animism

Animate life is not a uniquely human property

Does not include any requirement of reasoned or reflective endorsement of an inference

Anthropomorphism is no necessarily inaccurate

Considering an inference anthropomorphic only when it is clearly a mistake is itself a mistake

Some people anthropomorphize more than others, some situations induce anthropomorphism more than others, children tend to anthropomorphize more than adults, some cultures are notorious for their anthropomorphic religions and worldview

People reason about an unknown stimulus based on a better-known representation of a related stimulus, in this case reasoning about a nonhuman agent based on representations of the self or humans

Socially and effectance motivation

Sociality motivation is the fundamental need for social connection with other humans. When lacking social connection with other humans, people may compensate by creating humans out of nonhuman agents through anthropomorphism-increasing belief in anthropomorphized religious agents, GOD, or perceiving nonhumans to be more humanlike, PETS

Anthropomorphism can satisfy effectance motivation by providing a sense of understanding and control of a nonhuman agent, and should therefore increase as effectance motivation increases.

Participants in study1 who felt more chronically disconnected provided higher rankings of the supportive anthropomorphic traits than participants who felt more socially connected.

Participants who were chronically lonely would create agents of social support by anthropomorphizing their pets

Satisfy sociality needs

How effectance motivation may influence anthropomorphism

Dispositional tendency to seek understanding and control is facilitated by as stimulus that enables anthropomorphism

Seeing a nonhuman agent as humanlike not only entails the attribution of humanlike characteristics, but it also carries the consequence of moral agency

 

ANTHROPOMORPHISM AND SERIVEC HUAMNOID ROBOTS: AN AMBIGUOUS RELATIONSHIP

Due to the negative effect of anthropomorphism, right up to 2004, Pittsburgh Zoo did no publicly give name to its inhabitants – zoo decided to reverse policy and give animals name” because anthropomorphism can give rise to positive alternative effects

Use of the Paro robot in a nursing home: beyond the ethical question of quthenticity of the human-robot relationship, it clearly appeared that this therapeutic robot, developed by the Japanese AIST industrial consortium, was found especially comforting to patients with dementia

 

 

 

Thesis I – Project Blog 1

Main visual interactions:

  • At Attention – character interacting/listening
  • Present – Detected people
  • Alone/Idle – no one is visible and in idle mode

Audio interactions:

  • Silence – No real understanding and quite.
  • Understands the audio – move down the story
  • Bright/loud sounds – startles/stumbles the character

Interaction pipeline:

  • Starter Convo
  • General Questions – FILLERS Basic small talk with key words – gives hints of why and what he has saw (about people, and envirnoment). Could reply with a question to get info.
  • Rambling – He talks about his life stories, his time, his places, things about his personally. If someone questions the story, he stops explains in a little depth and then ramble about something else. *If loud noise detected or used keyword goes back to the main starter convo.
  • Keyword for fallback/home to starter convo

Life stories are randomly picked – Stories will have different audio with the same story but said differently.

IDLE modes should have different states – get attention, show emotion, move, look.

NEED indicator to tell if he is idle or not – candle could be flickering when active and static when not.

Other Notes:

  • Does not seem interested in people
  • He needs people to move and he likes the ability but hates to interact with people
    • In static could show discomfort, and want attention.
  • Think of different participators experiences – think of 4 words that they would say about the character personality.
  • How long should the interaction should be – MAX experience TIME????

Thesis Proposal Form

Thesis Proposal Form


What is your area of focus for your thesis research?

My area of focus for my thesis research is the uncanny valley and the experience of human-robot/A.I. interactions.

Please provide 3-5 artists whose work inspires you or that you plan to research as part of your thesis work.

Golan Levin, Edward Ihnatowicz, Kenneth Rinaldo, Lozano-hemmer

 

What type of work is your thesis project? (Game, installation, ARG, etc.)AND What technology needs does it require? Will you be using your own technology or Becker’s resources?

Installation. It will require a PC using Unity, Monitor, speakers, and Kinect V2.

Describe your project idea and how it ties into your thesis research in 1-2 paragraphs.

“Project Frame” is a dynamic interactive experience featuring a hanging oil painting that responses to visual and audio inputs. It will detect the presence of the viewers, follow faces and body movements, and listen for speech responses. All of which, will advance down a fixed “script” that the A.I. continuously references. The final presentation will consist of an environment that will be build around the frame for maximum immersion.

This project ties into my thesis because it involves human-A.I. interaction. I want to research the effect of the uncanny valley, positive or negative. Is there a usable use for it? Also, can different physical spaces promote this effect? Researching deeper into this, will help me heighten the overall experience of the project.

 

Describe your planned presentation method for your thesis project, (keeping in mind the gallery showcase at the end of the year).

I envision a space that mimics a Victorian library (with scope in mind). The main piece, the painting, is hung on the center wall. I plan to adapt the lighting in the space to hide the Kinect and any other sensors that are used in the piece. The painting itself will be a monitor with a frame built around the screen. Rest of the electronics, such as the computer, will be located behind in the walls.

 

Please list 4 major project milestones for your interactive work for this (You may also want to list your milestones for next semester but clarify especially where you want your project to be at the end of this 3-month semester)

  1. Programming A.I. branch layout/designed  *Me: character rigged and blendshapes done
  2. MIDTERM – Rough draft of the basic script finished. A.I. tree interacting with inputs. Alpha demo where users can say hello and have it hello back. Some character movements – eyes, etc. *Me: character textured, connect/import blendshapes with lipsync. | Help on the script
  3. Character face animated, have lipSync working with correct responses. Background done.
  4. FINAL – Ground work down for a interaction demo – Detects users and voice input affects the A.I. interaction using the script. *Basic interactions (audio, and visual) are smooth!

Detailed list:

  1. Art: Character is imported with textured, rigged, and blendshapes are applied. Prog: Must have branch structure designed which allows for input data effect the output visually. Such as (eyes follow detected user or head follows detected user)
  2. Art: Fill out the background of the scene and get lighting close to final. Prog: Have vocal input effect output.
  3. Art: Have basic on animations imported. (Arm movements on interaction, etc.) Prog: Have lipSync working, and Animations linked with the correct responses.
  4. Art: Post effects – overlays and any other polish work. Prog: Add more interactions with the data. Expand the tree and make sure it does not break during run time.

 

What are your project needs as far studio teams (ex- 2 artists, 3 programmers)?

  • 2 artists for background modeling, texturing and animations.
  • Writer for interaction mapping script
  • 2-3 programmers for hardware working, and basic A.I. work.
  • 1 audio engineer for voice and other ambient sounds.

* These numbers are guessed +/-. As I am not sure what help I may need.