Nowadays, it’s not easy to tell what is ‘real’ interaction and what is not. Along with the advancement of technology and how more and more people are entangled on the network, a lot of interaction happens ‘only’ as an idea manifested as data and displayed with computation. Looking at this from human-being’s perspective, it is important to think about one of the reasons why we interact in the first place: to let our existences validated by others. Only after we understand the why then we can try to figure out how the interaction gives and takes influence to and from us; whoever or whatever involves in it.

We are familiar with the interaction between The Evil Queen in Classic Disney’s ‘Snow White’, where she asked a magic mirror who was the most beautiful person in the world. The magic mirror became an important part of the story because it, supposedly an inanimate object, could answer a question and its answer influenced a person then created a major impact in a story. Would it be different if the magic mirror were a human? Probably not if in the relationship, The Evil Queen possessed the same extent of control to the other actant.

A hundred years ago, relationship between Evil Queen and Magic Mirror was merely a fiction but if we projected our relationship with our smart phone today, it would look even more imaginary. Nevertheless, both relationships tell something about human-being: human needs for its existence to be validated. In my previous essay, I argued that people tend to engage more on media that validates their existence by somewhat putting them (the audiences) as a part of the story; either by being relatable or letting the audience to contribute on the media itself. Following the essay, I created a website called ‘Picking My Monday Mantra’ where the audiences can repost any mantra that resonates with them or writing their own then submit it. In another word, contribution.

Contribution is only one of many forms of interaction practiced by media. It has been practiced and evolving for long time, for example when a reader sent a letter to a newspaper or magazine then the magazine would publish a selected letter and do something regarding it. Another example is allowing a listener to call and request a song on the radio. As the media evolves and the inclusivity of the internet put ever-growing number of individuals on the network, even more options of interaction have been made possible. The interaction can happen between humans, humans to inanimate objects, or as discussed in actor-network theory subject; between hybrid objects or actants, entangled on a network. (Latour)

However, I argue that in any interaction that involves a human-being, there is a matter of its existence validation; a means of giving or/and taking influence. It is a part of human needs of belongingness (Maslow 43-45) and self-esteem (Maslow 45-46). As the technology like Artificial Intelligence (AI), Virtual Reality (VR) or Augmented Reality (AR) evolves and hybrid actant, such as YouTube videos (created by human using hardware and software, then published as data) becomes more common than ever, it doesn’t make much difference anymore if the validation and influence come from what was once considered as inanimate object.

I take application using AR technology called ‘Skin and Bones’ (Figure 1), developed by The Smithsonian Institute (Smithsonian), as an example from my curation list. ‘Skin and Bones’ allow the user to interact with skeletons displayed at the ‘Bone Hall’ in National Museum of Natural History. The interaction can be initiated by as user by tapping an object (displayed skeleton) on the camera then a 3D version of it would pop up on the screen. The user can even make the 3D object on the screen move on the user’s hand. It is fun not only because it is surprising but also because there is a feeling of control to some extent felt by the user, at least the feeling of doing thing that makes an influence on screen.

According to Craig on his book ‘Understanding Augmented Reality’, interaction can be defined roughly as a mutual influence of one thing on another (Craig). Meaning that when one entity does something, the other entity responds in some way. Augmented Reality (AR) is an interesting subject because unlike virtual reality (VR) which participant is isolated from the real world, AR has the possibility of virtual interactions, real interactions, and hybrid; interactions that combine both the real and the virtual. AR technology happens somewhere in virtuality continuum (Milgram and Kishino).

Reality lies somewhere between what is completely physical and what is completely virtual along a scale of degrees, as depicted in figure 2, and can be described as a digitally modified reality in a form of a continuum that begins on one end with human perception of its surrounding (real environment) then extends through two stages of ‘mixed reality’ and ends with nothing but digitally produced images and sounds as virtual reality. (MacIntyre, Bolter and Moreno)

This possibility of hybrid interaction makes AR technology very potential to be implemented in other field such as psychology, for example the use of AR technology in ‘ARcockroach’ (Figure 3) (Juan, Botella and Alcaniz), an application that can be used to treat cockroach phobia. The use of AR in the treatment of these disorders has some advantage over VR because it involves real environment and real elements used by the patients. Only the elements that represent patient’s fear are virtual.

The conclusion of the exposure session using ‘ARcockroach’ with one patient was that the application was able to activate the participant’s anxiety then gradually decreased it. The patient scored 10 on SUDS scale at the beginning of the session and she scored 0 after the session. The patient experienced high levels of anxiety during the AR exposure session. The increment in anxiety was related to the different AR levels and the anxiety provoked by the introduction of several elements. Before the exposure session with ‘ARcockroach’, the patient was not able to approach a terrarium with an alive cockroach. After the session, the patient was able to approach a real cockroach, to interact with it and to kill it by herself. The result, although needed further assessment, was very encouraging because it demonstrated that AR exposure could possibly be effective for the treatment of a phobia.

Hybrid interaction also takes place on the other example on my curation list: MintChan (Figure 4) as virtual YouTuber (About MintChan). Virtual YouTuber (also known as ‘vtuber’) by definition is an online entertainer that uses a computer graphics-generated avatar. Commonly personating a bubbly teenage idol, MintChan sings, dances or does what a human YouTuber generally does. MintChan also interacts with her audiences, either be in text-based chats, or audio visual like live streaming ‘Question and Answers’ session.

We can talk over and argue about what MintChan actually is. MintChan is an example of ‘hybrid actant’. She’s an idea created by human, made of sets of hardware and software, learning to be as humanly as possible by using Artificial Intelligence (AI). This is where the actor-network theory helps us to discuss about how it affects us without having to define what it actually is other than hybrid actant. Above all, when it comes to discussion about the influence MintChan gives and takes, it is not different compared to what human celebrity has been doing all this time.

MintChan, just like human celebrities, creates a persona that resonates with its audiences. It influences the audience by doing things like cheering them up, singing, dancing, communicating that in a way validates the audience’s existence, feeling, or even aspiration. MintChan and other Vtuber also provide an answer to human needs to belong. The audiences not only created fandoms but also let the monetization scheme works. Fans may buy stuff like merchandise or their ticket concert as a symbol of belonging and loyalty; an identity; in other words: a kind of validation of one’s existence.

When we put it in a context, when a woman communicates with MintChan and gets told that she is pretty and she should cheer up, it is arguably similar compared to The Evil Queen and her Magic Mirror. Both interacted by giving and taking influence, and both created stories, yet like ‘black box’, the input and output can be observed without showing how it really works. Some people don’t know how MintChan works and it feels more magical and ‘real’. If someone sees, understands, and fully realizes how MintChan or other vtuber really works with all the robotic hardware (Archipel), he/she would be less engaged emotionally and less likely to buy the merchandise.

At the end of the day, knowing what is going on inside the black box is a part of human’s control, to decide to let the interaction happens in the first place or not. It is a part of human’s control, on how it would like for its existence to be validated, how it wants its questions to be answered, how it would influence and be influenced. Modern human will never run out of options but in being part of the rhizome called internet, making choice is getting harder. Especially when it’s getting even harder for human to tell what is real and what is not?

Responding to what I have written above, I created 2 AR (Augmented Reality) filters that can be used on Instagram and Facebook camera.

1. ‘Adapt!’

Adapt is an AR filter that applies imaginary face tattoo on audience’s face on the camera. Audiences can take picture or video with this filter. With this AR filter, I’d like to point out how someone now can show different persona using technology, just like a virtual YouTuber. Someone who, in real life, is not likely to have tattoos on his/her face can act like he/she does in a video or in live self-broadcast by using this AR filter. It would also make a significant difference for those who interact with the person. (for example, my conservative Moslem grandmother might have a mini heart attack if she ever saw me with this AR tattoo)

AR filter link for Instagram:
https://www.instagram.com/ar/222880699245782/

AR filter link for Facebook:
https://www.facebook.com/fbcameraeffects/tryit/222880699245782/

2. ‘O Crystal Ball!’

‘O Crystal Ball!’ is an interactive AR filter that works like ‘Magic 8 Ball’ toy. It would give random answers to Yes/No question, from 20 answers I wrote myself. An audience/user can ask their question, tap on the screen, and get an answer from the ‘crystal ball’. With this filter, I highlighted my argument on my essay: that one of the reasons of why humans interact in the first place is to get a sort of validation of their existence and it can be in a form of ‘answer’. A crystal ball made of AR technology is a hybrid actant, but at the end of the day, it’s a part of an interaction. For the human, it’s about existence validation; a means of giving or/and taking influence.

AR filter link for Instagram:
https://www.instagram.com/ar/450341112627080/

AR filter link for Facebook:
https://www.facebook.com/fbcameraeffects/tryit/450341112627080/

***

Bibliography

About MintChan. 2019. 13 December 2020. <http://mintchan.com/#About>.

Archipel. Japan’s Virtual YouTuber Phenomenon – Binary Skin. 26 October 2018. Video. 13 December 2020. <https://www.youtube.com/watch?v=7zZwjRmMWcY>.

Craig, Alan B. Understanding Augmented Reality: Concepts and Application. Waltham: Morgan Kaufmann, 2013.

Juan, M C, et al. “An Augmented Reality System for treating psychological disorders: Application to phobia to cockroaches.” IEEE Xplore (2004).

Latour, Bruno. We Have Never Been Modern. Trans. Catherine Porter. Cambridge: Harvard University Press, 1993.

MacIntyre, Blair, et al. “Augmented Reality as a New Media Experience.” (2001).

Maslow, Abraham H. Motivation and Personality. Harper & Row, 1954.

Milgram, Paul and Fumio Kishino. “A Taxonomy of Mixed Reality Visual Displays.” IEICE Transactions on Information and Systems (1994): 1321.

Smithsonian. Smithsonian Brings Historic Specimens to Life in Free “Skin and Bones” Mobile App. 13 January 2015. 12 December 2020.