AR @ Tiktok
I make fun, interactive AR effects to bring emerging technology into users’ content creation and daily life.
World AR - Lantern
- this project focused on introducing location based AR to Tiktok users. I worked as the lead interactive engineer to design, prototype and release the experience from 0 to 1.
[promotion video]
- For the CNY campaign in South East Asia, users will get to write down their wishes on a paper lantern, fly the lantern, and also pull down random lanterns from the sky to see what other nearby people has wished for.
- They will also see name of local Point of Interests (usually landmarks, shops, restaurants, etc) in the sky.
[User Videos]
World AR Phase 2 - Giants’ World
- This project was an experiment to further explore location based AR as well as introducing narrative worlds to Tikotk users.
- As lead interactive engineer, I and the team designed a fictional Giant’s world, and Tiktok’s first ever cross-effect persistent inventory system, to allow users to collect virtual artifacts.
- The interaction is unique in that we used WFC algorithm to help users generate a narrative image with just a few pieces of the puzzle.
The effect was featured in App store’s Tiktok page:
Elsland
multiplayer interactive narrative
Elsland is an interactive narrative experience designed for IAC’s 120ft long video wall in Chelsea, Manhattan. It’s a story about escaping the mundane and the search for an utopian destination called “Elsland.”
At 7:30PM on Dec. 20, 2019, over 150 people joined the experience by going to els.land on their mobile devices. Every single audience can contribute to this collective journey by doing some simple actions, such as turning on a light, turning off a TV or removing a small obstacle.
Each audience’s action exerts a small change on the world inside the screen in real time. Collectively, everyone’s efforts push the narrative forward.
Elsland At IAC Center, Dec. 20, 2019
Full Experience: (modified for 16:9 screens)
Our process:
Team:
Artistic Direction + Unity Worldbuilding - Carol Jingjie Chen
Interaction and Server Development - Rui An
Modeling and Character Animation - Chenwei Jin
Mobile UI/UX Design - Xinyue Li
Web Development - Qice Sun
Music and Sound Design - Chenyu Sun
The Peacock Gate
VR narrative experience
The Peacock Gate is an interactive virtual reality dairy, in which I attempt to transform fragments of personal memories and consciousness into virtual sculptures and virtual memories. It is a self-dissection and retrospect of my struggle for a clear cultural and social identity by enacting fragments of my memory, from an estranged hometown to my immigrant experience. In this process of recreating memories in virtual spaces and translating mental experiences to virtual experiences, fluid, intangible memories are solidified and contextualized, abstracted and reshaped. These memories constructed virtual spaces that became a ghost town of my reconstructed identity.
It’s an interactive vr experience where the audience travels to different memory spaces while hearing a voice talking about these memories. Each memory space involves interactions that allow the audience to enact the experience the narrator is recounting.
The experience of leaving my hometown and home country to live in different cultures and societies in my formative years, had a strong impact on my identity. It’s a loss of definition for “home”. It’s a rupture between my perception and action. It’s a compulsive desire to return to my origins.
I held on to these memories of my childhood and hometown, referring to them as the origin of my identity. But rapid urban development replaced all my old memories with new constructions, and I lost all material connections to my memories. These memories, objects, and spaces now only exist in my consciousness in a fluid, obscured form, and so is my identity. I try to recreate these objects, spaces and memories and put them in a virtual space. I used virtual reality as a form of embodiment of my consciousness. Having these memories dwelling in virtual spaces, lost connections to my origins are restored in a different form. These memories are immaterial, but tangible.
In this process of “migrating” my memories, I recreated, reinterpreted, and reconstructed my memories. Making virtual memories is both an experiment and a metaphor. They can never be an accurate representation of my original experience, but memory is constantly evolving, and these virtual spaces and sculptures are abstracted remains of that evolution.
Project still under development...
Mirage Well
immersive installation, projection mapping
Well, as one of the earliest man-made structures to access groundwater, was the first step to break the limitation of the land and explore a world beyond the plain and familiar. The circular structure of the well manifests multiple layers of semiotic meanings — it is a container, a frame, a portal to the unknown, a reflection.
In this versatile installation with no definite form, withered trunks, light and shadow, water waves and sounds merged together to create a space where reality and imagination crossover. The giant metal ring forms a well –– an entrance, a threshold and an opening to another realm.
We stand inside and look at the world framed by the well; we stand above and stare down at this black hole of unrealities.
Exhibition in Auto Space, Beijing China
Tools used include Kinect, Touch Designer, Madmapper, and Cinema4D.
Credits:
Produced and created by: Neonblack Studio -
Jingjie Chen, Chenshan Gao, Chenyu Sun, Elvin Ou, Zhe Wang, Yunze Shi
Complete Me
Multiplayer VRA multiplayer VR experiment to exchange “organs” and have different sensual experiences, in an attempt to explore empathy in virtual embodiment.
Two players enter this world but they will see and hear very different things, because each player has a different number of eyes or ears. To empathize with the experience of missing a sensing organ, we created this project to help people connect each other through the act of swtiching organs.
This project is developed in Unity game engine, and the networking framework is called Mirror.
Team: Carol Jingjie Chen, Chenyu Sun, Xinyue Li