Background

Shodō (Japanese calligraphy) is a widely celebrated artform in Japan, with great artworks known for being painted by calligraphers who achieve deep meditative states. Contrary to the mindfulness of the craft, these renowned artworks are often passively viewed in dull, crowded museums. We wanted to design a better way for people to actively engage and connect with calligraphy artwork.

This project was completed during a two-week design exchange program in Japan between Stanford University and Keio University. The broad goal of this exchange was to design an experience that bridges traditional crafts and new technology. During our trip to Japan, we learned about many traditional Japanese crafts, but became most intrigued by Shodō because it's one of the most widely practiced artforms in Japan, and Keio University is in close proximity to calligraphy museums and experts.

My Role

VR Prototyping in Unity, Animation

Deliverable

Virtual Reality prototype (HTC Vive)

Product Impact

Received interest from Keio University partners and vendors to create an installation in Olympic Park for the 2020 Olympics.

Timeline

2 weeks

Needfinding and key insights:

Needfinding interviews and observations

We took a Shodō class in Tokyo and interviewed the instructor to better understand the physiological and meditative practices behind the craft. The instructor helped us understand that breath control and good posture is important while mindfully painting strokes.

We visited the Taitō City Calligraphy Museum in Tokyo to observe how visitors move through the exhibits and view the Shodō artworks. We noticed visitors often had to slouch over, or stand uncomfortably to view the artwork.

Photos from our calligraphy class where we learned more about the posture and breathing techniques for creating Shodō.

Key insights that drove design decisions

Posture and breathing are connected. Both are important when painting each stroke of a character.

"... the whole idea of breathing is important, and how you put in power and less power — that transference of energy... Posture is really important. It's about taking in your breath and having your diaphragm ready."
-Shodō Instructor

Observing an expert's process is more compelling than seeing the final artwork in a museum.

"It's an entirely different experience than seeing Shodō in a museum. When seeing the person doing it, you're part of the process, you understand the process."
-Shodō Instructor

Contrary to the mindful posture and breathing in Shodō, artworks are viewed in uncomfortable positions.

We noticed museum visitors often had to slouch over, or stand uncomfortably to view Shodō artworks in display cases and on walls.

Defining the challenge:

There's an opportunity to make viewers feel more connected to the artwork by immersing them in the physiological, meditative state (posture + breathing) that Shodō experts describe. Ultimately, helping them better understand and appreciate the meditative process of the artist.

So...

How might we preserve the mindfulness practices in Shodō and translate them through an immersive art viewing experience?

Ideation and rapid prototyping:

I used binaural recording for guided meditation and nature sounds to immerse viewers in a 360° soundscape and engage them in breathing exercises.

I used the Unity game engine to create a simple virtual reality experience to immerse viewers in a 360° landscape.

I used After Effects to create simple calligraphy animations, allowing viewers to experience a Shodō artwork stroke by stroke.

I quickly storyboarded the expereience of how a musuem visitor would encounter and participate in the experience.

How Shizen VR works:

Viewers are immersed in a VR experience with storytelling and guided meditation

Accompanied by the voice of a Shodō artist, viewers are told the story of a popular poem while being guided into a meditative state.

Viewers use their own breaths to generate calligraphy stroke by stroke

As viewers are guided into a meditative state, they are instructed to recreate the Shodō artwork by taking controlled breaths. Each breath triggers a stroke of calligraphy. After the experience, their personalized shodō is theirs to keep in the form of a video.

Next steps for improving the experience

Unique calligraphy strokes from breathing nuances

For our prototype, we were able to use microphone audio input to trigger calligraphy stroke animations. Since we learned from experts that breathing informs the brush strokes, in the future we want to use a breathing sensor to capture the inhalation and exhalation thresholds — allowing users to generate unique calligraphy based on the nuances of their breathing. 

Rough example chart of how breath instensity might inform the appearance of a calligraphy brush stroke.

Building an installation to engage more of the senses

Viewers would be able to enter a booth built from materials that emulate the aesthetic, feeling, and smell of a Japanese temple, which is where many traditional Shodō artworks were painted.

Illustration was created by project teammate, Harry Krekoukiotis.

Feedback + Future implications

Photos from our presentation, user testing, and feedback sessions with our professors and Keio University faculty.

A new way to teach Shodō with focus on breathing and posture as opposed to excessive repetition: Traditionally, Shodō is taught via repetition and "drill and kill" methods with little focus on the mindfulness components.

An installation in airport terminals for visitors to experience before they arrive in Japan: Our professors and colleagues see our project as being an opportunity to bring awareness of Japanese art and culture to visitors before they arrive to Japan.

An installation in Olympic Park for the 2020 Olympics in partnership with a Keio University vendor: Keio faculty expressed interset in our project being a dedicated installation in Olympic Park for the 2020 Olympics in Toyko.

Reflections

Perhaps the biggest constraints were the two-week time frame and the language barrier. These challenges seemed daunting at first, but I believe we overcame them through the following:

• Tinker for solutions outside your comfort zone: We were pushed to get our hands dirty with new technology (i.e., AI, VR, AR, etc.) and craft an experience that people could interact with quasi-independently. We certainly didn't make in easier on ourselves by going the VR route, but we went for it despite the short time frame! With some experience in Unity/C# and AfterEffects/animation, I tinkered more deeply with what I already knew, and was able to collaborate with my teammates to craft a workable prototype in VR — while learning even more along the way. And while tinkering did take more time, the outcome was that much more fulfilling!
  
  
• Obersvational research helped overcome language barriers: Research interviews are an important part of the design thinking process. While I do speak some Japanese, I am not fluent enough yet to conduct interviews in Japansese. This made it challenging to extract more insights from calligraphy experts. To work around this barrier, we leaned on observational research by immersing ourselves in calligraphy classes and museums to gain different perspectives, and by asking questions in the moment to understand nuances in context.
  
  
• Prototype, fast, and even faster: Given limted work time we took on the mindset of rapid prototyping and made every trip to a Shodō museum, shrine, or historic site an opportunity to capture 3D audiovisual content, share our ideas with strangers, and an opportunity to ask questions to experts. We embraced an attitude of making our design work playful while informing iterations.

Credits

• Project teammates: Lauren Beaton, Jeff Kassab, Harry Krekoukiotis
• Peer group: Rika Ikeda, Cendikia Luthfita, Marcy Regalado, Landy Zhao
• Stanford faculty and mentors: Bill Burnett, Shelley Goldman
  
• Keio University faculty and affiliates: Masa Inakage, Yamen Saraiji, Kai Kunze, Kouta Minamizawa, Marcos Sadao Maekowa, Keiko Okawa, Hiroko Shimada, Elaine, Tomohiro, Feire, Miyo, Tanner, Andi
  

© Copyright Kenneth Fernandez 2019. Designed with ♥︎ in San Francisco, California.