babl
Babl is a language learning platform that uses mixed reality to provide an immersive experience for its users.
STATS: End-to-end semester-long project (17 weeks) I did with Ashlesha Dhotey, Nurie Jeong, and Eunjung Paik for Carnegie Mellon’s Graduate Design Studio 2: Microsoft Design Expo. Find our full project documentation in our Medium publication: Graduate Design Studio II: Mixed Reality.
ROLE: We each contributed to every phase of the project. I was specifically responsible for:
Managing project timelines, organization, and documentation
Leading verbal communication and presentation styles
Leading research, synthesis, and strategy
about
Microsoft’s Design Expo prompt for the spring 2017 semester was about “intentional design for positive cultural impact for mixed reality.” We chose to focus on the language acquisition process as a cultural learning process. We believe that language learning is not just about the mastering skills, but that it can also lead to a deeper understanding of cultures that are different than our own.
exploratory research
We narrowed our research to focus on the following questions:
How can mixed reality help language learners gain confidence?
How do cultural factors affect the language acquisition process?
How might mixed reality act as a multisensory platform that increases empathy?
Our exploratory research consisted of:
interviews with language acquisition experts, language teachers, and language learners
observation and participation in language-learning experiences
diary studies with people who were currently immersed in cultures other than the one(s) they grew up in
desk research
findings and implications
We used affinity mapping to synthesize our findings from our research as we went. Eventually, we translated the quotes and standout points from those buckets into seven main findings, and then into seven main design implications.
Through a quick journey mapping exercise for two different types of learning (in-class and immersive), we determined that mixed reality was best paired with the opportunity to ease the transition from one culture to another and provide them with confidence to use what they’ve learned.
focusing the solution
co-creation sessions
During our generative phase, we conducted workshops with students, teachers, and other language-learning experts to understand more about the learning journey. Each learner mapped out a timeline or journey of their own language-learning experience, calling out particularly painful or joyful moments. Then, we introduced “magic technology” cards to spark imagination, and the participants built, drew, or demonstrated a few brief ideas that would improve the experience.
concept development
By synthesizing the participants’ ideas and building upon them ourselves, we developed three concepts for further research.
“speed dating” storyboards
In our evaluative phase of research, we went through a series of quick prototype iterations, including storyboarding, bodystorming, and paper prototyping before moving to a digital mockup phase. We began to narrow down from our three concepts by creating storyboards of potential solutions and running “speed dating” exercises with language learners. By asking what resonated with them and what we should reconsider, we were able to determine that the most valuable experience to provide through mixed reality was an immersive social learning platform.
bodystorming concepts
To find potential pain points and opportunities, we bodystormed scenarios about the idea of language learners hosted by native speakers in their own realities. Embodying the experience helped us take a more critical look at our ideas as well as get an idea of what mixed reality could–and should–enable. We came across questions like:
How do we move within realities and from reality to reality?
How do different users interact with each other?
How do they interact with third parties?
Within these scenarios, we experimented with adding a few features from our initial three concept areas as well, including captions, conversation prompts, and vocabulary help.
The key takeaways were:
Put relationships first. From our previous research, we heard that relationships play a vital role in motivation and that interpersonal connections are where people learn best. We experienced that for ourselves through bodystorming. How could mixed reality help language learners become connected to a new community?
Visual cues like prompts and captions could be a barrier to building relationships. Its learning benefits weren’t worth that cost. Therefore, we want to create two separate modes: an immersive social interactive mode and an individual practice mode, to make use of helpful features while eliminating the barrier they might create between people.
Considering the social contexts that a language learner and a language partner might be embedded in, being able to interact with others in either space is a big value add for mixed reality.
We moved to low-fi paper prototyping to further refine our concept of connecting people for learning languages in context. Acting out scenarios using paper figures rather than our own bodies helped us to clarify whose reality we were talking about, who was a hologram, and who was “real”. We used these figures to address the following questions:
How does the system match learners and local partners?
What would be the best way to introduce each other and build trust with each other?
What is the most natural way of meeting each other in immersive space?
How do local people interact with holographic image of learners?
How does shared reality help language acquisition?
low-fi paper prototyping
scenario-based testing
We worked through this exercise ourselves and also brought these paper prototypes forward to our five evaluative workshops, where we asked participants to interact with them as well. We used paper prototypes and sketches to walk our participants through the experience of using the system. We asked for feedback on their immediate reactions to each scene, any hesitations they had about our concept in general or some parts of it and probed on them on some challenging questions such social boundaries and mixed reality visions of the future.
Different learners have very different needs in terms of what helps them learn best. One wants to be able to see grammar tables, another only wants to practice in a real social context.
Our system will need to set clear expectations for giving and receiving feedback on language use.
People had varying levels of trust for their hosts. They want to know what their hosts are getting out of it and if there’s any accountability–are they who they say they are? How can you communicate with either the host or the platform about your experience?
People loved the social aspect and the ability to experience the environment — before arriving, people want to know what the neighborhood is like: walkable? Friendly? Amenities?
People want to be as engaged in the moment as possible with as little intervention from the system as possible. Interventions and helpers like grammar tables, captions, and slowing things down might be helpful in an individual mode, but interrupting a conversation isn’t good.
illustrated narrative
From our evaluative research, we narrowed down to one concept and created an illustrated narrative to communicate how it could be used.
ui/ux prototyping
In this making phase, we made decisions about how a user could interact with our system and what that interface looks like. We started by playing with the Hololens to get an idea of what interactions exist now and what we’d have to consider. We mapped out the system and made a list of all the features we wanted to include.
We then moved into paper prototyping for MR, eventually deciding on a tool “tray” that can be expanded and retracted into the bottom/side of your peripheral vision. The tray would contain tools like doodle, pin, stop sharing view, switch perspective, and end call.
We also considered the screen UX flow. We each created a wireframe flow for the sign-on and matching process and then decided together on a singular flow. We also talked a little about the UX flow for the mixed reality experience: how do you navigate from turning on the Babl app in MR to exploring scenes in practice mode? Or to joining a “tour mode” call?
system overview
When users create accounts on Babl, they need to provide some information about themselves. The profile they create provides Babl with information about their preferences, interests, and personality, which will be used to match them to other Babl members down the line.
In mixed reality, users can access an individual practice mode and a hosted tour mode. To unlock the tour mode, learners need to show that they at least have basic conversation skills, which are evaluated in practice mode–which is accessible to all language learners.
In tour mode, hosts share their environment, language, and culture with Babl learners. Learners can practice the language with their host, interact with the environment, and experience things from their host’s point of view.
Practice mode is an individual mode where learners can access language lessons created by Babl and their own bookmarked scenes. In these immersive scenes, learners can take on the role of their host, slow the recording down, get grammar tips, read captions: basically all the helpful language-learning tools that would detract from building a strong relationship with the host.
Babl has potential to have a huge social impact. It can disrupt the mental models of the way language learning is perceived and also shift the paradigm of language learning. We believe Babl can ultimately reduce emotional barriers between two cultures by connecting people of two different backgrounds.