Future Hybrid Mixed Reality Design Studio Environments
Brief:
There is extensive research and development in wearable mixed reality devices. Unlike virtual reality, which completely immerses you in a virtual world by obscuring your entire view, mixed reality blends real and virtual to create hybrid experiences. For this project we are looking 10 years out, assuming that head-mounted displays, such as Microsoft’s Hololens, are ubiquitous. Imagine that in the future head mounted displays might fit into thick glasses frames and are as commonplace as smart phones today.
The School of Design at Carnegie Mellon is hiring you to develop a mixed reality hybrid (people both in person and remote) design studio environment to enhance the studio learning experience. Focus on one aspect of the studio experience.
What to do:
You will be designing and developing a low-fi prototype of one interaction in a hybrid studio experience for the client to review.
Goals:
To come up with better ways of continuing the traditional studio supports for student learning. The traditional studio setting creates many opportunities for new ideas, creativity, and social bonding.
Stakeholders:
Carnegie Mellon University
Key Considerations:
●What information about the studio is currently available in a traditional design studio and in what media?
● How might students and faculty maintain awareness of each other without increasing interruptions?
● What are some of the affordances of augmented reality?
● Does additional infrastructure/technology need to be added to the environment to support AR? How does our physical environment impact how we use digital devices?
● What are key “stories” that need to be told about what is happening in a design studio to students and faculty?
● What information is each type of user hoping to learn about the studio?
Studio Supports to think about:
Initial Research
In the team, there are a lot of roles that takes important parts of the designing process. If we used to utilize the traditional 2D design process as a fundamental idea to break down the elements in the past, people can now go beyond the traditional world of 2D devices, and explore the developing innovative mixed reality experiences. Nowadays, people encourage interactions with physical props in a real-world space. We will be using physical props to represent digital objects, user’s interfaces, and animations in a proposed experience.
Interviews about being in-person/remote during this pandemic
How often do you go to studio outside of class? What’s the benefit of going to studio or staying home?
Francis(in-person): I stop by studio almost everyday for varied amounts of time. I think I work better if I am in a “work space”. I feel a little more disciplined and productive in an environment that is meant for work with other students working along with me.
Chris(in-person): I try to go to studio everyday because I am more productive when I have a designated work space. Also when I go to studio it begins my day in a productive way and makes my day a lot better.
Ming(in-person): I started going to studio once in a while a few weeks into the start of the semester but having adopted a pet recently, I am more constrained as to the frequency I go
Did you have any interactions with remote people?
Francis(in-person): I was able to engage in group critiques in class in addition to Zoom calls during class. The interactions were decent/fine mostly due to the fact that my studio work is easily transferrable on a digital platform (C-mini work is mostly through software).
Ming(in-person): I have had several interactions with people studying remotely, in general they’ve progressed smoothly with the exception of internet issues. Sometimes communication is difficult due to the lack of a common space to work in and/or irregularity of people visiting studio.
Chris(in-person): For my 15–104 class, I had to work on a recitation project with a partner over zoom. I did not enjoy the experience bc it was hard to manage and it was hard to have the zoom interface as well as the assignment on the screen at the same time.
(Both) What is your remote academic experience like?
Dorothy(remote): It’s been adequate with classes and professors trying to accommodate and take care of us, however, all of my classes are synchronous and time zones difference has been a pain. It’s different from freshman fall because I don’t really get a studio experience and interact as much with other people inside and outside of class.
Ming(in-person): I find the change in social dynamic disconcerting and upsetting at times. We were able to enjoy regular, frequent interactions with each other which was one of the unique aspects of having a studio. It contributed to both our academics and general well-being. Not only does that dynamic not exist anymore, with all that is currently going on I feel less inclined/motivated to go to studio.
Francis(in-person): It’s very different and it’s easy to get bored and disengaged from the classes. I find myself consciously trying my best to make sure that I am participating in class discussions. Major differences lie in the level of energy created from being able to see people eye-to-eye; even though professors try their best to adapt, it will ultimately be impossible to replicate or even come close to on-campus classes.
When you interact with on-campus classmates, what is that experience like?
Hayoon(remote): Depends on how close I am with that person but mostly, it’s more awkward to chat with them through zoom. It doesn’t feel natural.
Dorothy(remote): In class it’s kind of weird because when I talk my head would be enlarged on a screen and everyone would just stare at the screen. I can’t tell how I’m speaking and how everyone is reacting to my words. I engage mostly for academic reasons because it’s hard for me to keep up with general fun stuff.
What would your preferable remote interaction be?
Ming(in-person): I’d like the interaction to feel real and genuine, we are always seeing each other either through screens or behind masks and it causes a loss of the unique intimacy and connection you’d usually get from normal social interactions.
Dorothy(remote): Interaction would be very smooth without feeling a lag between what someone is saying and what I’m hearing. Less awkward moments when instructors try to get a response. Maybe putting more emphasis on natural communication features like eye contact and body language.
Chris(in-person): I don’t have an ideal remote interaction idea. I wish everything was in person as hands on is where I learn best.
Any ideas that this remote experience could be improved?
Francis(in-person): Would be better if classes/professors have more explicit feedback sessions with students: I think some professors fail to realize that students will feel pressured or not the most comfortable with reaching out about things to fix, even if professors encourage it. Also, being actively open about feedback: Being open to truly compromise about class work with students and finding ways to make sure that the level of classwork is balanced with finding ways to make students enjoy and engage with the material.
Chris(in-person): I wish professors would pay more attention to the quality of education their remote students were experiencing, aka the sound, camera location, one on one meetings.
Hayoon(remote): I wish it could there were more ways for us to more actively communicate with the people on campus. I feel isolated and it’s hard for us to contact people.
Personas
What are Personas?
Fictional characters that represent different user types. Used in human-centered design to consider needs, experiences, behaviours, and goals.
Personas Types
- Goal-directed Personas
- Engaging Personas
- Role-Based Personas
- Fictional Personas
Why Personas?
Build empathy: help us understand the perspectives of who we’re designing for
Drive decision making: use as a guide for key features, functions, and usability, rather than focusing on ourselves as the end user.
I’ve created these three personas that represents the general body of the students after learning from the survey of my friends.
Sophia:
Better environment to see what other students are working on.
Wish to spend some time together with friends while they are doing their own work(like gather town).
Thomas(Professor):
Frequently experience hard time using technology features.
Have to set up an appointment and give feedbacks to students and have to accommodate and take care of students too.
Samantha:
Barely goes to the studio(who might use AR feature usefully).
Want to actively communicate with the people on campus.
Sometimes feel isolated when it’s hard for her to contact people but also need her personal time too.
Initial Storyboards
: as a way of exploring scenarios and your initial idea
My initial idea is generated from the thought of “how can we get better environment of working?” When we’re not in the studio, it is hard for us to concentrate from distractions.
I also thought about how I can combine the real and virtual space together. Virtually editing/commenting on the real poster that is actually pinned on the wall.
My work space
I am a type of person who spends more time at home whether I am working on design or not. I value the time spending by myself but also wants to socialize with other people while I manage my own time. I think the main point that I would be thinking about while creating this hybrid studio will be “how to allow users to maintain good amount of both personal time/socializing time.”
Instead of requiring all those connectors on the laptop, I would also have to think about creating a convenient way to connect collaborate iPads and laptops together.
Self Reflection
As the prevalence of digital media in our physical environments increases daily, what is the role and/or responsibility of designers in shaping our environments?
I believe the role/responsibility of designers in shaping our environment is based on how well we create the interaction between the object/subject with people. I think depending on how we provide/get the feedbacks based on the usage, the quality of relationship is determined.
Just like the fact that digital media is composing most part of our life these days, we tend more relay on these. Thanks to these digital media that creates our physical environments, be experience benefits on both physical and mental states. Even though AR/MR are mostly used as an entertainment and videos, I am be sure that it will expand more toward the academic/economic aspects too.
I also think that it is our designer’s role to figure out how/in what ways people can improve the interactions between the real world and the imaginary(virtual) world without confusion. Also considering different background aspects like people with physical disabilities for vision, hearing, and etc. Drawing a line between the real world and the virtual world is hard. But I believe finding the middle line between those to and balancing those two is one of the responsibility that designers have to consider as the prevalence of digital media in our physical environments increases daily.
What programs do you use to work, communicate, socialize? Make sketches of how you get tasks done
-> Most of the apps that I use for my design work. Only used from my laptop(larger screen). Requires a lot motion sensors(drag/scaling/click/etc).
->Requires video/voice/recording(etc) functions for interacting with other people working remote/in-person.
->Methods to take note/communicate/work at the same time. Sometime can share by adding people to the board or sending the link by sending URLs or links.
Prototypes
This a first-person point of view, clicking and dragging video prototype, for the purpose of sharing/saving projects. If the user wants to share/save what they’ve been working on, the user can double tap on the picture/artboard and simply drag it to the sharing board area. Sharing design will be
This is an interaction that I tried with Wiarframe. I tried to make when I stare at an icon, it is selected and moves forward. For this iteration, I planned to create an icon selecting section that people can adjust within the AR space. However, there were some limit that was able to conduct using Wiarframe. I had fine a way to bring my hand in front of it(which is impossible to do with this app). There for,
Oct. 8, 2020 Feedback
- Try to figure out how to create a public environment and private environment ( something that you could share as you upload & something that people can see as if you’re walking around the studio)
- Pin up for yourself/ Pin up for everyone to see
- Thinking about socializing aspect and working aspect is good but try to collaborate or focus more on a single aspects(try to develop deeper)
- Threshold?
- Mixed reality -> breaking the frame
- Not necessarily show both in-person/remote environment
Prototypes #2
This is a simple 2D prototype sketch of how it would look like using AR to look around 3D sketchUp models. I came up with an idea of having frequently used programs to be visible on the left side of the AR screen while the person controls the 3D model with the controller(on his hand). I’ve also set up that everything glowing with the color of orange is only visible while wearing the AR glasses.
3 Layers of Experiencing
- Sharing(Board)
- Real — Virtual Connect(Keyboard)
- Communicating(Chat)
Communicating Method
How?
I got this inspiration from facebook messenger app. When people are online, green dot lights up next their profile. Then, I can click on his or her profile to chat with that person. I thought this was a very helpful function when I wanted to talk to people on live/right away.
Oct. 15, 2020 Feedback
- Should decide whether it will be used in-front of the desk or an open space that people can move around
- How close/large/interactive?
- Any indication so that people can be recognize what class this is for?
Video Sketch Process #1
What is the problem? What am I trying to solve?
How can we combine reality and the virtual space to communicate with classmates while sharing/viewing their work at the same time
Main focus on my Video
- Share — Works can be shared anytime, anywhere/people can decide whether to be public or personal
- Communication — Chat with classmates online/leave comments, ideas, thoughts to their board
- Space — Still can experience the traditional studio environment vibe even though he or she is not in the studio/but also can have personal space
What do I consider the most?
- This will be a daily item that students and faculties will be using. Have to be easily accessible and approachable to everyone(including people who is not tech-friendly like Thomas from my persona profile.
- Should be beneficial and effective for students/other people to use(people should think it is useful and worth it)
Base Video
- First person point of view — Side view what other people with AR glasses might see
Communicate(Calling):
This is an idea for chatting service that students can use to check whether his or her classmates are online. When you want to talk to people, double click the further left icon and green lights will be on next to their name if they’re online. Select a person who you want to talk to and connect it to your laptop or an i-pad that has a typing function.
Typing function can be also connected with every machines that has a bluetooth and keyboard tying function. As long as the machine bluetooth is connected to AR glasses, data can be send anywhere(it can be an ipad, laptop, phone, apple-watch, etc).
I’ve used Discord to make a 3D version of each person. I wanted to give people a traditional studio environment where people are working together as if they’re in the same space. There for, instead of screen 2D zoom calling, I thought bigger image with more three-dimensional effect will allow students to feel closer to each other.
Share(Public Group — personal)
When you click on a certain person’s board, it will be selected and enlarged. For my sketch video, I’ve selected Samantha Park’s board to look around and zoom-in with your hands.
Space(Studio or Home)
This is an idea came from Miro that we often use to collaborate/share together. I wanted a space where we could share our works and upload/save art works freely no matter where we physically are. By clicking the laptop’s canvas screen, it will be connected to AR canvas screen at first but then the users can modulate the screen by click the app on the bottom right section.
The order is Group, Calendar, Tools, Board, and more info. Tool, in this section is design tools like adobe apps. Board icon refers the folder that people can upload stuffs like Miro. People can also choose whether to upload it on public group board or personal board where only you can view.
Video Sketch Iteration 1
Oct. 20, 2020 Feedback
Peter Feedback:
- Consider what is left out and included when AR screen is in-front of the laptop.
- Think about how to use connect the Real world interaction the AR experience.
Daphne Feedback:
- Any visible confirm? (How can we know which icon I selected)
- Icons are too small to click(should acknowledge that it will be clicked with our fingers)(any indication?)(should be big enough to click it with your fingers)
- How do we know which is an active windows? (which window are you currently using?)
- Because it is a fast past video, it will be very helpful if there is a voice, that follows through the process describing what is happening.
- What does “Group” exactly mean? How can this be more accurate?
Davis Feedback:
- Try to give a slight curve?
Video Sketch Process #2
Changes?
- Larger icons and text to be visible on the AR screen
- Deleted unnecessary icons on the chat bottom left
Changes?
- New “Add comments”(sticky Note) and Connect to Laptop” function
- Samantha board is still visible while communicating with other people
- Windows that I’m not active will be blurred (low opacity)
- Group -> Environment Fall 2030
- Icon clicking will show white circle on the screen and selecting a certain text will highlighted
- Real world interaction(laptop typing for the chat)
System Map
I think every parts have interconnections between each other. I believe our hand, controls and calibrates technology that is needed and from there, it spreads to hardwares like glasses, laptop, ipads. These machines play the role of displaying, communicating, and interacting. They are the fundamental aspects that should be built up first to come up with AR/VR space.
Then, there is a cloud, that contains the memory. It is like a bridge that connects between the computer desktop and the wearable hardwares. Memory plays a role of saving and sharing data that we need. In my hybrid mixed reality, memory is connected to the hardwares with bluetooth. As long as the machine bluetooth is connected to the glasses, data can be send anywhere(it can be an ipad, laptop, phone, apple-watch, etc)
Final Hybrid Mixed Reality Video Sketch
Self Reflection | Hybrid Mixed Reality
How were the skills you developed in the first project similar and/or different from the second project? What is your understanding of the role of an Environments designer?
From the first project, I was able to learn to figure out how people can have different mindsets inside them depending on what they are viewing. Because my first project author was an anti-technology person, it was a new experience for me to think in the opposite view. For this second project, I’ve learned A LOT of interesting parts. I was able to think about how creative we can be with virtual space. Something that inspired me the most was the video that we saw during class where people can create a 3D virtual design store for a client just in a short time. It was also a new thing to think about the relationship between reality and virtual space. While I only used computer graphics like SketchUp and Photoshop to collaborate to illustrate spaces for the first project, this time, I was able to film the actual space and had to think about how to collaborate and integrate 2D/3D dimensional aspects within it.
For me, understanding the role of an Environment designer is not only offering simulation possibilities with scales, costs, and risky situations that are impossible to approach in the real world but also has to deal with object’s interactions with its environment and the users, by demonstrating what this experience can bring to the user, by giving it some meaning. Before I really got to think about AR/VR environmental ideas, I only thought 3D just makes it look cooler. But after I’ve done this researches/learning, I think it is more than visual entertainment. It is creating an interrelationship with the audience and the object that could be creative endlessly. I am also confident to say that the emergence of hybrid virtual reality experience can bring a new field of exploration to my domain as a designer.
