Gestures to add/remove objects in virtual reality
Liang Men 19.04.2018
First, let me introduce LeMo (Let's Move, or Lè Mō/乐摸 in Chinese). LeMo is a multi-player virtual environment (VE), in which two users can interact with virtual musical interfaces and create music together (Men & Bryan-Kinns, 2018). LeMo is designed and built by me, under the supervision of Dr. Nick Bryan-Kinns. In the process of designing and building LeMo, I was always striving to eliminate interaction that based on dialog-boxes, and use intuitional bare hand based interaction as much as possible. Today, in this post, I want to share some of my thoughts on design gestures to add and remove an object in virtual reality (VR).
In the real world, human cannot create nor terminate something out of thin air, what we called "creating" is actually "converting". However, VR enables human to create or remove virtual object out of thin air in VR. To make this unnatural ability as natural as possible is what I am gonna talk about next.
First, generating objects in VR. In the real world, objects usually start from nothing and then expand from small to large. So in LeMo, I tried to mimic this, as this is an experience users all have in real life and can be easily recalled and mapped into VR. In LeMo, users can simply move their hands together, pinch and stretch to create an object (music interface), see Figure 1. This idea was initially inspired and mostly came from a VR demo called Blocks (Leap Motion, 2016).
Figure 1: Gesture to generate objects
Second, deleting objects. I still tried to mimic a daily experience. Imagine if you have a big balloon, there can be two ways to vanish it, either deflate it entirely or blow it bigger and bigger until it explodes (see Figure 2 and 3). To enhance the feedback, visual and audio effect are also added.
Figure 2: Gesture to squeeze objects
Figure 3: Gesture to explode objects
These gestures showed above can also be used in similar VR projects to achieve a fluid and natural user experience. However, they do have limitations, e.g. the deletion gesture does not ask the user to confirm a deletion, which means mistakes can be easily made. In the next post, I will show more details of how I address this limitation and programme to detect these gestures.
19 Apr 2018, in Queen Mary University of London, London, UK
References
Leap Motion: Orion. (2016). Leap Motion Blocks for Oculus Rift Playthrough. [online] Available at: https://www.youtube.com/watch?v=rnlCGw-0R8g [Accessed 19 Apr. 2018].
________________________________________
以下为中文译文:
首先简要介绍下LeMo,LeMo(实为Let’s Move的缩写), 中文土名“乐模”,是一个多人虚拟现实环境,在其中,双人可以共同与音乐界面互动并创作音乐。LeMo主要由我设计并编码,由我导师Nick Bryan-Kinns博士指导。在设计并编码LeMo的过程中,我一直在努力消灭传统的对话式人机交互,尽量多的采用裸手交互。今天在这里我想和大家分享下一些小小心得,此篇是关于如何在虚拟现实中生成/移除物体。
在现实中,我们其实并不能凭空生成一个物体、物质,人类能做的只有转化。而在虚拟现实中,人们是可以凭空生成或终结虚拟物体的。作为一个设计师,我要做的就是如何把这个不自然的过程做的尽量自然。
先说说如何生成物体,物体从无到有,均始于方寸之间。在生成过程中,物体也需从小到大。比如气球,均是慢慢充气,从小到大。类似的,我决定识别用户双手捏并拖拽实现生成一个物体,用户拖拽的距离越大,物体也就越大。
关于如果移除一个物体,我采用的拟物的思路,这样方便用户把此交互动作联系到真实世界的经验中。打个比方,这个物体恰好又是球形的,所以打算让用户认为这个物体是个气球,气球常见的破坏方法有两个:
1. 大量充气,爆破他;对应的,我编码识别了手势:一个双手是在物体拖拽这个物体,拉大它,达到一个限度时,此物体爆裂消失;
2. 挤压它,使其泄气;对应的,我编码了识别用户双手在物体内,捏住并挤压这个物体的动作。为增强反馈,声音效果和闪光效果也被加入以增强反馈,让用户理解这个物体爆破并消失了。
上面显示的这些手势也可用于类似的VR项目中,以实现自然的用户体验。然而,它们也有局限性,例如删除手势并没有要求用户确认删除,因此导致错误很容易发生。在下一篇文章中,我将详细介绍我是如何解决这个问题,并且讲解实现这些手势识别的编程思路。
©2024 by Liang Men