Abstract
In this project, we identify design guidelines for stereoscopic 3D (S3D) user interfaces and present the MirrorBox and the FrameBox, two user interface prototyping tools for S3D displays. As auto-stereoscopy becomes available for the mass market we believe the design of 3D UIs for devices, such as mobile phones, public displays, or car dashboards, will rapidly gain importance. A benefit of such UIs is that they can group and structure information in a way that makes them easily perceivable for the user. For example, important information can be shown in front of less important information. Based on core requirements for designing S3D UIs, we derive concrete guidelines. At the same time, the requirements serve as a basis for two depth layout tools we built with the aim to overcome limitations of traditional prototyping when sketching S3D UIs. We evaluated the tools with usability experts and compared them to traditional paper prototyping.
N. Broy, S. Schneegass, F. Alt, and A. Schmidt, “FrameBox and MirrorBox: Tools for Prototyping User Interfaces for 3D Displays,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, NY, USA, 2014. |
Introduction
in cinemas or home environments. While the need for glasses may be a major hinderance for their uptake, the advent of auto-stereoscopy may lead in the near future to 3D displays becoming ubiquitous, for example in mobile phones, public displays, and cars. This provides a novel mean for designing user interfaces and exploiting the third dimension, including the opportunity to present information on different depth layers. Prior work showed that depth combined with other salient cues (e.g., color, motion) can improve the performance of visual search tasks. For instance, frequently used applications on mobile phones could be grouped together, using depth as an indicator for the most recently launched application. Or in security-critical environments, like the vehicle, where users need to be able quickly and accurately perceive information, a comprehensive visual structure can help to immediately identify important information.
To build such interfaces, there is an increasing need to provide basic design guidelines to sketch and arrange graphical user interface (GUI) elements. Building such S3D interfaces is in many ways challenging. For example, designers need to cater to the fact that elements cannot equally comfortable be perceived in different locations in 3D space and that a minimum distance needs to be maintained between elements so that users can easily distinguish which element is shown further at the front. Hence, this paper puts a strong emphasis on understanding how we can support the user in prototyping successful interfaces.
To do so, we were interested in identifying requirements for layering information in 3D space. We studied related literature and complemented our findings by conducting a laboratory study which investigated object arrangement on different depth layers. We found that (a) positioning information on more than six layers significantly decreases user performance for depth-related search tasks; (b) that grouping objects can reduce error rates and task completion times for depth-related search tasks; and (c) that maximizing the distance between layers improves user performance when distinguishing information on different depth layers. These findings, together with the results from related work, serve as a holistic foundation to understand the requirements of layered S3D UIs.
At the same time, tools are needed that provide means to arrange items in 3D space. Particularly in user-centered design, low-fidelity prototyping is a powerful means to quickly and easily prototype new interfaces. Well-established techniques such as paper prototyping, however, cannot easily be adopted for S3D UIs as they do not provide means to position UI elements in 3D space. As a solution we report on the design and development of two depth layout tools for prototyping S3D UIs. Based on these requirements we built the Mirror- Box and the FrameBox (Figure 1). Many of the requirements serves as a basis for informing the design of the tools.
We evaluated the prototypes and the generated guidelines in the context of two concrete use cases – S3D-enabled mobile phones and S3D dashboards in cars. Feedback from expert users suggests that the proposed prototyping tools are superior to ordinary paper prototyping as they encourage to observe the collected guidelines, support collaboration, and increase creativity as well as the perceived fidelity.
Requirements for Layered 3D UIs
At the outset of our work we aimed to understand fundamental challenges that affect the design of feasible S3D UIs. Previous work uncovered a number of aspects related to content design that strongly influence the quality of S3D UIs. In the following, we collect guidelines from previous work to facilitate design decisions for high qualitative layered S3D UIs.
Comfort Zone: Prior work found that excessive parallaxes reduce the viewing comfort due to accommodation and vergence mismatches [10]. Hence, a comfort zone needs to be determined in which it is easily possible for the user to perceive information. As a rule of thumb, Lambooji et al. recommend limits for retinal disparities of 1 with respect to Panum’s fusion area [10]. However, there are studies that reveal more conservative values [31].
Show Content Behind the Screen Layer: Several studies recommend the use of positive disparities for information display. Hence, the main content should be positioned behind the screen. Negative disparities can cause longer task completion times [4] and visual discomfort [14, 19].
Minimal Distance: Stereoscopic depth is a super acuity which allows a depth difference assessment of 40 arc-sec. Broy et al. explored the minimal distance between two depth layers, that allows for quick separation [4]. They suggest a threshold of 2.7 arc-min.
Careful Use of Monocular Depth Cues: Monocular depth cues can support the spatial impression of stereoscopic visualizations, particularly in virtual environments [14]. However for abstract content (e.g. icons) that is likely to appear in layered information interfaces, monocular depth cues should be applied carefully so that scenes do not become unncessarily complex, for example, through shadows. Moreover, focal blur can reduce user performance in judging depth and perceived quality of the system [17].
Use of Text: It is advisable to display text parallel to the screen surface, since perspective distortions reduce readability. Stuerzlinger et al. found a sharp decrease for rotations larger than 60 degrees [25].
Highlight Objects by Combining Salient Cues: Combining salient features for highlighting objects has been shown to decrease search times [1, 8]. Hence, user interfaces should employ a foremost layer that depicts urgent information, such as warnings and notifications.
In summary, our analysis of prior work provided us with a rich set of requirements that we used (a) to inform the design of our depth layout tools. The complex nature of S3D UIs suggests that the tools should be able to literally guide the user towards the optimal UI design process. At the same time, (b) they help us to create guidelines for layered S3D UI design. However, there is a considerable lack of knowledge on how to structure and group information presented in 3D space, which is crucial for creating user interfaces for 3D displays [29]. Hence we aim to close the gap through the subsequent user study.
Depth Layout Tools for 3D UIs
In the previous section we explored the core requirements for designing and prototyping S3D UIs. With the tools presented in this section, we aim to provide a means for designers to adhere to these requirements.
The third dimension makes it particularly challenging to use existing, state of the art, protototyping tools for 2D user interfaces. As has been stressed earlier, paper has been shown to be highly useful in early stages of the development process [2] but is clearly limited when trying to arrange UI elements in 3D space. Hence, we built tools that can overcome the flat nature of paper for prototyping S3D layered UIs. The core idea is to augment paper prototyping in a way that maximizes the artistic freedom of the designer while providing implicit guidance towards usable products. In the following, we summarize the requirements and determine physical dimensions for tools that can support low fidelity prototyping in 3D space.
Viewing Distance and Depth Budget. First of all, designers need to identify the distance between viewer and screen on which the UI is being presented. Usually this distance is defined by the context in which the UI is to be used. For example, in cars the usual distance between driver and dashboard is 75 cm. The viewing distance then determines the depth budget in which UI elements should be positioned in order for the viewer to be comfortably perceived. Prior work showed this range to be 17.9 arc min to the front and 35.9 arc min to the back of the screen level [4].
Target Interface Dimensions. The second requirement is the envisioned size of the target interface (width and height), for example a public display, a mobile phone, or a dashboard. Together with the viewing distance and depth budget it then allows for calculating width, height, and depth.
Number and Position of Depth Layers. The depth layer positions depend (a) on the distance of the viewer from the screen, which determines the available space (comfort zone) in z-direction and (b) of the minimum depth distance (3.6 arc min, see previous section).
Catering to these requirements allowed us to build prototyping tools that support the user in creating UIs, that (1) minimize visual discomfort S3D can potentially cause and (2) allow different depth layers to be quickly and accurately distinguished. In this section we describe two novel prototyping tools – the FrameBox and the MirrorBox. Both tools were designed to be used for arbitrary S3D UI design tasks and to map the specified requirements in the best possible manner.
FrameBox
The core idea behind FrameBox is to allow users to work with a large variety of materials, including paper, foil, and 3D mockups created with a laser printer, and to spatially position the different elements. Hence, we designed a cubic box made of acrylic glass with a number of slots that represent the different depth layers and allow for positioning UI elements on the z-axis in discrete steps. Within each slot, UI elements can be easily moved in x-direction. Positioning on the y-axis can be achieved by means of paper-clips. In accordance to the specified requirements, we built a series of FrameBoxes for different application areas. One FrameBox was aimed for the design of automotive UIs and two for the design of mobile phones UIs (one for landscape and one for portrait mode).
The automotive FrameBox is based on the dimensions of a conventional freely programmable instrument cluster (12.3”)and a typical viewing distance of 750 mm. The viewing distance allows for positioning UI elements 44 mm to the front of the screen and 107 mm to the back of the screen in order to maintain the comfort zone. This results in the following dimensions of the FrameBox for an instrument cluster application: 293 x 110 x 151 mm (WxHxD).
For the mobile FrameBox we chose the size of the Samsung Galaxy S4 (5”)and a typical viewing distance of 350 mm. These values result in a comfort zone of up to 21 mm to the front and 50 mm to the back of the screen. The dimensions for the FrameBoxes amount to 111 x 62 x 71 mm.
All FrameBoxes were built using a laser cutter. The laser cutter templates are available for public use from our website1. As material we used transparent acrylic glass. A red line references the screen layer and makes positioning in front of or behind the screen easy for the designers (Figure 6).
MirrorBox
As a second prototyping tool, we designed the MirrorBox. We use a number of semi-transparent mirrors in the front and a surface-coated mirror at the back to allow users to see the mirrored image of a UI element projected from below (Figure 7). The mirrors are aligned one after another on top of a light source. Foils can be used to design UI elements, which are then sliced between the mirrors and the light source to make them visible to the user in front of the MirrorBox.
We constructed a multi-purpose MirrorBox consisting of 3 mirrors with a size of 125 x 78 mm. The mirrors are arranged behind one another and are horizontally tilted by 45 degrees. In this way, the mirrors generate three virtual layers by reflecting UI elements from below. We use a tablet as a lighting source. The mirrors have different reflectivity to provide the viewer with a similiar brightness for all three virtual layers. The rear mirror reflects almost 100%, the mirror in the middle has a reflectivity of 50% and the foremost mirror has 30% reflectivity. This results in a total reflectivity of between 30% and 35% for each virtual layer (Figure 8). Since the mirrors are tilted by 45 degrees, the virtual images have a height of 55 mm. The distance between the layers is 55 mm. The outer dimensions of the MirrorBox are 125 x 55 x 165 mm.
Discussion
This paper provides (a) basic guidelines for designing user interfaces for 3D displays, that were gathered through a literature review and complemented by a laboratory study, and (b) prototyping tools supporting the design process for S3D UIs. We used the guidelines for the design of the presented tools and for instructing the participants during the workshops.
In the workshops, we were able to observe that the participants made use of the presented guidelines to fulfill their tasks. While the MirrorBox restricts the depth layout to only three depth layers, the FrameBox provided participants much more artistic freedom. Nevertheless, the participants sticked to the guideline by using no more than six depth layers while prototyping with the FrameBox. Hence, this guideline could easily be followed by the users even if the tool does not explicitly implement this rule. For paper prototyping, we noticed that the participants had difficulties in observing the comfort zone as well as the minimal recommended distance between depth layers. Consequently, it is beneficial to integrate these guidelines in the tool in a way such that the designers can take the available depth relations into account. Furthermore, the participants proposed an iterative process which involves a first concept draft based on paper prototyping and a later refinement using the FrameBox.
The conducted workshops show that our tools complement the strengths of paper prototyping – particularly low effort and support for creativity – with high usability and means for expressiveness. Our workshop participants tried to cope with the fact that paper prototyping makes it difficult to position elements in 3D space. This comes at the cost of increased effort. For example, participants spent a considerable amount of time during the paper prototyping session to make post-its ‘stand’ behind each other to reflect several layers or tried to build 3D objects using paper. In contrast, the positioning of elements in 3D space is supported by MirrorBox and Frame- Box. The use of our tools allows participants to concentrate on ideas, exploring the positioning of objects, adjusting font, and object size. The participants liked to use paper prototyping in combination with the FrameBox as it has a low barrier for participation in the prototyping process but supports the spatial arrangement and the integration of 3D objects well. In contrast, the MirrorBox narrows the depth layout to three layers and does not allow the integration of 3D objects.
Furthermore, we have seen in the workshops that the prototyping tools help to effectively communicate design decisions within the team and to outside observers. The participants reported that it is easier to understand the idea of the other group as they presented their results with the MirrorBox or FrameBox in contrast to seeing the results just as a paper prototype. This finding was supported by the results of the questionnaires regarding the expressiveness of the approaches. A low barrier for participation and the expressiveness is especially important for the UI development process as it often involves developers with different backgrounds.
Participants also expressed a clear interest in increasing the fidelity of the MirrorBox. While in our case, the iPad was only used as backlight, they suggested to use an external monitor or tabletop surface that would allow to sketch UI elements and display them immediately on the iPad.
Finally, we see potential in augmenting our tools with 3D elements, for example, from a 3D printer. In the automotive workshop we observed participants that exploit ways of modeling 3D objects. One group used the upper back part of the sticky notes to attach interface elements to paper. Another group used transparent scotch tape to create objects like mountains or buildings for the navigation system.
Related Publications
N. Broy, S. Schneegass, F. Alt, and A. Schmidt, “FrameBox and MirrorBox: Tools for Prototyping User Interfaces for 3D Displays,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, NY, USA, 2014. |