Abstract

Due to advances in technology, displays could replace literally any surface in the future, including walls, windows, and ceilings. At the same time, midair remains a relatively unexplored domain for the use of displays as of today, particularly in public space. Nevertheless, we see large potential in the ability to make displays appear at any possible point in space, both indoors and outdoors. Such displays, that we call midair displays, could control large crowds in emergency situations, they could be used during sports for navigation and feedback on performance, or as group displays. We see midair displays as a complementary technology to wearable displays. In contrast to statically deployed displays they allow information to be brought to the user anytime and anywhere.We explore the concept of midair displays and show that with current technology, e.g., copter drones, such displays can be easily built. A study on the readability of such displays showcases the potential and feasibility of the concept and provides early insights.

Introduction

Displays can be found in many public spaces in the form of advertising and information displays or as artistic installations. Through novel display technologies (e.g., eInk, OLED), we envision that in the future, literally every surface could be transformed into a display, ranging from floors and walls to ceilings and windows. Such displays can reach passersby in different situations and places. Yet, they are usually installed in a fixed place and rely upon location, orientation, and viewer distance to be optimally perceived. In contrast to such static displays, we see large potential in autonomous, free-floating displays that can change their position to appear at any given point in space and approach the user – we refer to such displays as midair displays. In an emergency situation such as a fire or earthquake, statically deployed displays may become unusable due to power outage or may even be destroyed. In such cases, midair displays could be used to show emergency instructions and guidance to people. Further scenarios include navigation as well as (personalized) group displays delivering information to people doing outdoor sports or tourists exploring a city. With our work we aim to lay the foundation for future research on midair displays, particularly interaction with the display and means to adapt the display based on the user context. The contribution of this paper, in which we explore the concept of free-floating midair displays, is threefold. We first describe scenarios that outline the potential of the approach. Second, we present a functional prototype consisting of a copter drone to which we mounted a remotely controllable iPad. Third, we quantitatively and qualitatively explore the concept by demonstrating our prototype, running a brief reading test, and conducting a series of interviews.

A free-floating midair display

Current copters can fly at a maximum speed of 40 km/h and can carry up to 3.5 kg of payload. This allows for carrying 60 inch stateof- the-art e-ink displays. As such displays do not depend on backlighting they are readable even during exposure to direct sunlight. We envision that with further developments in copter technology and lightweight displays, the size of carried displays as well as the flight time can be increased and their price reduced. Depending on the use case we envision different form factors, shapes, and display technologies for mid-air displays. As a basic display for individuals and small groups we imagine a small scale (e.g., 20 cm x 30 cm) free floating e-paper display – much like a floating sheet of paper in front of the user. For larger groups we can believe larger planar display (e.g., 1m x 1 m) or different form factors (e.g., curved or cylindrically shaped displays) are suitable. Besides using displays, this concept can also be extended to projection technologies mounted on the copter, where the technical feasibility has been shown in [13]. By adding sensors such as a camera or distance sensor, midair displays could position themselves in a way that puts users into an optimal perspective. Furthermore, sensors can be used to control midair displays with gestures or speech commands, hence, making them a useful companion in everyday life. To better understand the potential of the concept we created two midair displays – first, a physical mockup of a lightweight display that showcases the envisioned form factor and second, a proof-ofconcept prototype based on an octocopter carrying a tablet.

Interactive MidAir Displays

We see particular potential in making midair displays interactive. This does not only include the control of the display itself (i.e., where the display is flying to), but also interaction with the content (e.g., browsing through information about a particular place). The following section discusses different means and challenges for interaction with midair displays. Today, an increasing number of interactive displays can be found in public space. Prior work identified a number of challenges when it comes to enticing passersby to interact, including means to raise the attention, communicate interactivity, and provide easy-to-understand interaction techniques [7].

We believe that midair displays are particularly suited to raise the attention of passersby since they can be made to appear in the visual field of the viewer. At the same time, this is also a challenge, since people currently not interested in the information provided by such displays may feel annoyed. Hence, future research should investigate the users’ view on such displays, the development of means to determine the current interest, and an understanding about in which cases it could be most valuable to approach people.

Communicating interactivity may be a major challenge for midair displays. Unlike for static displays where analog signage or the honeypot effect could make users aware of their interactive capabilities [8], we believe that midair displays need to employ means for communicating their interactivity mainly through their content. This can be done, for example, through calls-to-action or through attract sequences.

Touch, mobile phones, and gestures have been identified to be the most suitable interaction techniques for static displays. We believe the latter two techniques to be particularly promising for midair displays. From a technology perspective, mobile phone based interaction could be realized via an Internet connection or a local network (e.g., the midair displays employing a WiFi hotspot). For gestures, the display could employ a (depth) camera. Particular challenges include the movement of the display as well as exposure to sunlight in which it may be difficult to correctly recognize the user’s input.

For controlling the movement of the midair display, we imagine both explicit and implicit means to do so. Today, drones are mostly steered through specialized remote controls. Some drones, however, can already be controlled using mobile phone apps, for example, through tilting the phone in the direction to fly. At the same time, gesture-based control may be suitable, where the user is simply pointing to a direction the display should move to. We believe the optimal interaction technique to strongly depend on the task and the distance to the user. For many of the presented scenarios, implicitly controlling the midair displays seems most natural. For example in the sports and crowd control scenario, the midair display should approach and follow the user. At the same time it may need to efficiently guide the user towards a particular direction. From a technology perspective, this infers also some challenges. For instance, as soon as the user squints into the sun, the drone would need to move to increase the readability of the attached display.

Technical Limitations

Current copter drones have some technical limitations that need to be addressed to be usable in the wild. First, current copter drones only have a flight time of about 10-20 minutes. By carrying heavy objects, the flight time further decreases. As possible solutions, we see a combination of blimps and copters as promising. The blimps creates uplift so that the flight time increases and the rotors are mainly used for steering. As another possible solution on-the-fly charging could be possible – either through solar charging or onthe- fly battery pack replacement. A large flying base-station could provide docking stations with fully charged battery packs. Additionally, current copter drones are quite noisy. Talking directly next to a flying copter drone is hardly possible. Particularly for the group information and flying guide scenario, this limitation needs to be overcome, for mid-air displays to become ubiquitous.

Discussion and Conclusion

In this project we introduce the concept of midair displays and present a prototype which combines an octocopter with a 10″ display. In addition we showcase scenarios in which such displays can be useful. Interviews reveal that participants see the main benefit for situation, in which mobile phones as well as glasses are difficult to be used (e.g., during swimming or skiing) and where content needs to be shared with a group of people (e.g., in emergency situations). We see mid-air displays as a complementary technology to wearable displays (such as Google Glass) rather than a competitive technology.

During a user study, we gained insights into how mid-air displays can be used. Our results indicate that knowing whether the user is standing or walking is a central prerequisite. Particularly in situations in which users need to be able to precisely recognize the display content, this information could be used to adapt the layout and font size accordingly. Furthermore, the adaption to the number of users is important and, thus, sensors for tracking them (e.g., cameras) should be integrated with mid-air displays.

Participants saw large potential for situation in which they are approached by the display. Similar to static displays, value could be added by providing personalized services rather than scattershot ads. Hence, midair displays could likely benefit from knowledge about public displays. Future work could focus on more realistic reading tasks, means to identify the user, provide suitable interaction techniques such as remote interaction [2], and investigate social acceptance.

Related Publications