Before exploring the concept of gesture-based computing, it may be useful to first look at the larger category that it encompasses - User Interface, or UI. The notion of interacting with machines to solve problems is not a new one, and Karl Tate, of LiveScience, has done a rather good job of hitting the touchstones in the following infographic. I have chosen to link out to the graphic since its native site has much better resolution (and readability) than importing it directly into this page.
Karl Tate's UI Infographic
WIMP GUI:
For most of us, the concept of the WIMP GUI is the most familiar and comfortable; so much so that most of us interact with it completely naturally and with very little thought. Though some confusion may have gone into the foreboding acronym at the beginning of that last sentence. WIMP (Windows, Icons, Menus, Pointers) is the Graphical User Interface (GUI) that almost all of our computer technology operates on. The interface was piloted by XEROX, but did not gain widespread popularity until Apple utilized it with their Macintosh in 1984. WIMP GUI remains the most widely used interface even today, thirty years later, but its main drawback is that it is an entirely abstract notion, and not immediately intuitive.
This drawback hinges on the fact that the mouse is an abstract extension of the user's hand. It moves, points, pushes, and grabs, but these tasks are completed separately from the screen. What we need, as users, is the ability to touch something. And that ability had been in development as early as the 1960s.
Touch Interface:
Touch interface found its origins in the late sixties with musicians experimenting with touch response on synthesizers and electronic instruments, and then the release of the PLATO IV, an educational computer developed by Control Data, which utilized a 16x16 inch touchscreen. The University of Toronto's Input Research Group produced the first true multi-touch interface in 1982 (two years prior to the release of the Mac and its WIMP interface); the display was capable of recognizing multiple fingers as well as registering very basic pressure sensitivity.
The concept of touch interface continued to be explored over the course of the next quarter century, and many of the gestures that we are familiar with today were developed:
- Tapping to select items
- Pinching and releasing to zoom the camera
- Swiping to scroll the screen
However, it could be argued that touch interface did not truly "take-off" until the introduction of the Apple iPhone in 2007.
Steve Jobs, Apple Keynote, iPhone Introduction, 2007:
The introduction of the iPhone, the integration of multi-touch technology to the iPod, and the subsequent introduction of the iPad, brought touch and gesture based computing to the vast majority of the population. It could also be argued that the introduction of Apple's devices "upped the game" for the competition, and companies such as Microsoft and Android released their own products that provided effective touch interface.
A final argument could be made regarding the notion of whether Apple truly invented the technologies that are used in their devices, as they state, or if the more correct notion might be that they made some very important innovations to existing technology. But that is a topic for another conversation.
Gesture-Based Computing:
So, Apple and its competitors have made multi-touch GUI's, and the gestures that control them, commonplace, but computer interaction that is solely gesture-based is a concept that is only now starting to enter into the everyday tech experience.
Perhaps one of the most interesting, comprehensive, and accessible gesture-based computing systems is the the G-Speak/Mezzanine set-up that has been created by Oblong Industries. Oblong is headed by John Underkoffler, an MIT grad, who was also one of the main consultants for the gesture-based technologies that were on display in the 2002 film, Minority Report. Underkoffler can be seen in the video below, from 2010, demonstrating G-speak - the real world technology behind the concepts seen in the movie.
John Underkoffler, TEDtalk, 2010:
Some interesting characteristics of the G-speak system are its unique gestures and its ability to process the user, and pixels, in actual 3-dimensional space. Particularly intriguing is when Underkoffler demonstrates the ability to pull an element from the front display onto the "worktable" display in front of him. Robert Scople, of Rackspace, interviews Underkoffler at the Bay Area headquarters for Oblong in the following two videos - providing a much more in depth look at the specific gestures and capabilities of the gesturing systems of G-speak, as well as how these gestures have been adapted into the Mezzanine system.
Oblong Interview, Part I:
Oblong Interview, Part II:
A few things really strike me regarding Oblong's systems - the first is that the G-speak gesturing system is incredibly complex and expensive in terms of its sensors, and it still requires the user to wear gloves for the gestures to be recognized completely. And while that seems somewhat negative, the second observation is positive - early on in the interview, Underkoffler mentions that the entire system has been designed to run in tandem with extant operating systems - Windows, OSX, and Linux - and machines.
Unfortunately, this hardware/software setup is still out of reach for most users - Oblong Industries openly admits that their system is currently only available at an enterprise level. And while they will not publicly state a price, it can be inferred from the videos above that the cost of setting up the Mezzanine system in a conference room would easily be in the six figures.
Gesture-based Computing for the Masses:
More consumer friendly adaptations of gesture-based computing have been made available to the public, but are mostly relegated to the genre of gaming. Microsoft's Kinect, released in 2010 for its Xbox 360 game console and then again in 2012 for Windows-based PCs, provides users with both speech and gesture recognition, but no where near the level of fidelity presented by the Oblong systems. Continuing the trend of bringing gesture-based computing to every user, the Leap Motion controller was released by Leap Motion, Inc. in 2013. The controller is designed for use with personal computers, and functions in a much smaller physical space than the Kinect, focusing on hand gestures to control apps onscreen. Reviews have been mixed so far with the Leap Motion sensor, but Oblong has begun developing prototypes that utilize the technology, and Underkoffler openly states his enthusiasm for the device in the interviews above, so it should be very interesting to see how gesture-based UI's develop in the very near-term future.
No comments:
Post a Comment