After reading through your blog, all I can say is "WOW"! Before I read the History and Purpose of gesture-based computing, I honestly had no familiarity with the term. While learning about this emerging trend, I realized that gesture-based computing is something that we all interact with on a daily basis. I can remember the days of my old Motorola cell phone, then my old slide phone with T-9 texting, and thinking that I would never be able to switch from that to typing on an iPhone, for example. Now, I can't imagine not having a smart phone that worked with a touch screen. There are definitely times where I feel that a computer mouse is limiting, as you said, and I wish I could just touch the screen to manipulate what I needed. As you wrote, it is counterintuitive to have to use a device (such as a mouse, keyboard, game controller) to control what we see in front of our eyes. I feel that part of this is simple eye-hand coordination. The development of gesture-based computing over the years is astonishing. I was amazed by the video that you posted of Francis Tsai, and his artwork that is created using only eye motion sensors. Thank you for explaining this emerging trend in a way that anyone can relate to and understand!
My thoughts on the limits of the keyboard and mouse are only reinforced every time I see my sons (2 and 4 years old) effortlessly navigate the iPad or iPhone and then sit completely flummoxed in front of our desktop. I have to admit that I am a little wary of purely gesture-based systems though. I think there is something inherently human about being able to touch what we are interacting with - after all, that's what the mouse is intended to simulate (a poke or a grab when we click the button) - so I feel as though I will always gravitate more towards touch screens than something like the G-Speak system.
As for Francis Tsai, this sort of technology really blows me away. I think the levels of integration that we're going to see over the next few years are going to be right out of science fiction. Thanks for the kind words!
I really learned a lot by reading your blog! Gesture-based computing is getting bigger and bigger. While we all cannot gain access to our own personal Jarvis, I do not think we are far off from that possibility! I think it is interesting that the Xbox One, one of the newest and most sought after gaming consoles, is sold with the Kinnect. The fact that every user of the Xbox One will have gesture-based gaming capabilities, I believe that most, if not all, new games made for the system will incorporate the Kinnect into the game itself. I am excited about how this may impact music education in the future. Music is very difficult to teach from a distance. With gesture-based computing, it may become more interactive, simulating an in-person experience. Thanks for all your hard work on your blog on gesture-based computing!
You bring up some good points regarding the Xbox and the Kinect, and I have to confess that I have little experience with the latter. Though it did become apparent that there is a marked jump between the fidelity of the consumer level systems such as the Kinect and Leap and the systems used by Oblong. So it should be interesting to see what kinds of gains are made at the level where everyday consumers will be able to take advantage of them.
I loved your introduction paragraph; I was immediately hooked. I also watched the entire TED talk video you placed on the blog and the gesture based mathematics would be incredible in the classroom. The ideas can be used for volume, surface area, and other geometric formulas and properties. It was also interesting that the movie "minority report" was based off of old MIT problems. The video really does remind me of jarvis! (It's coming soon, can't wait either).
I like how to talk about the kinect being very similar to this technology which is true, but if you have ever truly used a kinect, it is terribly inaccurate and some moments in time and very quirky. It will be great when the motion is one to one and has zero lag input.
Finally, the gesture technology is really incredible in the Francis Tsai drawing piece you wrote about. The eye gaze device is incredible for disabled individuals and its insane that he can do all of this work with just his eyes. This is truly life changing for some people. For the technology to run this, it must be expensive. You wrote about G-Speak being one nof the main gesture interface technologies and I was wondering if you can purchase it for home use. Is there a home package? If so, how much does it cost?
I can imagine the possibilities with gesture-based work and teaching volumetric mathematics being very exciting.
As for the Kinect, I don't have any hands-on experience, but I have read several reviews that point out the very things that you mention. The issue of lag time (or relative lack thereof) also seems to be something that Leap Motion prides itself on.
And, as for G-speak's availability. There's a spot at the end of one of the videos where the interviewer guesses at a price and the developers, while not affirming any specific number, indicate that it is intended for the "enterprise" level. So I'm guessing somewhere north of six figures. However, oblong offers a service called Greenhouse for developers that allows access to their development suite for free - as long as everything is for personal or educational use. Greenhouse can be accessed through their website, and there are some pretty slick videos out there with devs using it in tandem with the Leap Motion controller and iPhones.
I'm really glad someone chose this topic to blog about because it is something that I don't know a lot about. From the name of it, I had a good guess as to what it involved, but I was not aware of all of the various technology that has been developed for gesture based computing and all of its uses. I think this would be a wonderful learning experience for the students in many of our Allied Health programs at Delaware Tech. One of these devices could detect their movements to simulate how they would work with a patient. This would give them practice, but without the risks of dealing with actual people. I hope that as the technology develops further, some of these devices become more affordable so that we can use it with our students!
You bring up an excellent idea with the incorporation of physical therapy and medical application simulation. I think this is one of the biggest strengths of incorporating gesture-based interaction with technology, especially if we can work the kinks out of the lag-time, as James mentions above.
Where this gets particularly exciting for me is the incorporation of tactile feedback. If developers can provide some kind of sense of feedback for the user - surface tension/quality, etc. I think that starts to open up all kinds of possibilities for creation.
This technology will prove to be effective in making students with disabilities more successful and motivated in school. Students with disabilities will be able to navigate a computer system much more easier now with this up and coming technology.
Smaller handheld devices can now save more space for better hardware by removing the buttons and creating more hand gestures to navigate through the software. A good example of this is the iPad/iPhone, it has 5 buttons the rest of the space is used for a much needed bigger screen size.
And in the end gestured based computing just looks cool too.
If you get a chance check out this short video. http://www.youtube.com/watch?v=aXV-yaFmQNk
That's definitely a though-provoking video, but I'm not really sure how to process it. Watching my children operate iOS is simply astounding, they are completely at home with the navigation and interaction. And while this certainly points to the efficiency in the design, I don't know if I would be so quick to throw out the importance of print and physical objects.
I guess this is my first "old" moment, even though I'm only 30. I have always enjoyed the feeling of something in my hand, and at least right now, I still have a hard time wrapping my head around the fact that "stuff" on all of the pretty screens is "real."
Though I am somewhat of a hypocrite here as well. Just before Christmas, I downloaded the Kindle app on my iPhone and now have several novels loaded on my phone. I love reading on it. However, I have two shelves full of art books in my study that are literally sagging from the weight. Maybe it's simply in the purpose of what I'm doing - but when I look at art, I want to hold it in my hand. Words...not as much.
Anyway, that got away from gesture-based and into philosophy, or something. Thanks for pointing out that video, it really does give great insight into the basic human-level intuition of the gestures.
Robert,
ReplyDeleteAfter reading through your blog, all I can say is "WOW"! Before I read the History and Purpose of gesture-based computing, I honestly had no familiarity with the term. While learning about this emerging trend, I realized that gesture-based computing is something that we all interact with on a daily basis. I can remember the days of my old Motorola cell phone, then my old slide phone with T-9 texting, and thinking that I would never be able to switch from that to typing on an iPhone, for example. Now, I can't imagine not having a smart phone that worked with a touch screen. There are definitely times where I feel that a computer mouse is limiting, as you said, and I wish I could just touch the screen to manipulate what I needed. As you wrote, it is counterintuitive to have to use a device (such as a mouse, keyboard, game controller) to control what we see in front of our eyes. I feel that part of this is simple eye-hand coordination. The development of gesture-based computing over the years is astonishing. I was amazed by the video that you posted of Francis Tsai, and his artwork that is created using only eye motion sensors. Thank you for explaining this emerging trend in a way that anyone can relate to and understand!
Caitlin,
DeleteMy thoughts on the limits of the keyboard and mouse are only reinforced every time I see my sons (2 and 4 years old) effortlessly navigate the iPad or iPhone and then sit completely flummoxed in front of our desktop. I have to admit that I am a little wary of purely gesture-based systems though. I think there is something inherently human about being able to touch what we are interacting with - after all, that's what the mouse is intended to simulate (a poke or a grab when we click the button) - so I feel as though I will always gravitate more towards touch screens than something like the G-Speak system.
As for Francis Tsai, this sort of technology really blows me away. I think the levels of integration that we're going to see over the next few years are going to be right out of science fiction. Thanks for the kind words!
I really learned a lot by reading your blog! Gesture-based computing is getting bigger and bigger. While we all cannot gain access to our own personal Jarvis, I do not think we are far off from that possibility! I think it is interesting that the Xbox One, one of the newest and most sought after gaming consoles, is sold with the Kinnect. The fact that every user of the Xbox One will have gesture-based gaming capabilities, I believe that most, if not all, new games made for the system will incorporate the Kinnect into the game itself. I am excited about how this may impact music education in the future. Music is very difficult to teach from a distance. With gesture-based computing, it may become more interactive, simulating an in-person experience. Thanks for all your hard work on your blog on gesture-based computing!
ReplyDeleteAmy,
DeleteYou bring up some good points regarding the Xbox and the Kinect, and I have to confess that I have little experience with the latter. Though it did become apparent that there is a marked jump between the fidelity of the consumer level systems such as the Kinect and Leap and the systems used by Oblong.
So it should be interesting to see what kinds of gains are made at the level where everyday consumers will be able to take advantage of them.
Robert,
ReplyDeleteI loved your introduction paragraph; I was immediately hooked. I also watched the entire TED talk video you placed on the blog and the gesture based mathematics would be incredible in the classroom. The ideas can be used for volume, surface area, and other geometric formulas and properties. It was also interesting that the movie "minority report" was based off of old MIT problems. The video really does remind me of jarvis! (It's coming soon, can't wait either).
I like how to talk about the kinect being very similar to this technology which is true, but if you have ever truly used a kinect, it is terribly inaccurate and some moments in time and very quirky. It will be great when the motion is one to one and has zero lag input.
Finally, the gesture technology is really incredible in the Francis Tsai drawing piece you wrote about. The eye gaze device is incredible for disabled individuals and its insane that he can do all of this work with just his eyes. This is truly life changing for some people. For the technology to run this, it must be expensive. You wrote about G-Speak being one nof the main gesture interface technologies and I was wondering if you can purchase it for home use. Is there a home package? If so, how much does it cost?
James,
DeleteI can imagine the possibilities with gesture-based work and teaching volumetric mathematics being very exciting.
As for the Kinect, I don't have any hands-on experience, but I have read several reviews that point out the very things that you mention. The issue of lag time (or relative lack thereof) also seems to be something that Leap Motion prides itself on.
And, as for G-speak's availability. There's a spot at the end of one of the videos where the interviewer guesses at a price and the developers, while not affirming any specific number, indicate that it is intended for the "enterprise" level. So I'm guessing somewhere north of six figures. However, oblong offers a service called Greenhouse for developers that allows access to their development suite for free - as long as everything is for personal or educational use. Greenhouse can be accessed through their website, and there are some pretty slick videos out there with devs using it in tandem with the Leap Motion controller and iPhones.
I'm really glad someone chose this topic to blog about because it is something that I don't know a lot about. From the name of it, I had a good guess as to what it involved, but I was not aware of all of the various technology that has been developed for gesture based computing and all of its uses. I think this would be a wonderful learning experience for the students in many of our Allied Health programs at Delaware Tech. One of these devices could detect their movements to simulate how they would work with a patient. This would give them practice, but without the risks of dealing with actual people. I hope that as the technology develops further, some of these devices become more affordable so that we can use it with our students!
ReplyDeleteLaura,
DeleteYou bring up an excellent idea with the incorporation of physical therapy and medical application simulation. I think this is one of the biggest strengths of incorporating gesture-based interaction with technology, especially if we can work the kinks out of the lag-time, as James mentions above.
Where this gets particularly exciting for me is the incorporation of tactile feedback. If developers can provide some kind of sense of feedback for the user - surface tension/quality, etc. I think that starts to open up all kinds of possibilities for creation.
This technology will prove to be effective in making students with disabilities more successful and motivated in school. Students with disabilities will be able to navigate a computer system much more easier now with this up and coming technology.
ReplyDeleteSmaller handheld devices can now save more space for better hardware by removing the buttons and creating more hand gestures to navigate through the software. A good example of this is the iPad/iPhone, it has 5 buttons the rest of the space is used for a much needed bigger screen size.
And in the end gestured based computing just looks cool too.
If you get a chance check out this short video. http://www.youtube.com/watch?v=aXV-yaFmQNk
Chris,
ReplyDeleteThat's definitely a though-provoking video, but I'm not really sure how to process it. Watching my children operate iOS is simply astounding, they are completely at home with the navigation and interaction. And while this certainly points to the efficiency in the design, I don't know if I would be so quick to throw out the importance of print and physical objects.
I guess this is my first "old" moment, even though I'm only 30. I have always enjoyed the feeling of something in my hand, and at least right now, I still have a hard time wrapping my head around the fact that "stuff" on all of the pretty screens is "real."
Though I am somewhat of a hypocrite here as well. Just before Christmas, I downloaded the Kindle app on my iPhone and now have several novels loaded on my phone. I love reading on it. However, I have two shelves full of art books in my study that are literally sagging from the weight. Maybe it's simply in the purpose of what I'm doing - but when I look at art, I want to hold it in my hand. Words...not as much.
Anyway, that got away from gesture-based and into philosophy, or something. Thanks for pointing out that video, it really does give great insight into the basic human-level intuition of the gestures.