Project Avatar - An immersive, gesture controlled telepresence robotics system

Project Avatar - An immersive, gesture controlled telepresence robotics system

Martin Foertsch

Martin Foertsch

Unterföhring, Bayern

Project Avatar is an immersive, gesture controlled, multimodal telepresence robotics system which allows a real out-of-body experience.

Game Development, Intel RealSense™, Robotics, Internet of Things

  • 2 Collaborators

  • 5 Followers

    Follow

Description

"Avatar" is an American science fiction film by James Cameron premiered in the year 2009. The film is about humans from earth colonizing the planet Pandora which is populated by the local tribe Na'vi. Na'vis are a humanoid species indigenous in Pandora. To create trust the humans developed a system which makes it possible to incarnate as a real Na'vi. In the film Captain Jake Sully (Sam Worthington) slipped into the role of such a native of this local tribe on Pandora to accomplish one’s mission. With this very special human technology he was able to look and feel like a real Na'vi.

The goal of "Project Avatar" was to build a project where you can have such an out-of-body experience using off-the-shelf hardware. The idea was to control a robot at another location using full body gesture control. The teleoperator on the the remote is equipped with sensors to gather environmental data. Using feedback channels this data can be transferred back to the human controller.

The human operator actually should be able to see the world through the eyes of the Nao robot including the field of view in 3D. This is realizable by using RGB stereo cameras. Using tactile sensors the robot is able to send touch-events over the communication channel back to the human controller. So the controller can actually feel sensor based events occurring in the environment at the robots site comparable to force-feedback.

Our plan is to introduce the Intel RealSense camera for full body tracking and Intel Euclid as additional camera sensor to extend the senses of the Nao robot. The Intel IoT Gateway is implemented as well to transfer the huge amount of data seamlessly.

Links

Showcase Demonstration Video

Mjvkm9o8

Martin F. added photos to project Project Avatar - An immersive, gesture controlled telepresence robotics system

Medium f43acf32 d346 4e03 857f 43c779329198

Project Avatar - An immersive, gesture controlled telepresence robotics system

"Avatar" is an American science fiction film by James Cameron premiered in the year 2009. The film is about humans from earth colonizing the planet Pandora which is populated by the local tribe Na'vi. Na'vis are a humanoid species indigenous in Pandora. To create trust the humans developed a system which makes it possible to incarnate as a real Na'vi. In the film Captain Jake Sully (Sam Worthington) slipped into the role of such a native of this local tribe on Pandora to accomplish one’s mission. With this very special human technology he was able to look and feel like a real Na'vi.

The goal of "Project Avatar" was to build a project where you can have such an out-of-body experience using off-the-shelf hardware. The idea was to control a robot at another location using full body gesture control. The teleoperator on the the remote is equipped with sensors to gather environmental data. Using feedback channels this data can be transferred back to the human controller.

The human operator actually should be able to see the world through the eyes of the Nao robot including the field of view in 3D. This is realizable by using RGB stereo cameras. Using tactile sensors the robot is able to send touch-events over the communication channel back to the human controller. So the controller can actually feel sensor based events occurring in the environment at the robots site comparable to force-feedback.

Our plan is to introduce the Intel RealSense camera for full body tracking and Intel Euclid as additional camera sensor to extend the senses of the Nao robot. The Intel IoT Gateway is implemented as well to transfer the huge amount of data seamlessly.

Medium mjvkm9o8

Martin F. created project Project Avatar - An immersive, gesture controlled telepresence robotics system

Medium f43acf32 d346 4e03 857f 43c779329198

Project Avatar - An immersive, gesture controlled telepresence robotics system

"Avatar" is an American science fiction film by James Cameron premiered in the year 2009. The film is about humans from earth colonizing the planet Pandora which is populated by the local tribe Na'vi. Na'vis are a humanoid species indigenous in Pandora. To create trust the humans developed a system which makes it possible to incarnate as a real Na'vi. In the film Captain Jake Sully (Sam Worthington) slipped into the role of such a native of this local tribe on Pandora to accomplish one’s mission. With this very special human technology he was able to look and feel like a real Na'vi.

The goal of "Project Avatar" was to build a project where you can have such an out-of-body experience using off-the-shelf hardware. The idea was to control a robot at another location using full body gesture control. The teleoperator on the the remote is equipped with sensors to gather environmental data. Using feedback channels this data can be transferred back to the human controller.

The human operator actually should be able to see the world through the eyes of the Nao robot including the field of view in 3D. This is realizable by using RGB stereo cameras. Using tactile sensors the robot is able to send touch-events over the communication channel back to the human controller. So the controller can actually feel sensor based events occurring in the environment at the robots site comparable to force-feedback.

Our plan is to introduce the Intel RealSense camera for full body tracking and Intel Euclid as additional camera sensor to extend the senses of the Nao robot. The Intel IoT Gateway is implemented as well to transfer the huge amount of data seamlessly.

Bigger mjvkm9o8
Black Belt
  • Projects 4
  • Followers 65

Martin Foertsch

Intel Software Innovator and Black Belt, Oracle JavaOne Rock Star, Computer Scientist, Speaker, Author for ParrotsOnJava.com

Beta-Straße 13A, 85774 Unterföhring, Germany

Bigger bob duffy 3d head avatar
  • Projects 4
  • Followers 126

Bob Duffy

Folsom, CA, USA

Bigger screenshot www.facebook.com 2016 03 31 16 28 35
  • Projects 0
  • Followers 65

Wendy Boswell

Program Manager, Intel Software Innovators

Portland, OR, USA