Tactile Telerobot – the story so far

The Shadow Robot Company is proud to be a part of a new technological advancement stemming from a collaboration between a global engineering team. Together, we have pioneered the world’s first telerobot to transmit touch to the operator from around the world.

This type of telepresence allows the user to feel as if they were present at a place other than their true location.

We can now deliver remote touch and dexterity for people to build on for applications like safeguarding people from hazardous tasks, or just doing a job without having to fly there!

Rich Walker – Managing Director of the Shadow Robot Company

What does the technology consist of? 

The system integrates the pioneering technologies of the three organisations, each leaders in their categories: Shadow Robot Company (London & Madrid) in dexterous robotic hands, SynTouch (California) in tactile sensors, and HaptX (California) in realistic haptic feedback gloves.

The SynTouch BioTac sensor can sense everything your fingertip can feel including force, vibration, and temperature. It’s the most biomimetic tactile sensor available, and it’s a natural fit with the Shadow Dexterous Hand which is the most anthropomorphic hand for robots available. The two combined create the most human-like robotic hand available today.

HaptX Gloves provide the most realistic touch sensation ever achieved through a haptic wearable. The gloves apply detailed tactile feedback, powerful force feedback, and sub-millimeter motion tracking. While HaptX Gloves are primarily used for virtual reality, they are a natural fit for teleoperation and telepresence within robotics.

How it works:

Motion data captured by HaptX Gloves controls the movement of the anthropomorphic dexterous hand by Shadow Robot Company. SynTouch’s BioTac sensors are embedded in each fingertip of the robotic hand to collect tactile data that’s recreated as haptic feedback by HaptX Gloves is transmitted to the user’s hand. This sense of touch provides the user with a stimuli for advanced telepresence.

How we began testing it:

The international engineering team included members of SynTouch and HaptX in California and Shadow Robot Company in London and Madrid. In a demonstration of the system, an operator in California used a haptic glove to control a dexterous robotic hand in London under the guidance of team members from ANA Holdings (the parent company of ANA the airline), in Tokyo.

As the robot typed on a computer keyboard, tactile sensors on the robot’s fingertips detected the press of each key. Haptic data travelled over a network back to the human operator in California, who received the haptic feedback in real-time. The haptic telerobot’s first words typed on the system were, “Hello, World!”

Speed Multiplier x2

Why is it ground-breaking?

To our knowledge, this is the first time anyone’s typed words by controlling a robotic hand from a distance and it’s certainly the first time anyone’s been able to feel what they type through a robot. It’s a major advancement within teleoperation and telepresence in robotics.

“It’s not touch-typing yet, but we can feel what we touch when we’re typing!” says Rich Walker, Managing Director of the Shadow Robot Company.

“We know from psychophysical studies that the sense of touch is essential when it comes to dexterity and manipulation,” said Dr. Jeremy Fishel, Co-Founder of SynTouch. “This is the first time anyone has ever demonstrated a telerobot with such high-fidelity haptics and control, which is very promising and would not have been possible without the great engineers and technologies from this collaboration.”

Where can we use this kind of telepresence and teleoperation technology?

Three key ways:

One: Where the job is risky for the person

Example: Nuclear decommissioning, bomb disposal, construction and space exploration.

See here to find out more how Shadow Robot Company are using this innovative new technology to help transform work within radioactive environments and solve key industry challenges via remote handling in glove boxes.

Two: Where the person is a risk to the job

Example: The manufacture of pharmaceuticals, in which human beings can contaminate pharmaceuticals, but you have to have a human in the process because there are skilled manipulation steps.

Three: Where Immersive Robotic Technology is needed

Example: Creating a completely personalised, immersive travel experience by visiting a destination without having to fly there!

Leadership:

ANA Holdings funded and facilitated this integration as part of the company’s global Avatar initiative. Fully vested in catalysing the Avatar industry, ANA has one simple objective: to pioneer real-world Avatar technology to connect and positively impact the world.

Possible Future Developments:

Lower latency communications: Building better networks between the two ends of the system instead of/ in addition to the internet.

Immersive vision system and remote viewing: Such as a VR headset, a camera system and/or multiple screen views. We’re currently using off-the-shelf video conferencing software.

Arms that more appropriate for specific tasks: Different designs for robot arms that are suited to the constraints of the environment e.g. snake-arm for tasks that require more agility and/or longer reach.

Manoeuvrable mobile structure: Add the system to a linear track or wheel-based platform.

Grounded force feedback: measure the forces being applied to the arm and transfer it back to the operator.

Bi-manual system:  Two robot hands working at the same time (right and left hand) and better means of controlling this system.

New version of the Hand Plus: We currently use the Shadow Hand Plus (this is the latest, more reliable version of our five-fingered hand), however, further improvements can be made to inform future generations of the Hand.

Find out more!

Applicants of the ANA Avatar XPRIZE programme and interested customers can buy the full system or its components. Please visit our official tactile telerobot page to find out more along with purchase information!