Completed2023-2024

Haptic Intelligence: Robotic Sensing Meets Machine Learning

Learning to tell objects apart from a multi-layer electronic skin

Student
Zhihua Zhang
Degree
MSc Bioengineering
Institution
Imperial College London
My role
Primary supervisor

Co-supervised with Prof. Etienne Burdet, Dr Xiaoxiao Cheng, Anirvan Dutta.

Haptic Intelligence: Robotic Sensing Meets Machine Learning
Image
1 / 5
e-skintactile sensingmachine learningroboticsdataset

The question

Can an electronic skin give a robot something like a human sense of touch: telling objects apart by their shape, stiffness and surface, and recognising a property it has never met before? And does stacking a second sensing layer actually help?

What was built

A multi-layer e-skin assembly mounted on a robot arm, with an experimental rig that pressed, rotated and slid the skin over sets of soft and rigid objects: embossed letters, wave profiles, and pads of varying stiffness. The recordings became a reusable dataset for tactile machine learning. An unsupervised model (a variational autoencoder) compressed the high-dimensional skin data into a small latent space, where objects were clustered and compared, for single-layer and dual-layer skins.

What was found

The learned representation separated the objects well: clustering in the latent space recovered the object identity for most of the 36 items, best for sliding contact. The second layer raised classification accuracy only modestly, by about 8 %, but was clearly better at resolving subtle differences such as flat versus convex features and small changes in stiffness. A model trained on one family of objects partially generalised to another it had never seen, which is the property a general tactile sense needs.