Skip to main content
Talk

Analysis of Spatio-temporal Representations for Robust Footstep Recognition

Dr. Omar Costilla Reyes was invited to speak at the University of Cambridge Computer Laboratory on January 25, 2018, on footstep recognition. The talk covered spatio-temporal representations learned with deep residual networks.

Outside the William Gates Building, home of the University of Cambridge Computer Laboratory, on the day of the footstep-recognition seminar, January 25, 2018. Credit: Image courtesy of Dr. Omar Costilla Reyes.
Architecture figure for the footstep-recognition paper: each stream passes through convolution blocks with skip connections, a fully connected layer, and a linear support vector machine before score fusion. Shared when the paper was published in IEEE TPAMI, February 2018. Related photograph from Analysis of spatio-temporal representations for robust footstep recognition with deep residual neural networks. Credit: Image courtesy of Dr. Omar Costilla Reyes.

A seminar at the Computer Laboratory

On January 25, 2018, Dr. Omar Costilla Reyes gave a seminar in the William Gates Building, home of the University of Cambridge Computer Laboratory, which listed it as a Systems Research Group Seminar. He was then a research associate and doctoral candidate in Electrical and Electronic Engineering at the University of Manchester, and he had announced two days earlier that he would present research recently accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence.

Recognizing people by their footsteps

Footstep recognition is a biometric method: it asks whether the pressure pattern of a person’s footsteps, recorded by a sensor in the floor, can verify who is walking. The talk proposed spatio-temporal representations of footsteps built from floor-sensor data alone, meaning patterns of pressure across the floor that also change over time, and evaluated them in deep residual neural networks, which add shortcut connections between layers so that very deep models remain trainable. The models were designed to separate the fine-grained variability of footsteps between legitimate users of a system and impostors. The seminar listing described a large database of footstep signals from many people, with many impostors and few clients, and three scenarios defined by how much training data is available: airport security checkpoints, workplaces, and homes. The listing reported recognition rates that improved on earlier work, which supports learned spatio-temporal representations as a basis for footstep biometrics that a floor sensor captures without a camera. The architecture figure on this page shows the two parallel networks, one reading a single footstep image and one reading a sequence of frames, whose scores are fused.

Connection to his doctoral research

The seminar drew directly on the paper “Analysis of spatio-temporal representations for robust footstep recognition with deep residual neural networks,” written with Rubén Vera-Rodríguez, Patricia Scully, and Krikor Ozanyan, and on the floor-sensor gait research of his University of Manchester thesis. It belongs to a sequence of 2017–2018 talks, including the Arm Research Summit, the Max Planck Institute seminar, and the Advances in Data Science Conference, in which he presented the same line of biometric work to different research communities.

Topics