Skip to main content
Talk

Analysis of Spatio-temporal Representations for Robust Footstep Recognition with Deep Residual Neural Networks

On May 21, 2018, Dr. Omar Costilla Reyes presented deep residual networks for footstep recognition at the Advances in Data Science Conference. The two-day conference was held at the Museum of Science and Industry in Manchester.

Speaker badge for Dr. Costilla Reyes at the Advances in Data Science Conference, Manchester, May 2018. Credit: Image courtesy of Dr. Omar Costilla Reyes.
Title slide of the May 21, 2018 talk, naming the presenter’s Manchester research group and his incoming MIT postdoctoral position. Credit: Image courtesy of Dr. Omar Costilla Reyes.
Announcement graphic for the Advances in Data Science Conference, held May 21–22, 2018. Credit: Image courtesy of Dr. Omar Costilla Reyes.

A final Manchester conference before MIT

Advances in Data Science 2018 brought delegates from around the world to the Museum of Science and Industry in Manchester for a two-day international conference on recent developments in data science and its potential to support societal well-being, jointly organized by the University of Manchester’s Data Science Institute and the Cathie Marsh Institute for Social Research. Dr. Costilla Reyes was announced as a speaker in April and presented on the first day, May 21, alongside talks from the Office for National Statistics and UN Global Pulse. He described it afterward as his last conference before leaving for MIT.

The talk, given with coauthors Patricia Scully and Krikor Ozanyan, presented “Analysis of Spatio-temporal Representations for Robust Footstep Recognition with Deep Residual Neural Networks,” the paper published in IEEE Transactions on Pattern Analysis and Machine Intelligence in 2018. A biometric is a measurable characteristic used to recognize a person; here the characteristic is the pattern a person leaves on a floor sensor while walking. The spatio-temporal representation keeps both the shape of each footprint and its timing, and a deep residual network, which uses shortcut connections so that many layers can be trained together, learns layered features from those signals. The aim is to tell legitimate users apart from impostors despite the natural variation in how people walk.

The paper tested the approach on a large database of footsteps from over a hundred people, in three scenarios defined by how much training data is available: airport checkpoints, workplaces, and homes. The approach reached low error rates, which supports learned spatio-temporal representations as a practical basis for recognizing people from the way they step on a floor, whether a system has a little training data or a great deal.

Press coverage and what followed

The research drew press attention in 2018, including University of Manchester news articles in May and June and a Deutschlandfunk feature on German national radio in July. It followed his invited talk at the University of Cambridge Computer Laboratory in January and the seminar at the Max Planck Institute for Intelligent Systems in March, and it closed the biometric strand of his doctoral work before his move to computational neuroscience at MIT. The University of Manchester awarded his PhD in Electrical and Electronic Engineering in May 2018, and he began his MIT fellowship in June.

Topics