Skip to main content
PhD thesis

Pattern recognition from raw spatio-temporal data for gait analysis in healthcare and security

Dr. Omar Costilla Reyes’s 2018 University of Manchester PhD thesis applies machine learning to floor-sensor gait analysis for healthcare and security. Two floor sensors support one line of work on healthcare and one on biometric security.

Dr. Costilla Reyes beside his poster on smart-carpet gait analysis at the EuroScience Open Forum 2016 in Manchester, July 2016. Related photograph from What Can Machine Learning Infer about the Way You Walk? Credit: Image courtesy of Dr. Omar Costilla Reyes; poster by Omar Costilla Reyes, Patricia Scully, and Krikor Ozanyan.
Doctoral research on multimodal sensor data and early dementia detection presented at a NeuroDPlatform session, University of Manchester, March 8, 2017. Related photograph from Deep Learning and Multimodal Sensing for Early Dementia Research — NeuroDPlatform. Credit: Image courtesy of Dr. Omar Costilla Reyes.
Architecture figure for the footstep-recognition paper: each stream passes through convolution blocks with skip connections, a fully connected layer, and a linear support vector machine before score fusion. Shared when the paper was published in IEEE TPAMI, February 2018. Related photograph from Analysis of spatio-temporal representations for robust footstep recognition with deep residual neural networks. Credit: Image courtesy of Dr. Omar Costilla Reyes.

Research contribution

This PhD research studies human gait and footsteps as fine-grained spatio-temporal signals, patterns that vary across both space, where pressure falls on the floor, and time, as a step unfolds. It connects machine learning and sensor systems to two applied settings: healthcare, where gait can reflect cognitive and neurological change, and security, where footsteps can support biometric verification. Dr. Costilla Reyes carried out the work from October 2014 in the Sensing, Imaging and Signal Processing group of the University of Manchester’s School of Electrical and Electronic Engineering, supervised by Krikor Ozanyan and Patricia Scully, with Iracema Leroi and Rubén Vera-Rodríguez as collaborators on the healthcare and security themes respectively. A CONACyT fellowship from Mexico’s national science council supported the doctorate.

Two floor sensors, two themes

The thesis designs deep neural networks that learn features from the sensor signals automatically, rather than relying on hand-crafted gait features, and compares them with classical machine-learning methods. The healthcare theme used the Manchester smart carpet, a floor sensor of plastic optical fibres whose light transmission changes under the pressure of a footstep. It produced the UoM-Gait-13 and UoM-Gait-69 datasets, in which healthy adults walked in several ways and also while doing a dual task. A dual task asks a person to walk while doing something mentally demanding, here reading, naming animals, or counting backwards, so that the effect of cognitive load on walking can be measured. The security theme used a different floor sensor, mats of piezoelectric sensors, which respond to the force of a step and give fine spatial and temporal detail over a stride. It produced the SFootBD footstep database from a large group of volunteers, and deep residual networks with separate spatial and temporal streams were tested on benchmarks that simulate security settings.

Results and what followed

The thesis reports high accuracy in telling normal walking from a cognitively demanding task in its best healthcare experiments, and a very low error rate for footstep verification in its best security case. In healthcare, this shows that cognitive load leaves a measurable trace in the way a person walks, which is groundwork for research on early detection of cognitive decline. In security, it shows that deep residual networks verify people from their footsteps with very few errors, and that more training data per person lowers the error further.

The thesis is organized around a series of manuscripts: journal papers in the IEEE Sensors Journal, IEEE Transactions on Industrial Electronics, and IEEE Transactions on Pattern Analysis and Machine Intelligence, a study of age-related differences in dual-task walking, and IEEE SENSORS conference papers from 2016 and 2017, one of them the rodent spectral-analysis study, which appears as an appendix. The degree was awarded in May 2018, and he began a postdoctoral fellowship at MIT Brain and Cognitive Sciences in June 2018. Related pages on the site include the footstep-recognition paper in IEEE TPAMI, the tomography-sensor paper in IEEE Transactions on Industrial Electronics, and the 2019 book chapter that consolidates this work.

Topics