Deep learning in gait analysis for security and healthcare

Cover of the Springer book Deep Learning: Algorithms and Applications, Studies in Computational Intelligence volume 865, edited by Witold Pedrycz and Shyi-Ming Chen. — Cover of Deep Learning: Algorithms and Applications (Springer, 2019), the volume that contains the chapter on deep-learning methods for gait analysis. Credit: Book cover published by Springer; image courtesy of Dr. Omar Costilla Reyes.

Diagram of two residual networks, a spatial stream fed one footstep image and a temporal stream fed several frames, whose class scores are fused. — Architecture figure for the footstep-recognition paper: each stream passes through convolution blocks with skip connections, a fully connected layer, and a linear support vector machine before score fusion. Shared when the paper was published in IEEE TPAMI, February 2018. Related photograph from Analysis of spatio-temporal representations for robust footstep recognition with deep residual neural networks. Credit: Image courtesy of Dr. Omar Costilla Reyes.

A presenter stands beside a printed poster titled “What can machine learning and a smart carpet infer about the way you walk?” at the ESOF 2016 poster session. — Dr. Costilla Reyes beside his poster on smart-carpet gait analysis at the EuroScience Open Forum 2016 in Manchester, July 2016. Related photograph from What Can Machine Learning Infer about the Way You Walk? Credit: Image courtesy of Dr. Omar Costilla Reyes; poster by Omar Costilla Reyes, Patricia Scully, and Krikor Ozanyan.

Research summary

This October 2019 Springer book chapter surveys deep-learning methods for sensor-based gait analysis in healthcare and in biometric security. It draws on Dr. Costilla Reyes’s doctoral research.

The chapter

“Deep learning in gait analysis for security and healthcare” was published by Springer in October 2019 in the volume Deep Learning: Algorithms and Applications (Studies in Computational Intelligence, volume 865, edited by Witold Pedrycz and Shyi-Ming Chen). It was written by Omar Costilla-Reyes, Rubén Vera-Rodríguez, Abdullah S. Alharthi, Syed U. Yunas, and Krikor B. Ozanyan. It was Dr. Costilla Reyes’s first book chapter. It consolidates work from his PhD at the University of Manchester and was published during his Picower postdoctoral fellowship at MIT.

Gait analysis for two purposes

Gait is the way a person walks, and it can be measured with cameras, wearable sensors, or sensors built into the floor. Deep-learning methods learn patterns directly from large amounts of raw sensor data. The chapter surveys such methods for representing and analyzing sensor-recorded walking patterns, covering gait measurement in healthcare research, where movement can provide behavioral information, and footstep recognition in biometric security.

Gait analysis for the two settings asks different questions of similar data. In healthcare the goal is to detect change within or between groups of people, for example under cognitive load or with age, while in security the goal is to tell one specific person from everyone else. Each application therefore needs its own data and its own evaluation criteria. The chapter sits alongside Dr. Costilla Reyes’s doctoral thesis, “Pattern recognition from raw spatio-temporal data for gait analysis in healthcare and security,” and the related Manchester studies on footstep recognition and gait classification described on this site.

Publication details

Authors: Omar Costilla-Reyes, Rubén Vera-Rodríguez, Abdullah S. Alharthi, Syed U. Yunas, Krikor B. Ozanyan
Venue: Deep Learning: Algorithms and Applications
Year: 2019
Pages: 299-334
Publisher: Springer International Publishing

BibTeX citation

@book{costilla2019deeplearningin,
  title = {Deep learning in gait analysis for security and healthcare},
  author = {Omar Costilla-Reyes and Rubén Vera-Rodríguez and Abdullah S. Alharthi and Syed U. Yunas and Krikor B. Ozanyan},
  year = {2019},
  
  doi = {10.1007/978-3-030-31760-7_10},
}