Deep neural networks for learning spatio-temporal features from tomography sensors

This 2017 paper classifies gait from the raw signals of a tomography floor sensor with a convolutional neural network. The network learns features directly from the data, without first reconstructing a footprint image.

Published in IEEE Transactions on Industrial Electronics in 2017 by Omar Costilla-Reyes, Patricia Scully, and Krikor B. Ozanyan, the paper demonstrates accurate classification of spatio-temporal gait data from raw tomography sensor data without reconstructing images. Tomography recovers a picture of a quantity over an area from measurements taken along many paths. The University of Manchester’s iMagiMat floor sensor measures how plastic optical fibers deform under footsteps and ordinarily feeds those measurements to a tomographic solver to reconstruct a footprint image.

The paper instead arranges the raw sensor readings into spatio-temporal raw sensor matrices and trains a convolutional neural network to learn features from them end to end. It uses the UoM-Gait-13 dataset, which covers a set of walking styles and cognitive tasks that together form thirteen gait pattern types.

The authors report a very high F-score on the tomography-sensor dataset, with little variation between repeated runs; the F-score combines how many of the identified cases are correct and how many true cases are found. They compare the raw-matrix approach with features derived from tomographic reconstruction, visualize the learned filter maps, and compare training times of the convolutional models against shallower methods. Skipping reconstruction removes a computationally expensive step while preserving the information needed for classification, which is the paper’s central methodological point. Because the network is compact and learns directly from the measurements, the same idea carries naturally to other industrial tomography tasks, where a trained model can replace a costly reconstruction stage and deliver decisions closer to real time.

The paper is the second manuscript of Dr. Costilla Reyes’s PhD thesis and was published open access. In January 2018 IEEE recognized it as the top-downloaded paper in IEEE Transactions on Industrial Electronics, and it was the journal’s most popular article in February 2018. He presented a related talk, “Towards making neural networks interpretable: feature visualisation from deep neural networks,” at the Data Visualisation Summit in Manchester in December 2017.