When beginning this project, we knew we wanted to build on the idea of how technology in a shoe could be useful and informative. Our first steps were conducting experiments in the realm of obstacle detection. We affixed two ultrasonic rangefinders to the front of one of our shoes using a 3D-printed mount and angled one outward and one downward. The idea behind this was that we wanted not only to be able to detect obstacles in the wearer’s path, but also any ledges or dropoffs. While the forward-pointing sensor worked well and was able to accurately detect the locations of upcoming obstacles, the sensor pointing at an angle did not reflect the soundwaves it was producing back to the receiver.
We realized that this was a fundamental problem with the way these sensors worked; a signal can only be returned if it can be bounced back off of a relatively perpendicular surface, which was not the case with the ground-angled sensor. We had also planned to make use of moving reference frames in order to detect obstacles while walking, but based on our plot data, it became clear that this was not necessary. When the sensors are pointed at anything but an obstacle, the sensors’ readings are out of range and can easily be filtered out. The plot below shows the real-time readings of our two sensors as the wearer is walking in the shoe. For this test, the lower sensor was angled nearly parallel to the ground, which allowed us to detect the ground but only detect ledges when the user was right over them. The blue circles show the distance to the detected object decreasing over time as the user approaches the obstacle, while the ‘X’s show the ground remaining relatively constant.
Shuffle Detection Using an Accelerometer
This figure shows the readings from an ultrasonic sensor approaching an obstacle over time, and another sensor aimed at the ground.
Using Accelerometer Data to Detect Shuffling
These first few tests told us that the problem of obstacle detection can be tackled in either a very simple or a very complex manner. To accurately detect ledges and dropoffs, we would need technology that can produce a more robust scan, like a lidar, which was unfortunately beyond the scope of the project at the time. After these realizations, we decided that our time and effort would be better spent elsewhere.
When we took a step back and looked at the issue of mobility in a larger context, we realized that a person’s walking style also contributes to their likelihood of falling. Older adults often shuffle their feet and walk with much less pronounced steps, which makes them more susceptible to tripping. We recognized that we could attach an accelerometer to a shoe and analyze the upward accelerations to determine how much a person was lifting their foot when they walked. After a few preliminary tests to make sure we could distinguish between two walking styles, we conducted motion capture analysis on three different styles of walking with varying degrees of shuffling. We visualized the change in angle over the period of one single step shown in the plot shown below, which helped us get a feel for what types of shuffling would lead to tripping.
Each line represents a different walking style with varying levels of shuffling that tracks the foot’s angles over time.
Next, we collected data from twelve people by affixing a phone to their shoe and having them walk normally and then shuffle. We then wrote an algorithm that compared the shoe’s accelerations in the z-direction to a shuffle threshold determined by that individual’s walking patterns. To do this, we implemented a method of peak detection that found the mean of the z-accelerometer peaks. This value was then compared to the person’s shuffle threshold. The plots below show the accelerations in the z-direction of several people’s data and compare their normal walk to their shuffled walk. The title of each plot shows the name of the subject and their style of walk, followed by our algorithm’s prediction of their walking style.
Shuffle Detection Using an Accelerometer
Each subplot represents a different trial from each test subject and shows their vertical accelerations compared to the shuffle threshold. The final word in the title is the algorithm’s prediction.
We also attempted to perform a frequency analysis on this data by taking the Fast Fourier Transform (FFT) to put it into the frequency domain. This plot is shown below. Not much useful information can be gleaned from this plot, though, because the trends we are focused on detecting have more to do with the acceleration’s amplitude than the frequencies of these amplitudes. It is still easy to see that the amplitudes are generally lower when shuffling versus walking normally, but the frequencies are generally the same from trial to trial.
Accelerometer Data Frequency Analysis
Each subplot represents a different trial from each test subject and shows the frequencies of the amplitudes of a person’s walk. This shows lower frequencies when shuffling but is not as useful as the previous plot in the time domain.
We then created an accuracy plot that showed the percentages of the number of successful predictions for a specific threshold. This was done by sweeping through our threshold parameter and comparing the logical outputs from the algorithm’s predictions to the actual intended walking style. We found that the greatest accuracy, 91%, was achieved between threshold ranges of 0.6 and 0.8 G. This information is shown in the plot below.
The highest accuracy achieved was 91% between the threshold ranges of 0.6 and 0.8 G
The SmartStep Mobile App
Finally, we turned this algorithm into a functional app that can run while a person is walking. It makes use of the same processes as our previous algorithm but is able to collect and process the data in real time and allow for user calibration. The app alerts the user by vibrating the phone when a person’s shuffle is detected.
During the calibration phase, the user walks normally and shuffles.
After calibrating, the phone can monitor the user's walking style. If it measures adequate vertical acceleration, indicative of high steps, the background is green.
If the user begins to shuffle, thus decreasing the vertical acceleration to below the calibrated threshold, the background turns red and the phone vibrates until the user begins walking normally again.