How valuable are exoskeletons?
You know, sometimes life throws some unexpected good surprises your way. One such surprise happened to me in the summer of 2019, when I was flying to Sydney, for RoboCup 2019 (in actuality to be a tourist in Australia while my Georgia Tech friends worked on RoboCup, but who’s counting). I had a connecting flight in LAX, and on my way to Los Angeles from Detroit, I happened to sit next to a guy named Drew Margolin, who is a professor of sociology at Cornell. Being the friendly guy I am, I struck up a conversation with him, and we discussed why we were flying, where we were from, as you do.
Well we also discussed our respective research, and I told him I worked with exoskeletons and prostheses, and happened to mention my study on the perception of metabolic rate, and how the average person couldn’t detect the benefits from modern exos. He found this interesting, and asked whether people actually found these devices valuable. I paused at this, since that’s honestly not something I had considered before. Are exoskeletons actually valuable? That’s not the sorta question that gets asked in a STEM field, since it tends to involve studying squishy humans and actually talking to them. We within the exoskeleton field have almost exclusively focused on the metabolic cost reduction metric to judge if augmentative exos are actually useful. But it was still an open question whether people actually cared about these benefits, and certainly my prior work on perception offered up an argument that they wouldn’t. The economist in me, long dormant, began thinking: if we could quantify how much value an exoskeleton gave you, that’s a pretty good proxy for if you’d actually want to use the device. Heck, in terms of mass adoption, even more fundamentally, the value determines if you’d want to even buy the device!
At this point I could see there was something to this question, and I asked Dr. Drew if he knew of a way to pull an experiment like this off. He indicated to a tool he’d used before in the field of economics that I might find useful: the Vickrey Auction. In short, he explained, the Vickrey Auction involves people bidding to buy a good, physical or otherwise, and, through game theory wizardry, would elicit a person’s true value for that object. He postulated that if we could somehow use the Vickrey Auction to quantify how valuable people find their walking times with and without exoskeleton assistance, the difference, in dollars, would be how much value the exoskeleton added. I deplaned with my mind furiously thinking about how we could pull something like this off. As a field, we exoskeleton researchers are no strangers to multidisciplinary studies, but we’d never brought in such a disparate field as economics. I frantically messaged my advisor that I was on to something, and spent the next week enjoying Australia.
Upon my return, at the hastily scheduled meeting with the boss, I explained the idea Dr. Drew had given me to actually quantify exoskeleton value, plus the details of the Vickrey Auction. If we could capture this value, it offered up the ability to use economic value as an alternative metric for the design and control of exoskeletons, a metric which would more closely get at what people actually find valuable, since by definition, that’s what we’d be measuring. Naturally, Elliott was a fan of the day, and dubbed it ’crazy-cool’, which I took as the green light to go ahead and launch the study.
Pictured: Los Angeles International Airport and 1996 Nobel Laureate William Vickrey, aka reasons I got my PhD
The protocol I developed involved a series of walking trials each with a Vickrey Auction. Subjects would walk uphill on our treadmill at a ten degree incline in bouts of two minutes, which would quickly tire them out. Then, every two minutes, they’d be asked to participate in a Vickrey Auction and bid honestly about how much they’d like to get paid to continue walking for the next to minutes. In essence, they were selling their walking time to us in exchange for actual cash. If they won the auction, they’d get the cash reward, and continue walking, which acted as incentive to keep going. If they lost, they’d instead rest until the next auction rolled around. Predictably, as they walked more, they got more fatigued, and demanded more money for their time. After they did this experiment without the exo, they then repeated it with exoskeleton assistance (once again, the Dephy ExoBoot). If the exoskeleton added value to their time, we’d expect to see less fatigue, evident in lower bids at the end. The difference between their bids with and without the exoskeleton was its value.
The whole process is illustrated with the flowchart on the right, in which a human participant bid against three other participants (represented by the robots) repeatedly. This eventually produced a result that looked like the graph outside the flowchart, which described the person’s price-to-walk with and without the exoskeleton. The area in between the curves was dubbed the Marginal Value (MV) of the exoskeleton.
There were essentially three different outcomes participants had after this study. Participants could have positive MVs, in which they bid less with the exoskeleton than without ( (b) in the figure); they could have negative MVs, in which they demanded more money with the exo (e.g. you’d literally have to pay them to use the device, (c) in the figure); and near-zero MVs, in which the exo didn’t cause a change in bids ( (d) in the figure ).
On the left are histograms of all the MVs for the three aspects of the exoskeleton tested: the total value of the exo (e.g. it’s physical mass and volume) plus its assistance, the MV of just the exoskeleton’s mass unpowered (obtained by having subjects to the experiment with the exo on but turned off, essentially just strapping a weight to their leg), and the MV of just the exoskeleton’s assistance. Predictably, the exo added mass caused a negative change (red) in people’s average value (though there was one guy who got a slight benefit; humans are weird). Additionally, the presence of the assistance itself was by-and-large beneficial (blue) in terms of value added to the wearer.
However, once you combined these to the net effect, the net MV was nearly zero for the average person. Importantly, it wasn’t that the average person didn’t receive value. It was instead that there were essentially two groups of people. The first group received large positive value from the exo, while the second group, you’d literally have to pay them to use the exo. These two groups canceled each other out in terms of value.
The study itself was easily the wildest I’d ever done from a pure science perspective. I had to delve deeper into behavioral economics to really hammer out the protocol (not that I minded) Melding together economics (already near and dear to my heart) with my research into wearable robotics was a great exercise in multidisciplinary science, and certainly I think upped the cool factor of the work. It was also highly informative, as now we know a few things:
You either get large value from the exo, or you don’t
Following from 1), the first group would likely drive mass adoption of the device into widespread usage, as by definition, those guys would actually fork over cash to buy the valuable device
We should really target/investigate these ‘responders‘ to understand why they gain value where others don’t
We can now use the Marginal Value metric to compare exoskeletons, in terms of not only physical design, but also their controllers! With bidding, it’d be feasible to repeatedly poll people how much value they’re getting and adjust the controller to optimize for this value in real time (really similar to human-in-the-loop optimization of metabolic cost done by Steve Collins and his crew over at Stanford.
Airplane flights are an underrated source of potentially awesome studies.