MachinePix Weekly #9: Arthur Petron, technical staff at OpenAI
Arthur Petron on building better prostheses and AI robot hands. This week's most popular post sparks a debate on shawarma vs. doner kebab š
This week I sit down with Arthur Petron, the mechanical engineer behind OpenAIās robotic hands. In a funny twist, Arthur was the creator behind a previous @MachinePix post on prostheses, but I didnāt realize until Richard Whitney (the interviewee behind the OK GO music video) reached out to tell me! š¤Æ
The most popular post last week was industrial shawarma production (or is it doner? itās been controversial). As always, the entire weekās breakdown is below the interview.
Iām always looking for interesting people to interview, have anyone in mind?
āKane
Interview with Arthur Petron
Weāve known each other for a while, but I didnāt realize until recently that you were the creator of the tissue compliance measurement device! Whatās the story behind it?
I needed to do something for my PhD, and I chatted with my advisor Hugh Herr who runs the Biomechatronics Lab at the Media Lab. I asked him āwhatās a hard problem you havenāt been able to fix yet?ā He said āsockets suck,ā so I decided to do that.
Ok, dumb question: what is a socket?
Sockets are the way you connect an amputated limb to a prosthesis. What makes it difficult is you only have soft tissue to connect to. Hard joints are all connected by bone, which is easy and aligned. Amputated limbs have squishy tissue at the joint. Itās kind of like trying to make a shoe thatās very comfortable but has a rigid connection to your foot. Climbing shoes and ski boots are pretty good, but theyāre super tight. Thatās sort of how it came about. The first thing I tried to do was make a socket simulator.
When you say simulator, is that software or hardware?
No, it was a hardware simulator. Initially I wanted to 3D print these bladders that would inflate around the amputated limb and simulate the socket. The mechanics didnāt work out because I couldn't get the pressure differential to work properly.
Thatās when I started on the Fit Socket concept. The idea here is I didnāt need to measure everywhere at once, I could measure one place at a time while rotating and translating around the limb. I rotate it because if there was any slight calibration difference between actuators, I ended up using the same one across the limb.
When you say āmeasure,ā what are you measuring?
Force displacement and time. Thereās a hysteresis curve because tissue is a hyper viscoelastic material. Hyperelastic is like a rubber band, where strain hardens the materialāit gets harder to pull as you pull it. Viscoelastic materials have decreasing spring constants over time. When you push your nail into your skin, and it leaves an impression that eventually goes away, thatās viscoelastic. Tissue has hyperelastic and viscoelastic properties, and thatās what we were measuring. We used something called an Ogden model.
How did people measure sockets for prostheses before? What did Fit Socket do better?
The Fit Socket is a very very high quality data collection device. We had two dynamic models for sockets: one for the skin, one for the tissue underneath. We used MRI data and tuned an Ogden model to match what the Fit Socket measured. Traditional measurements were just basic spatial measurements.
Why does this make the prostheses better?
Now that you have a 3D accurate model of the residual limb, you can solve for whatever you want. One thing we solved for is minimizing tissue strain from the prosthesis. If you think about your calf, itās soft and squishy. You can put a hard piece there. But if you put a hard piece on your shin bone, it will hurt. We could computationally design sockets for people and test them in simulation. We used a Connex 500 3D printer which could do multiple materials and hardnesses in the same print. Where the limb is soft we could make the socket hard, and vice versa.
What prostheses do right now, the sockets are usually carbon fiber for weight. Theyāll just leave a gap where the hard tissue is. But now you canāt load that area. Our models allowed us to more evenly load the prostheses and reduce strain for the patient. People told us it was like walking on pillows.
What happened after you finished it?
I built it as a tool with a full GUI application and live display of the actuator positions. They used it for their research for a while after I left Herrās lab. It ended up being used to help fit a Boston Marathon bombing victim, which was a very solemn moment.
Now youāre in charge of the robots behind the OpenAI dexterity research; whatās it like being a mechanical engineer at an artificial intelligence lab?
Interesting! Itās pretty fun, most people are programmers and Iāve always programmed a little bit. Everyone is an amazing resource for programming so I feel very supported in that regard.Ā
Thereās an interesting collaboration challenge: on the bleeding edge of any field there are a lot of technical terms, and sometimes those terms are overloaded between machine learning researchers and mechanical engineers. Even the term āoverloadedā in a mechanical sense versus a computer science sense yields very different interpretations. Actually, āyieldā means different things too. āSingularityā is understood differently between the robotics side and the machine learning side. āBiasā and āconflictā too.
Another thing is we all have a context and understanding in our heads, and we all do this, but often we assume knowledge in other people. I try not to be patronizing, so Iāll just explain things in my context. But the challenge of working at the bleeding edge is weāre creating context as we go, and so itās often a mistake to assume that the context is shared between individuals attempting to communicate.
The whole thing is just a big learning experience. Itās super fun. Iām one of two mechanical engineers there. I want to build the coolest robots I can, and I feel enabled. The best robots will have the best software.
Tell me about the hand used at OpenAI - what are some challenges working with it?
The interesting thing about the Shadow Robot Company hand we use is that itās actually human hand sized. The Wonik hand is much larger. The Shadow can interact with objects the same way we can.
Itās a research robotāitās not an industrial piece of hardware. Things break and we have to fix them.The mean time to failure is on the order of a week instead of years. Mean time to repair is pretty fast though. We worked a lot with Shadow to increase the former and decrease the latter.
Itās just a cool hand. The tendons are a special kind of Spectra. The body is aluminum, nylon, teflon, and steel. Itās a 24 degree of freedom robot! There are 20 motors in the base driving the tendons.
Does it have feedback?
There are hall effect position sensors on every joint. This does mean it gets confused manipulating magnets. Iāve tried it. It fails really fast.
In the OpenAI dexterity demo, what are the cameras and markers?
Itās an active motion tracking system by PhaseSpace. Optitrack and Vicon I used in grad school, but theyāre passive. They use IR reflective dots, but if they ever come together, you lose track of which oneās which. The active ones are individually encoded. We got half a millimeter precision with this system.
The fingertip markers let us back out the joint angles and test them against the joint sensors to see if oneās off in calibration.Ā
What donāt people appreciate about robots, or common misconceptions?
People think theyāre coming to take everyoneās jobs soon. But theyāre only as smart as the software that can run it. They can do a lot of things that are simple, but thereās an interesting misconception. Everything you get says Made in China, I think many people think thereās factories with robots doing this. There is a ton of by hand labor to make these things. Take the iPhone. Apple owns the most CNC machines in the world, and theyāre still doing so much assembly by hand. Itās really hard to get good small actuators. Thereās a long way to go. Programming and setting up robots is still pretty hard. This is what weāre trying to addressāhow do you have a general purpose robot?
Any cool side projects youāre exploring now?
Iām installing a finer nozzle on my 3D printer so I can print some drag chains to do some wire management on the lights in my workshop. I could just buy them but thatās less fun. I just bought a sewing machine too. Iām also starting to brew my own beer. Craft a Brew makes this great fermentation chamber that lets you drop the yeast without changing vessels.
Whatās your favorite simple (or not so simple) tool that you think is under-appreciated?
Buy a scalpel. They are the sharpest knives you can buy. If you do anything you need hobby knives for, theyāre better. Also, cigar boxes make great storage. My sewing kit is in one. Theyāre usually solid wood and have great joinery. A pleasure to use.
The Week in Review
Very cool and a little anxiety-inducing to watch the timing across all the grippers.
Maarten has a ton of wacky clocks in his portfolio.
A lot of food posts this week, and many followers lamented about when they had to tediously roll dough by hand in bakery jobs.
This weekās most popular post! This kicked off a debate on if the food pictured was shawarma or doner kebab. Itās still unresolved, please let me know if you know.
Diesel walker is some intense retrofuturism. Walking is terribly inefficient compared to rolling, which is why modern āwalkingā excavators still get around mostly on wheels.
Postscript
Working on interesting robotic problems or research? Arthur and I would learn about it.
If you enjoyed this newsletter, forward it to friends (or interesting enemies). I am always looking to connect with interesting people and learn about interesting machinesāreach out!
āKane