MachinePix Weekly #10: Jeri Ellsworth, Valve hardware team founder
MachinePix Weekly #10: Jeri Ellsworth, Valve hardware team founder
Jeri Ellsworth on founding Valve's hardware team and VR/AR gaming. A fire-fighting aircraft is this week's top post.
This week I sit down with Jeri Ellsworth, who founded Valve’s hardware team and now runs Tilt5, an augmented-reality gaming company. She shares stories about the challenges of shipping gaming hardware and hard lessons learned 👾
The most popular post last week was a fire-fighting tanker aircraft. As always, the entire week’s breakdown is below the interview.
I’m always looking for interesting people to interview, have anyone in mind?
—Kane
Interview with Jeri Ellsworth
What got you excited about VR and AR hardware?
It took me a long time to get to VR and AR. It goes back to ham radio. I was interested in electronics and computers as a kid, in this dinky town where there were no resources except for the library. I met some old farts in the ham radio scene that helped me out. They gave me tools, some really antique voltmeters and stuff, which helped me get started.
There were a few VR attempts in the 90s, I remember the i-O i-Glasses and playing Descent on them. They made us all puke, but we were all huddled around it thinking this was the future. It was a terrible terrible experience. I couldn't even sell them in good conscience. I forgot about VR for a while and worked in electronics. I didn’t think about it again until I joined Valve.
Valve was one of the early companies to experiment with VR content, and you were brought on to build out their hardware teams. How did that come about?
Valve tracked me down. I had started a YouTube channel to give back to all the mentors to help me along the way: the ham radio operators and the dozens of engineers in the valley that took me under their wing. My channel was all about hard tech you could do in your garage: I was making chip foundries, circuitry design, etc.
Someone at Valve found my channel. It was a funny story, I hadn’t played PC games for a long time, so I didn’t recognize them initially. I started getting messages on social media from this place called Valve Software. I get a lot of random messages so I didn’t take it seriously. They showed up at Maker Faire to find me. Finally one day Gabe Newell emails me and says he wants to meet me in Portland where I was living and talk about Valve, and it clicked and I said fine. Over lunch he explained some visions they had for hardware and bringing the family together around gaming.
They invited me to come up to Seattle to visit and told me it wasn’t an interview. It was totally an interview. There were like eight people in a room asking me questions. I was pretty dubious at first. They started selling me on a grand vision of what they wanted and it was really hard to say no. At the time, Windows was threatening Steam with their new store, and we were just on the heels of the Wii. Michael Abrash and Gabe walked me around valve HQ, showed me this big space where they told me I could do whatever I needed, hire whoever I needed.
I had just come out for the day, and they pressed me to stay. I didn’t have any clothes or toiletries, so they gave me clothes from the swag store. By the end of it, I was sold on what to do. I was in the middle of another project, so I wasn’t sure when I could start. They offered to just pay off the other employer, it was super flattering—but I had given my word. So I finished my project and headed to Valve.
What was the most fun project? The most challenging?
The story would be incomplete without talking about the team. They said there was no budget limit to put this together, and we moved a lot of really amazing folks to Valve as a result. The first year was just setting up the team and doing pure research which was a ton of fun.
We did a ton of experiments on controllers, since our audience was primarily on keyboard and mouse. We played with active feedback controls, galvanic skin sensors to measure stress as an input for the game. We could see where musical cues in Left 4 Dead caused stress, even if there weren’t zombies.
I was a really big fan of wand style controllers, which are now the de facto format for AR and VR now. I built a wand with haptics under every individual finger so you could simulate movement like a sword slipping through your hands. We made a glove with strain gauges that could resist your fingers to simulate you holding different shapes. The hardware lab was always a destination for VIP tours. Will Wheaton came through one time. John Carmack came through. He didn’t think wands would be a thing for AR/VR and we had a big discussion about that. He was strongly convinced that people would play VR seated with an Xbox style controller. From our user testing I was convinced gamers would want direct interactions in 3D space with wands and moving around the room was the best way to overcome motion sickness.
You’d start to converge on stuff that worked really. For the controller, there was a system with two haptic touch pads, and another with a trackball. When we play-tested the trackball, it was a great replacement for a mouse. We tested with Counter Strike GO, which wasn’t released at the time. We would run people through it with an Xbox controller, a trackball, and the touch pads. We’d time and check their accuracy on the training course. The trackball was close to a keyboard, and the touch pads weren’t as good.
Gabe had a vision of an all inclusive family gaming experience, but the DNA of the company was still hardcore gaming which was always in tension. This is my opinion, but we ended up going for the trackpads out of an aesthetic preference. If you presented anything “cute” to the internal play testers it would often fall flat.
What’s hard about building gaming hardware?
The Steam Box was a big challenge. There was a constant conflict about how we should do it. We tried to make it work with the internal DNA of hardcore gamers, but also appeal to the grand vision of family gaming.
The BOM (bill of materials) to appeal to the hardcore gamers would have been really high. But it would have been really expensive to subsidize that. The entire industry, with the exception of Nintendo, sells consoles at cost or at a loss, and we’ve been stuck in that model.
Valve considered having OEM partners manufacture the Steam Box, but I was strongly against that. 3DO tried this in the 90s and flopped. You can’t control your messaging and your product can get really expensive and confusing. We ended up with three tiers for the Steam Box, and it became a messaging challenge to describe the tiers. It really hampered the success.
What’s a misconception about designing for AR or VR?
You can’t just port games over and expect them to work. Dan Newell, Gabe's brother, had some VR headsets. They were all these military style systems. We did a ton of user research. A ton. We took some of the best military headsets and tracking we could get, and invited developers to come play Left 4 Dead, and everyone was getting sick. The feeling of playing an FPS in VR is so horrible. We’d ask people to play until they felt queasy and had to stop. Then ask them to return and try again. We wanted to see if it was something that people could acclimate to. After the second or third time they would refuse. A bunch of us concluded that traditional games and game movement couldn’t be directly ported to VR, so we started experimenting with room scale experiences. To this day, that's the dominant format of VR games.
I thought AR fitted our mandate much better at Valve - from Day 1 of our hardware work, Gabe wanted build a gaming experience for the family, instead of just the hardcore gamers. Nintendo just gets it. The headsets we were using were these huge spaceman helmets, but some of us could see the potential. We had an AR testing rig with this really dumb game we made for leading a character through a maze on a table. People would come with their friends to play the AR game. There were also some zombies, we literally copied some assets from Left 4 Dead. The VR experience was very much a novelty, people would try it individually for a few minutes. With the AR game, we’d have to force people and their friends to leave.
It sounds like running hardware at Valve was really challenging.
It’s kind of a conflicted place. They pride themselves on a flat hierarchy with no managers. You have a mandate of just “do the right thing for the company”. It allows you to try a lot of things but sometimes causes conflict. There’s a strong push for cultural fit, and I think a lot of people weren’t sure how the new hardware team would affect the culture. This was challenging for recruiting sometimes. When I left Valve, they wanted me to sign papers to not talk about the culture fit thing, but I left a lot of money on the table and didn’t sign—it was important for me to have my story. I’ve always been very open about it.
Valve’s really good at what they do, and their hardware and family gaming vision was extremely ambitious. In the end, I think the strong company culture, the challenge of hardware, and the new vision of family gaming was really hard to reconcile all together. It was frustrating at times, but through our research I think I got a look ahead at the next 20 years of gaming. Gabe still comes and visits us at Tilt5 still!
Any side projects you’re working on right now?
Oh god I have all kinds of things going on. I like retro stuff. I’m playing with vacuum tube radios. I’m building a Fallout 4 style, ‘50s-’60s style radio. The last few years I've been creating my own holographic optical combiners in my garage by exposing photo plates with structured laser light.
What’s your favorite simple (or not so simple) tool that you think is under-appreciated?
I’ve mostly ditched my imperial measurement tools and gone all metric. A metric tape measure and dial calipers set to metric.
The Week in Review
A follower mentioned that this looks like a snow gun (specifically, a fan snow gun) used by ski resorts. The similarity is pretty uncanny, with the exception of nucleators in the snow guns not necessary in a firefighting application.
We are in a golden age of hobbying where you can buy equipment like this on Amazon. The mechanism is as simple at is clever: beads rotating will eventually align its hole to the axis of rotation since that orientation has the highest moment of inertia (ie. the most mass the greatest distance from the axis of rotation).
I love fiber optics and I love trains.
This week’s most popular post! Incredible bravery from tanker pilots flying in low altitude, low visibility situations like this. The red fire retardant is known as “slurry,” although lake water and even pool water works in a pinch.
Despite the amazing footage of the DC-10 from the previous post, I felt compelled to warn followers to not get close to tanker drops (and so did Cal Fire, apparently). The slurry carries a lot of energy and is very dangerous to anyone where the drop.
Postscript
This week I also guest-edited for one of my favorite newsletters, The Prepared—I cannot recommend it enough if you like manufacturing, systems thinking, or both.
If you enjoyed this newsletter, forward it to friends (or interesting enemies). I am always looking to connect with interesting people and learn about interesting machines—reach out!
—Kane