Microsoft’s Bryce Johnson and John Helmes Join Make: Editor Caleb Kraft
Bryce Johnson works at the Inclusive Tech Lab at Microsoft who is one of the co-inventors of the Xbox Adaptive Controller and the Surface Adaptive Kit. John Helmes is an industrial designer at Microsoft who began creating and customizing devices for his daughter, Jara, who has cerebral palsy to help her interact with a PC. His work has led to Microsoft Adaptive Accessories, which allow users to adapt devices using 3D printed parts. Make: editor Caleb Kraft has his own side project, The Controller Project, which he and volunteers around the world 3D print accessories for free for people to be able to game. Bryce, John and Caleb have a fascinating conversation about how these capabilities are changing lives.
Podcast
Audio/Video
Transcript
Customizing Adaptive Devices for People with Disabilities
Bryce: And so for us in Xbox, we had to basically come to grips with the fact that all interactions on the console were through a controller. And if you couldn’t use the controller, you couldn’t play the console. I remember one conversation in particular with Erin. She would basically just tell me, yeah, Xbox isn’t my thing. I play on PC. And what she was really saying was that she couldn’t use the controller, right? Like you have to play on controllers. Controllers weren’t built for her. She didn’t have the strength. That was really powerful. We knew we had to do something.
Dale: Welcome to Make Cast, a podcast about how we make the world better. I’m Dale Dougherty. Make Cast is brought to you by the members of Make Community who support makers in their community and around the world. To learn more about membership, visit make.co.
In this episode, Make editor Caleb Kraft talks with Bryce Johnson and John Helmes, both of Microsoft, who are involved in developing assistive hardware devices that can be augmented and customized by makers, and most importantly by the community of users they serve.
Caleb: All right. This is awesome. Thank you so much for joining me today, guys. I know we’ve talked before in the past, but I’m really excited to share you with our community here at Make Magazine. Today I have Bryce Johnson and John Helmes from Microsoft who are both working in areas of accessibility and inclusivity. I’m gonna first let them introduce themselves. Bryce, if you wanna go first..
Bryce: Yeah, sure. I’m Bryce Johnson. I work on accessibility for Microsoft devices. I’m one of the co-inventors of the Xbox Adaptive Controller, the Surface Adaptive Kit. And thanks to John here, you know, I got to work on the Microsoft Adaptive Accessories which is our latest and greatest to be honest.
So I’m gonna pass it off to John.
John: Thank you Bryce. Thank you Caleb for for having me. My name is John Helmes. I’m a senior designer at Microsoft, and I’ve been at the company for about 10 years, I think. Started in research, currently in Azure. And it’s been awesome to be working with Bryce and yes, and particularly I’ve been working with him for quite a while now on the adaptive accessories and also we are an amazing team.
That work actually started with with my daughter and the hackathon two years ago. My daughter has Cerebral Palsy and I figured there should be other ways to interact with the PC, make it more adaptive. And that’s actually how the ball started rolling.
And it started rolling really small, but by the end of it, there were so many people involved, a lot of excitement. That’s my involvement actually with Bryce and the Inclusive Lab as well, inclusive Tech Lab.
Caleb: Wonderful, and I’m gonna ask you more about that in a minute, but just to fill people in who may not know what all we’re talking about.
Microsoft has a series of devices over the past few years that they’ve put out, like Bryce mentioned, the accessible controller for gaming, which is a kind of low production piece of hardware. They’re not producing billions of these– that are specifically for people with physical disabilities and limb difference, to be able to plug in different buttons to be able to game.
That’s awesome. John has worked on a system that is for general computing use that is a little modular system that allows for using the mouse and keyboard. And a little joystick and stuff that has all these 3D printed accessories. There’s tons of really cool stuff coming out of Microsoft in this area, and it seems to all be partially because it’s so low production, but also it seems like you guys have embraced a maker ethos in this a little bit.
Let’s talk about this, the devices John is working with. John, can you tell me a little bit more? First, what the devices are now. What, what exists now that’s out there to the public. Can you explain it to me a little?
John: Yeah, of course.
It’s called the Microsoft Adaptive Accessories and there’s quite, quite a few.
It’s a family of products. So we have the adaptive mouse, which is a small mouse core that you can customize to your likings with 3D printing, but you can also use the core to stand alone if you want it to. For example, there’s the travel mouse. Then in addition to that, we have the Microsoft adaptive tail and thumb support to go with that mouse core.
So when you attach that tail or thumb support to that mouse core, you end up with a more traditional mouse. Also to mention that the thumb support is symmetrically designed, so you can flip it both ways around, for left hand orientation or right hand orientation, basically. We have the Microsoft Adaptive Hub, which is the brain, as you will, to connect our three new adaptive buttons, marks of adaptive buttons to the PC. And then on the button side we have a dpa, we have a jaw button and we have a joystick. And they all have the same 8 digital switches in them in the base. So it’s a little bit hard to explain this.
I’m thinking about it now without using any visuals. So to describe them but just to, to emphasize that it’s not an analog joystick. So it’s a base. It’s about five by five centimeters with 8 digital switches inside. And then yes, as mentioned, we have different taste, different flavors of toppers as we call them, that go on top.
So d-pad joystick, jewel button. But the core idea really is that those flavors, those toppers as you call it, can be replaced with anything that’s 3D printed and that matches your requirements. So that goes for the mouse core and for these new buttons. And then the hub, that brain actually takes existing 3.5 jack interface switches for example as well.
So it supports both obviously our new buttons but also existing assistive tech that’s out there on 3.5 jack. Bryce, is that fair?
Bryce: It is fair. But I think there’s a little bit more that we can add, like I think in the context of disability and stuff. The mouse is a mouse, and it’s small for a reason.
John: It’s for people with degenerative muscle conditions like muscular dystrophy or spinal muscular atrophy. It’s a 45 gram mouse. So when John called it a travel mouse, it is. It’s a great travel mouse, but it was intently designed to enable a part of the population that’s quite often forgotten, which is people who have a hard time just having the strength to push a mouse around.
So while the mouse can be extended through the tails in innumerable ways, it is still a very accessible mouse to a certain group, to a certain population, by itself. One of my favorite features of the mouse, which is hard to talk about but I’m gonna nerd out and talk about it a bit here, is that we actually have a setting where you can adjust which way the mouse is up.
So if you think about a regular mouse, if you pushed it up, like up is where the buttons are, where your fingers go, right? That’s up. But I can actually, we can actually adjust it. So where your fingers are is actually down. So in a way so if you imagine that someone has a hand that might be differently formed, right? They might push up and pushing up to them feels like pushing right? So what we’re allowing them to do is we’re allowing them to say, no, this is what up is. So you don’t have to change your brain to to meet what your hand is doing. You can actually adjust the mouse. And that’s really powerful I think in that sense too.
And when John talks about the brain and the hub, the hub exports any mouse or keyboard command, anything that you can do like with media keys, basically that throughout the HID specification, we can nerd out a bit here about like ID. So we can basically pump out anything from ID in either a simple or complex macro.
Caleb: That’s awesome. I do wanna go back real quick and explain a little bit for people who are not in this space, who don’t really necessarily know what we’re talking about when you were talking about reorient, orienting up.
Bryce: Yeah.
Caleb: So this is extremely important. It sounds like such a simple thing, but it’s extremely important because if somebody has muscular dystrophy, spinal atrophy, or they’re quadriplegic or some kind of injury where they have muscle loss, the tendons in their extremities will tighten up over time and the orientation of their hand itself changes.
So their hand will be forced to sit on the mouse sideways or backwards. And people listening to this may not have been aware and they’re thinking why is that so important? It actually turns out that there’s a big chunk of people that this is a problem for. And so it’s great to hear that there’s a solution for that.
Bryce: Yeah. I’m glad you appreciate it.
Caleb: It’s fantastic. It really is. So one thing that people don’t realize also with a lot of these devices, so back to the maker aspect of things. The devices that John put out have all these 3D printed accessories, which of course is extremely powerful because then you can of make your own, design your own, 3D print them at home, customize. A lot of people don’t even realize like what Microsoft did with the gaming controller. That’s kinda just like the seed and then everything around that you have to make on your own, build on your own to be custom. So this is all extremely relevant to makers. . And what people, what I think Microsoft doesn’t get enough credit for is the day before the Microsoft accessible controller came out, the cheapest you could buy a device like this called a switch box was about $1,100.
And then the Microsoft Accessible Controller comes out and it is effectively what’s called a switch box. And it’s what, $110? $120?
Bryce: Maybe it’s a hundred US. It’s a hundred dollars.
Caleb: Yeah. So the price that people had to pay, and none of this is covered by insurance by the way. The price that people had to pay to be able to gain was reduced by tenfold thanks to this program at Microsoft. That is incredible.
So let me ask what does the near future look like for– let’s go back to John’s project with the 3D printed accessories. I know that you guys have put out a library of parts that people can order off of Shapeways. What does the future look like for that? Are you going to develop more parts or have some way for people to develop their own? Or is there a plan, a roadmap in place for further development?
Bryce: I don’t wanna promise a roadmap because I think we wanna make sure that we’re working on what we can build. But we definitely completely recognize that community engagement with this work is gonna be really important.
I think we just have to figure out what that is yet. Everything that’s up there today, John has made, and we make the STL files available, like from Shapeways. You can download them, you can, build off them yourself. I think from my perspective and from what we learned from the adaptive controller is figuring out how we cultivate this community.
John’s done some incredible work and given the community some amazing things to build off of, but how do we actually continue to cultivate makers to go out and build more. And how do we not only bring the makers into it, but how do we bring people with disabilities into — to the makers?
Caleb, you and I met because of the controller project. People reached out to you, you built them controllers, you had that community connection with the disability community. I think where I fit in is to try to build those connections with the maker community.
Caleb: Wonderful.
That’s gonna be super powerful moving forward. I’m really curious.
So we talked about the products a bit, and now I’m a bit curious about you two as individuals. John, I’m a little bit familiar with your story, so we’ll do it second. Bryce, let’s start with you. How did you even, how did you even end up doing accessibility hardware?
John: I do want John to tell his story and so I’m gonna be quick. The reason why we did accessibility hardware was quite honestly in Xbox, we were building assistive technologies onto the console for the first time in 2015. We were building things like screen readers and magnifiers and captions and things you’d expect from a modern operating system, assistive technologies you’d expect from a modern operating system.
But through the lens of our inclusive design methodology, the first principle is recognizing exclusion.
Bryce: And so for us in Xbox, we had to basically come to grips with the fact that all interactions on the console were through a controller. And if you couldn’t use the controller, you couldn’t play the console.
John: And so when I would go out into the disability community,
Bryce: I remember one conversation in particular with Erin
John: but Erin was with us for the adaptive controller. She has decreased strength. She doesn’t have a lot of strength and
Bryce: she would basically just tell me, yeah, Xbox isn’t my thing. I play on PC. And what she was really saying was at the time anyway, that she couldn’t use the controller, right? Like you have to play on controllers. Controllers weren’t built for her. She didn’t have the strength. That was really powerful. We knew we had to do something and so that’s what sort of started our journey. We started at the hackathon. We kept building from there, and it was really important for us to do that, and that spark led to a bunch of different things. It led to us building a hardware accessibility practice here at Microsoft and by the time John started his idea we had some experience in this area and we were excited about the work that John was doing.
Caleb: Fantastic. That’s a great story. So yeah, John, let’s hear let’s hear your story. How did you get into accessibility hardware?
John: Yeah, that’s a good question. I would love to hear more from Bryce but given that he passed on the baton.
It’s amazing to work and learn from Bryce. I’m relatively new to the accessibility community and rolled basically into it, as I mentioned because of my daughter Jara, right? But I think I have to rewind a little bit. As a designer and I’m trained as an industrial designer, I can remember as a kid I was always taking things apart and building new things from– remember one day, building a little toy car from an old radio player and just reutilizing the like little motors. And so I’ve always been interesting in figuring things out and making things work and hacking away at things.
And which probably also got me into research and design. And I think this is a little bit what I’ve been doing as I’ve been growing up, right? And all the things that she in her environment bumps into, atoms quite literally. And looking at, Hey, how can we, how can we enable her to excel, right?
And how can we give her the tools that she needs to excel? And I think this has started early on with looking at her orthotics. She had an ankle, foot or orthotic that is really good. And she provided a lot of support. But I looked at it and I was like, I think there should be other ways, other opportunities, right, to approach this. And I actually started looking at creating a solution that’s a little bit lighter weight, less expensive, and actually 3D scanned her lower leg and created these 3D printed orthotics for her. They were kind, yeah, as mentioned, they were lighter weight, easier to fix if something would break.
And so that’s how I’ve been approaching things as she’s growing up. And then I guess at some point the computer came round and it was like, okay, we’re gonna try for Yara to use a computer. And I remember her OT saying conventional mouse is not gonna work. We tried that. She’s continued pressing the buttons, like her finger slide off the mouse surface.
Initially you’re like, I’m not the expert. The OT knows everything what she’s doing. So we moved to a track ball, BIGtrack, and that actually went quite well. But I also thought like it’s a missed opportunity. It feels like there’s something in this space that could be interesting if we can just modify more to her body. Could we make something so that ankle foot orthotic just better fits her needs? And that’s actually when my kind of industrial brain and this kind of curious researching brain kicked in and was like, let’s hack something. Let’s try something.
Two and a half years ago during the hackathon, I thought, Hey, what if we were to apply that same strategy with 3D printing that I did for the orthotics for her ankle foot orthotic to some sort of orthotic between the mouse and her hand, right? Providing that support that she needed to maintain her fingers on the mouse buttons, but also create a little bit of dampening so that she wouldn’t continuously trigger the buttons.
And I guess that’s how the ball got rolling. And it was actually full cost from our Microsoft Center Harran in Shenzhen who picked up the hack. So I shared a simple hack. He was super enthusiastic from day one. And I guess the rest is history. It’s like where Bryce came around very soon after that. And he’s been such an inspiration and learning from what they’ve done with the Xbox adaptive controller and then broadening the ecosystem from the mouse to the button input. And I think there’s been so many people coming together across the company, which has just been a blast, basically. To be able to work on this and work with this amazing team to design these accessories. But that’s in a long nutshell.
Bryce: There’s one thing I want to point out there about the approach that John did. The reason why we all got excited really quickly was that he figured out a way to augment a mouse that we already had, really simply.
And the problems that we had before this project, and which is funny because Caleb, I think, now that I think about it, you’re in the same boat that John was in, but we weren’t in this right head space. So we had a problem with 3D printing because we kept thinking about 3D printing complete forms. Right?
Like a complete form as opposed to an augmentation on an existing form.
Caleb: Yeah.
Bryce: And and so like when John came out and had and did his work there and brought that to the table, it was really interesting because he did it in such a simple way. We were all, we all basically just felt silly.
We were all like, oh look, it was right there all along. Like we just,
John: simple idea.
Bryce: Yeah.
Caleb: That is the same thing that happened to me whenever I …. Go on.
Bryce: And it really has affected, I think the approach. Like when you, when we think about software accessibility, I often say that all software accessibility is personalization.
That takes into account human diversity. But when it comes to hardware, it’s about creating these systems. And the way that I like to talk about how we structure these systems is devices, accessories, and augmentations. And how do you mix and match those things together to create a setup, like a rig that works just for you.
And how do you think about how those augmentations work together, right? Yeah. And the fact that like during the hackathon, and we’ll have some of this stuff too soon, so I’ll spoil it. But you’ve seen the forms that are out there, but we had we had mice and flip flops, we had mice and shoes, we had all types of things.
It was really important to understand where that’s going. So we know that there’s a lot in front of us that we can do. And, but at the same time, like we also really recognize that this has to come from the community. And so we know this, but it’s like we also all have day jobs that are building products.
So we have to figure out how we get this other stuff out there in the world. How do we find the time?.
Caleb: Yeah. Yeah, for sure. So just for reference, people listening, I have a side project called the Controller Project, where me and volunteers around the world 3D print accessories for people to be able to game, for free.
And I’ll plug that later, but that’s what Bryce is mentioning. And I had the same epiphany I was doing it a very complicated way before, and then one day I was like, wait a minute, I’ve got these 3D printers sitting here. Instead of trying to come up with a whole solution, why not just tackle each little difficulty in a way that works as a system?
But yeah. No, it’s beautiful. Beautiful. One thing I really wanna point out, and I believe that Microsoft deserves a ton of credit here. like much more credit than they’re getting, is that you were just talking about the strength of interacting with the community and involving the community and this stuff.
And not only is Microsoft putting out devices to help and other companies do that too. In the devices alone, you’re far ahead of everybody else. You’re like years ahead of everybody else, but you’re taking an approach that includes community involvement, interacting and feedback.
For example, Bryce in the background there, you have the actual Inclusive Tech Lab where that happens, right? Yeah. So tell us a little bit shortly about the tech lab and then we’ll probably start working towards wrapping up.
What is the Inclusive Tech Lab?
Bryce: Yeah, the tech lab really is a place I will say that when we started the Inclusive Tech Lab in 2017, it wasn’t that different than what most people would think of in the industry as an empathy lab.
We built the place basically because people in Xbox could not understand that someone couldn’t use a controller as it was designed. It just was so foreign to them and I get it, right. It’s like telling someone water’s not wet, right? It really is. You have to show them how people move and how people do that. And so at first we were very much a place where the developers would come and learn. We’ve evolved that practice because we have a higher goal for the lab now in that this is the place where the disability community can find a home on the Microsoft campus.
We’re a big place, right? This place is their place. And the developers and engineers and designers that come here are visiting the House of Disability. They’re the visitors, right? So that’s the aspiration is that this is the most accessible place. We want to engage the disability community here.
We want them to come in. We wanna show folks like what’s possible. I think one of the interesting things that John mentioned about his interactions with his OT. His OT said she can’t use a mouse. And then John was like, I’ll show you. There’s certainly a little bit of that.
There’s a balance there, right? We’re not trying to force things on the community, but we are using our expertise in problem solving and engaging directly with the community to come up with new things. I work with a lot of therapists. They keep wanting to like frame things in terms of ideas that they already have and that’s completely understandable because they’re pragmatic people who are trying to improve people’s lives. They’re not inventors. And so it’s our job to be the inventors, right?
John: And that’s the beauty of it, right? Like where I initially interacted early on with OT and it was like, oh, okay, this is not gonna work exactly what you say, Bryce. And then it’s oh, we oh wow, there, there’s an opportunity here.
And then utilizing both expertise, right? Because without OT and the continuous commitment of the OT throughout these two and a half years, the product wouldn’t be what it is today. So it’s it’s finding the right ingredients and have that kickstarter of that idea and then utilizing all those expertise to build out as accessible a product as we could possibly build.
Caleb: I was just gonna say for the people listening, OT is occupational therapist. And that’s usually somebody who’s helping you figure out what works for you, what doesn’t work for you, what you can do to get by. Now I did wanna point out that this community involvement is so powerful for peer learning.
Because you’ll have people, if you’re living with, for example, a missing arm, you come up with your own tips and tricks of things that work for you. And if you can come into someplace like the Inclusive Tech Lab and share those tips and tricks, they propagate and you’re helping other people.
This community involvement is so powerful. Not just for the optics of it. It’s actually an extremely powerful tool..
Bryce: Yeah and we have to make sure that we’re engaging with not only therapists and engineers and designers. That’s important, but the primary people we engage with is the disability community.
Like we don’t do anything without them from the very beginning. We don’t have an idea until they validate that what the challenge is, right? We’re not making up things or inferring things, ever. And we have to be vigilant about that. Like it’s, we have a lot of experience. Like we’ve done this a long time. We have if someone gave me the opportunity to go run and build things for the rest of my life, I probably could. But I also know. I have that backlog of ideas, but I also know that a lot of them would be wrong and that people’s situations change and it’s always crucial to engage with the disability community because like, it doesn’t matter how much your assumptions, you’re not living that experience.
Like your knowledge is only as good as the person you’re talking to.
Caleb: I’d like to wrap this up. I’m thinking that people listening might want to learn more about what you do. Let’s start with you, Bryce. Is there like a website or a specific place people could go to learn more about the tech lab and the accessible gaming controller?
Bryce: So I think let me send you the URLs, but I think off the top of my head, if you go to aka.ms/inclusivetechlab, that’s the Inclusive Tech Labs website. And if you go to aka.ms/XboxAdaptiveController, that’s the adaptive controller website. And if you go to aka.ms/microsoftAdaptiveaccess, That’s probably the adaptive accessories website.
Caleb: Okay, fantastic. Now for people listening that want to get involved in this kind of stuff, you can of course go check out all of those. There’s my project, which is called The Controller Project. You can look up the controllerproject.com and volunteer to 3D print things.
There’s a great group based out of Canada called Makers Making Change that does 3D printed accessories, sometimes mixed with electronics. They don’t focus specifically on gaming, but they have a huge library and a lot of people that need help. You can volunteer there and I think that’s about it for volunteer organizations that I can think of.
There are also charities like Able Gamers, Special Effect s and those are the only ones I can think of off the top of my head that people could Google and check out. We’ll try to have all these links down in the description, the associated article if possible. Anything else you guys wanna add before we wrap?
Bryce: I would like to ask like the people listening to this, like how would they want to engage with us? Because to be honest, I think part of the challenge is we don’t necessarily know the right place to be. I’m pretty confident that no one wants me to set up like a public team site and do stuff there when I could do a Discord or something like that.
We wanna go where the people are, right? We’re not trying to bring people to where we’re at, so I would really love to ask people for their advice on what they think we should do, the best way to get started.
Caleb: Wonderful.
John: And just to add to that, I think, we’re learning as well, we’re trying to put things out there that are useful, helpful, and like what Bryce is saying, also resource wise. But we also need to learn. Yes, feedback would be great and share any ideas, context, views for others to learn, be absolutely amazing.
Thank you, Caleb, for having us.
Bryce: Yeah
Caleb: Thank you both for joining me. This has been fantastic.
Photo: Courtesy of Microsoft. Link.
Links:
https://www.microsoft.com/en-us/store/b/accessible-adaptive-devices-accessories
https://www.xbox.com/en-US/accessories/controllers/xbox-adaptive-controller
ADVERTISEMENT
