Heather KnightI love the idea of there being a robot at a cocktail party that can help. Like, “See that guy in the striped shirt? Just so you know, he’s really into flyfishing too.” Maybe they won’t be the full-out social characters, but they certainly will have the capacity to make the connections that we’re not always so good at.
When do you first remember thinking of yourself as a designer/artist? How did you end up doing what you do today?
I’m an engineer first, but I still feel like an artist sometimes. As a robot designer, there’s so much technological knowledge required to do anything that’s even slightly interesting with robotics – and I’m very interested in actually moving the state of the art forward –, that I’m first and foremost an engineer. I really care about interactive systems, I’m really interested in what we can learn about being human from recreating the means of ourselves through social technology. One of the things that keeps me coming back and makes me so excited to be in the field that I’m in is definitely that I do get to be creative and that I do get to play with art and installation and theater in my work.
What excites you about being a designer/artist? Why do you keep doing it?
“Artist” seems like such a [makes sound] sort of position – you can’t translate that into typing. I feel more comfortable thinking of myself as just being an engineer, because that’s my explicit training. I have a couple of degrees in engineering, and I’m working on another degree right now. I’m in an engineering school.
My paper’s titles are like “A Saavy Robot Standup Comic: Online Learning through Audience Tracking” or “Devilishly Charming Robots and Charismatic Machines.” My most recent one is “Eight Lessons Learned About Non-verbal Interaction Through Robot Theater.” So all of it is very heavily in the entertainment world. I’ve been making robots for about nine years, but since officially starting strongly with this robot theater project, everything I’m doing now really is at that intersection. It’s funny because I’m still explicitly in this engineering school. I’m going to have a harder time convincing people I’m not some kind of artist the more I do this work.
I’m very much of an ideas person. I really enjoy inspiring people. Obviously, I’m very excited about robots and I’m very excited about social technology. I like this idea that there is a possibility to transform technology into a space where it brings us together rather than pushing us apart or that it can empower people rather than replace them. A lot of times when you hear that someone designs or builds robots, you’re either thinking “Ooh, science fiction,” or “Oh, so you’re making something for a factory.”
I think that automation is like the old-fashioned robots. It’s not that there’s not still a place and utility for efficiently creating new technology. Obama’s really excited about this robotics initiative right now that could reduce the development time between the invention of a new technology and its implementation in society. Obviously, that requires a lot of machining etc., but I’m really excited about pushing the boundaries of what’s possible with technology and what we can do with technology and how we can really use it in our everyday life. I believe that making interfaces – like thinking about sociability as an interface, or like touch screens were a disruptive interface –, can make us be slightly more aware of how groundbreaking some of these new applications in that space could be.
What do you think is your most important skill, and why?
That’s a really good question. I’ll just start talking and see what happens. I don’t think I have a single answer, but something that has turned out to be really useful in several of my past projects is the fact that I’m a systems person. I’m really good at making a whole project come together, maybe because I’ve worked with the technologies for a while, but maybe also just because of how I’m wired.
I feel like I have a pretty good sense of what’s possible and how to bring pieces together. It often takes collaboration with other individuals, because robotics is one of the most interdisciplinary fields you can possibly be in. Even if you just keep it completely engineering, you need to be able to program, be an electrical engineer, and be able to physically construct the object. Whatever application space you throw it in to, you need to be able to build it. Whether it’s working with entertainment or whether it’s working with psychologists: it’s going to be in everyday life. Even if it’s just harvesting plants, you need to understand how agriculture works and what the state of farming is. No matter what you’re doing with robots, you always need to keep your eyes on many, many balls – it’s always a juggle.
I think the systems engineering perspective that I bring is one thing. But also – in terms of a unique skill set – my familiarity with social robots. There are not that many people that are thinking about that yet. I like the idea that perhaps I can spread knowledge of that possibility and bring in a lot of other talented people to co-create that future.
Tell us about the community you’re working in.
Right now, the community I’m working in is very interdisciplinary because I’m exclusively looking at the intersection between entertainment performance and robotics, particularly live but also filmmaking. I founded the Robot Film Festival, which took place for the first time on July 16 2011. We had over 70 submissions from filmmakers, animators, motion graphics people, hobbyists, and more art and technology people that were creating installations that were actually onsite at the event. So this year a lot of us have contributed our individual videos, but for the future, I’m really excited about getting them to meet each other. You could imagine it as speed dating for technologists and artists. I really believe that bringing people with different backgrounds together at the table will allow us to create new ideas and new technologies that would be impossible if we just keep those communities isolated.
In what ways does working within a community influence your work?
It totally has an influence. How, that’s a little bit harder. It’s maybe the exposure to the idea of what is possible. I think that it’s hard to really achieve dramatic things in your field if you’re not passionate. One thing that I really enjoy about spending time with designers and with artists and with performers, is that you get this perspective on passion, and “why”, and critique and really thinking about what actually matters. Not just what matters to you, but what matters to the people that you’re designing your objects for.
What role does collaboration with others play for your projects?
I feel like my number one collaborator right now is my robot, Data. It’s been really interesting to watch how that developed as I’ve been in this realm of exploring his character. It’s not like he’ll wake up at night and talk to me, that doesn’t happen. But he does have his own Twitter account. It’s been really interesting to hear as things have gotten more visible, like to be proposing ideas for performances or even writing content for my robot. I’m just like “Data would totally not say that. He just wouldn’t do that.” I started not just curating but actually trying to find a consistency within the robot. He has some motives, like he’s an aspiring celebrity, he’s a little pompous but he’s also like a born yesterday character. He doesn’t fully get it, and so he can make these ridiculous mistakes. It’s interesting. All of that character is obviously in the minds, but I really think it makes the work more coherent as well and how he can relate to different audiences and people. It’s been interesting to have a collaborator that is a consistent machine.
What do you wish to achieve with your work?
I’m really interested in enabling there to be everyday robots in our lives. More of the technologies already exist than we usually acknowledge, but I don’t think energy is necessarily being put in how to glue them together. The voice recognition is limited, but if there’s one thing I’ve learned, it’s about working within constraints. Say you’re talking computer vision and you’re just ensuring your installation is in a place where you have a clear background, so that you can actually see the person in front of it to create a visual – like a bird coming out of your hands or music formed by negative space and your silhouette –, you have to construct this space within the limitations of your sensing capabilities to make something really compelling that touches people.
That’s the kind of intelligence and talent that I would like to bring to my field. I feel like there is a lot of technology pre-existing. As engineers we are often perfectionists, we really think about making the technology absolutely the pinnacle of what it could possibly be before deploying it in to the general world. I think that designers and artists are really good at working within constraints to make something that is compelling, and I would like to start seeing that in my field as soon as possible.
What do you think you can actually achieve with your work, where are the limitations?
One thing that I always struggle with is speech technologies, where it can be hard for a robot to parse what you’re saying or for you to understand how it’s pronouncing something. I really think there’s a great space to incorporate tracks for emotion and modulation and timing into the voice which I haven’t seen from a technological standpoint. That has applications well beyond robotics, say for enabling people who have lost their voice to speak in a natural way. We have systems that can be trained off of their past voice samples, but it still doesn’t sound like them because you don’t have any of the natural modulation. That’s one of those cases where we actually have to do a lot more research about how we’re structured and better understand the social behavior of the voice before we can try to replicate that.
You really learn a lot about being human by trying to replicate some of our social capabilities like that. In my past work, I was thinking a lot about touch. If you had a robot teddy bear, there are some natural things it could start doing when speaking to you. If it’s doing a good job and you like it, you would give it a hug. How do you say hello? If it’s snoring, maybe you should wake it up. There are some gestures that you tend to use for that. Is it ticklish? Tickle Me Elmo is so successful, right?
Imagine if you could take that from being a one bit thing. If you could start scaling up the richness of things besides just vision, like with speech, touch, proximity, and the way we move. If you could start to parse in a much more complex way the way we are expressive and the way that we can interpret behavior.
How has your work changed over the years? How did your goals change?
It has changed a lot. It’s interesting because one thing that really bootstrapped my work in the field was that I started out at the very beginning working with a really awesome group of people and an awesome lab at MIT in the media lab run by Cynthia Breazeal with maybe twelve grad students. Cynthia Breazeal basically invented the field of social robots. She has this awesome robot Kismet that everyone should Google if you haven’t seen it already. It’s a social robot that responds to the tone of your voice because that’s the easiest way to track your emotional state. It’s easier than facial expressions because sometimes we smile when we’re not happy, but we tend to use the same voice profile when we’re angry versus sad or bored or whatever, there are less false positives and you have a more consistent mapping. That obviously had a huge influence on thinking about sociability.
Another great influence was also a member of that lab, Guy Hoffman. He’s now a professor in Israel. We had shared an office for a little while. He’s more of a software person, whereas I’m more of a sensors and physical person. I’m doing software now, but that’s not where I began. It’s not always where I’m most comfortable, but it’s necessary to make awesome behaviors happen. Machine learning is important too. Guy Hoffman did a thesis where he was thinking about a hybrid control system, a robot actor that was playing onstage with two human characters. It could do some things locally that were autonomous, like orient towards someone’s face or play through an animation. To make sure everything happened in order, to keep things timed and to make it feel like a real interaction, he would make sure things happened in sequence. It was a scripted play, which was a really helpful thing. That’s why theater’s great: it’s repeatable. It’s never exactly the same. Acting methodology includes the idea of reacting off the other actor’s face or off their actions rather than thinking about playing through the script because you can very easily sense that there’s a difference. There is a dyssychrony if you’re not making it real every time. That’s something that robots could learn that they’re not that great at right now.
He started thinking in that direction, and it’s interesting because he’s still doing some work there and now I’m doing work there as well. I’m kind of making that my central thing, but we still have such different takes on it and it’s so complementary because he really is thinking in the mind of the robot actor and I’m really thinking about the sensing. I’m really in the mind of the audience. I want to make this compelling experience for the people that are there. I still think very much in sensors, I can’t get away from my electrical engineering background. I really care about that interface where it seems a little bit more in the AI. Like, what is the meaning of all of this for me, as a robot?
Did your goals change?
Yeah, definitely. It’s interesting because I didn’t always know that I wanted to make robots. I started building some projects with this group and watched people engaged with the piece, seeing it impact people and feeling the way they cared got me addicted. I don’t think I admitted that to myself until later. My dad was an engineer and my mother was a Peace Corps volunteer in Brazil and worked with a lot of international non-profits. So I have this interesting culture and I have this interest in impacting the world. I always wanted my work to be able to translate to people. I think I’ve just gotten more and more explicit about that as my career path has gone forward. I have this very explicit interest in creating interdisciplinary work and intersecting fields that I didn’t necessarily have a long time ago. I feel like dialogues across different communities are super, super important, and I definitely care about the artistic application and larger reflection, even the ethics of what we’re doing in a way that wouldn’t have necessarily occurred to me as kind of a pure nerd back at the beginning.
Maybe I’m getting worse at being a nerd. I said that to Jer Thorp once. I said “I’m worried if I do too many cool things, maybe I’ll start losing my nerd street cred.” He was like, “Heather you make robots. You’re going to be fine.”
If you had the chance to tackle a really big problem our society is confronted with, what would it be?
There are issues that I think are most fundamentally important, and there are the issues that I feel like I’m most qualified to actually have a real impact on in a short time frame. I’m very conscious and care quite a lot about our relationship with nature and the environment in general. From my current technological capacity, I like the idea of empowering people through technology. It’s something that I’m most capable of and that actually impacts the way your life is structured and moves forward and what you’re capable of doing.
I had no idea that losing the ability to drive a car was a real marker for certain elderly people. A transition from independence to dependence. This idea that you can look at artifacts like that and think about how to make that break not so harsh or to make it so that people can be independent longer, I think that is really interesting. For people to be able to achieve more, to actually move around in their lives and not be stuck in front of their computers all the time, for us to be able to communicate with each other rather than always interfacing through technology. I love the idea of there being a robot at a cocktail party that can help. Like, “See that guy in the striped shirt? Just so you know, he’s really into flyfishing too.” Maybe they won’t be the full-out social characters, but they certainly will have the capacity to make the connections that we’re not always so good at. We’re not constantly connected to the cloud. We don’t have the chip in our heads yet.
Heather Knight is a roboticist who creates socially intelligent robot performances and sensor-based electronic art. She founded the Robot Film Festival to inject a sense of playfulness into traditional science and engineering and explore new frontiers for robotics before the technology is even possible.