Here is Mary’s talk at JS Conf EU 2012, which gives a lovely overview of Isla and includes that choice quote about syntax.
I was reading some of the links from your website and playing with the projects and all that sort of thing. And in one of the articles you linked to, you mentioned that as a 14 year old, you picked up the 700 page book Foundations of Mac programming, and that was perhaps your introduction to programming. Is that true? That was your first encounter with it?
Yeah, yeah, pretty close. It was a difficult book that I picked up that was called Mac Programming Starter Kit that you could buy in my local bookshop, but it was the same idea. It would take you through programming from the beginning, both in C on the Mac. And I in fact bought Foundations Of Mac Programming, because I’d found the prior book too hard to understand what was going on and Foundations was easier. And so that’s the one that I ended up working through in full.
And what was it that drew you to want to do that programming? To go from I assume not having much exposure to programming to then picking up a quite a large tome about it.
I played around with a tool called ResEdit that was came with the Mac back in the day. And the system seven days of Mac OS and back then Mac programmes were composed of two chunks, some code as a binary code and then some what they called resources. And those were things like sounds and images and text. And with ResEdit, you could edit those parts of any Mac programme. You could change the icon for example, or change the images. If it was, let’s say a game or something like that, or change the sounds. And so I’d been messing around with my parents computer using that tool. And then I somehow found my way to the programming section of the local bookshop and saw there were programming books. And I think I bought the kind of couple of easiest looking ones that promise to take you from the beginning and a crucial component was that they came with an IDE that you could instal, which would kind of at least get you going really with a environment where you could actually write and run code.
That makes so much sense. I also was a Mac user my entire life. I remember quite fondly ResEdit And as I talk to people and learn about their histories and how they got into programming, if they were on the Mac, it seems like HyperCard, or ResEdit were the two doorways into programming. It’s neat in that both of them start you off playing with the tangible parts of the software, the GUI parts of it rather than the command line say that you would run into if you were on the, on the DOS side of things, that would be your first sort of encounter with a programmer interface. But on the Mac, it was ResEdit for, like you said, editing sound, assets or icons and that sort of thing. And in HyperCard of course it’s very visual.
So, and when I think back about my own early days of programming, one of the things that I reflect on fondly is the difference between what I imagined programming was before I learned for real what programming was like. And I’m wondering, I want to start asking this question. You’re the first person I’ve asked this question to, but I think it’s going to be one of my common things going forward. Do you remember anything about what you imagined programming was going to be like before you learned? And do you remember how that changed or what it was like as you sort of came into really understanding what programming is?
Yeah, I do. I remember my dad said that computers are all just ones and zeros. And I asked him, I mean, how do they show numbers higher than one, if that’s the case? And he said, “I don’t know.” And so I had some vague notions that code would somehow get turned into ones and zeros, but that seemed… I almost didn’t believe that because it seemed farfetched that you could encode everything in as long numbers. So that was my initial idea. But I think I actually want start building programmes essentially tutorials that were in these books, then I think a new mental model just probably completely eclipsed, whatever had been there before, because I could almost figure it out .empirically they say, “Oh, when you type in this bit of code, then this window appears on the screen or something like that.”
So I don’t think I’ve built my new mental model on the ashes of any previous ones. My favourite mental, like pre-programming mental model I’ve heard that I wish I’d had is that somebody really liked Super Nintendo games. And so they’d think, “Well, the images are quite big on my TV, but the cartridge is only four inches long by maybe three inches high.” So do they sort of fold up the pictures inside it because if the images are bigger than they could possibly fit inside the cartridge. So how does that work? And I think that’s like a lot more, I really liked that because it’s so physical and in many ways it makes more sense than whatever crazy ones and zeros I had in my head.
Yeah. Whatever poisoned sand and all of that. That literally is. Yeah.
Yeah. That’s fantastic. You were 14 when you first picked up those books or thereabouts and imagine continued playing with programming. What was it about programming that hooked you and how did that nurture and develop as you went forward?
Yeah. I wrote a series of small desktop applications for the Mac starting out from tutorials and then kind of making my own things. So I remember I wrote a thing that you could type in an error code that the Mac showed you, and it would tell you what it meant. And I wrote a very, very simple word processor where you could… It was just like a notes app but you could type into it and save stuff. So I wrote little programmes like that and then I never really liked maths very much. Though I was okay at it, but I just didn’t really like it that much. And I thought that serious programming would require lots of maths, so I assumed I would not want to do Computer Science at University. And so I was thinking about doing something else and then just through luck, my uncle David, he put me in touch with… Who was a professor at Oxford.
And he put me in touch with a professor of Computer Science who was also at Oxford to just have a talk about what my aspirations were. And this professor said, “Oh, math can be part of Computer Science, but you don’t need to spend your whole time doing math in order to enjoy programming and enjoy Computer Science.” And so genuinely because of that conversation, I decided to do Computer Science, which why I’m so very grateful to that person because I really don’t think I would have done it unless I’d been disabused of that notion about math being integral to programming.
I love moments like that where you can, and they tend to be frequent at around that age where you’re forced to decide perhaps for the first time in a big way, what you want to do with your life. It was this conversation with this person that put me on the path that I’m on. There’s something special about that. And I’m curious, because you mentioned it just very briefly in passing. Do you know what you would have done if you hadn’t studied Computer Science, what you would have studied instead?
Yeah, I think I would have done English most likely. That would have been a bit weird because by that point I would have been doing A levels and I did Maths and Physics and Geography. So that would’ve been a bit of a jump to get over to English, but I think maybe possible just because I really like artsy stuff as well. I mean, there’s some outside chance that I would’ve done structural engineering instead. But again, I think that was probably just a kid saying that I was interested in and those two feel like the salient ones that the arts and programming basically.
And of course, Structural Engineering, you’re going to have a lot of Math in that one.
At least that’s my impression as an outsider.
Mine too. Yeah.
Yeah. I’m going to jump ahead a few years and ask about this, mostly for my own personal curiosity. So I’m going to try to resist or to jump up and down too much with fan boyish excitement, but you eventually went to work at Ableton and I’m personally really excited about Ableton because I will be using Ableton Live to edit this podcast. I think that the things that they have in their user interface, especially for some of the sound processing plugins that you can use devices, that’s what they’re called. The little sound editing devices, like their equaliser, the parametric equaliser and that sort of thing. I think some of the UI work that is in there is just absolutely phenomenal. And so I’m curious to hear what it was like to work at Ableton, what you worked on while you were there, what that whole chapter of your life was about?
When I was there, we were about a hundred people. So it was in 2010, 2011. And I worked on the… They essentially had an internal team that built their website and their shop. And so I was on that team, writing Python mostly. And so we built a bunch of the shop, basically, that used to buy Ableton and then a fair amount of the stuff that built documentation and things like that. They’ve since changed their website. And obviously it’s been like 10 years, so there’ve been a lot of changes, but so we built that kind of stuff. And to work there was very special because I think pretty much everyone who was there really loved Ableton Live and you either used it or cared about it in some way. And so it felt great to work at a place where everyone’s excited about it.
And there were also lots of people who were essentially professional musicians who maybe had a part-time or second job Ableton, moving to make some extra cash or whatever. And so there was just tonnes of music people around as well. And that was really exciting too. So it just felt like there was a real culture of caring about what the company was making. And the other thing that stayed with me was how… So I only observed the development of the product itself, one step removed because I didn’t work on it directly, but it felt like the people who worked on live itself was so careful about adding things to the product and thinking very carefully about how it should work and making sure it was extremely robust because obviously it’s a live music tool.
And so you don’t want it to crash. And just a sense of carefulness that I think married with what felt to me, I think is still kind of next level like, so UI design really put into me. So it have some deep core of like, you can be careful and also innovative at the same time, which I think is something that those two ideas don’t often go together. And I think able to improve that it’s possible. I think, Live 8 was the most recent version that come out. So that was in 2010 when I joined. And then, there have not been too many more versions since then which I think is in many ways a good thing, because it is very careful taking, making sure what they’re building is the right thing I think is really cool.
That’s good to hear because that lines up with my experience as somebody who has used Live both in a hobbyist capacity for things like editing this podcast and as a very serious professional tool as part of live performance and as part of my job and it definitely does feel like software that is built with a great attention to stability and performance and robustness, but also on the design side, like you identified like it. I’ve been using it since version five, which I think would have been in 2004, 2005, somewhere in there, I started using it. And since then, things still work in the same way that they did back then. And they’ve added a tremendous amount of capability and power and new features to the programme, but they’ve done it in a way that to me never really broke my workflows, never really broke my expectations, or if they did change something that required me to slightly adjust how I work, it was something that made sense.
It was something that was not a complete departure from what I was used to. It was like a slight augmentation or an additional thing just to cope with the greatly expanded scope that the product has had in that time. And that’s very interesting to me, like you said, and that it proves that it’s possible to do this. And I think that that’s a really important thing to reflect on in the face of… And I’ll just pick on one example, companies like Apple who are beloved for their design prowess, but are also notorious for breaking people’s workflows or for doing a complete reinvention of something when perhaps that’s not necessary.
And so it’s nice to have these counter examples of things that are exquisitely designed, but that are also very respectful of the expectations of their longtime users and do a very very good job of leveraging familiarity against the need to… And I guess, one reason why perhaps Apple does what they do or other companies do what they do in that reinvention when it bad kind of way is because there’s that pressure to constantly be adding new capabilities so that you can justify charging for upgrades. And I think, yeah, Ableton are a great example of a company that’s found a way to keep working on their product and keep growing and building something without having to force people into an upgrade cycle that is not beneficial.
Well, one example that comes to mind is I’ve not used this feature. So I believe it came in Live 9 where you can essentially turn an audio or some audio into MIDI. So, a wave back into MIDI and I believe that came in Live 9 and up. When I was at the company that feature was being talked about. And I think that was to me, emblematic that they said, “Yeah, this is going to be Live 9. So probably in like three years time.” And that was just like, it’s very cool. I think it’s like, “Okay, we’ve got this cool feature. It’s going to take a while.” We’re looking that far ahead. And it’s not like, they’re just saying, “Oh, it’s out when it’s out,” but it’s like, “There’s no rush. It’s just going to make some things better in certain ways.” And it takes however long it takes to add. And so, exactly like you say, I think it’s an example of just, we only add things when they make it better, not for some release cycle or to justify charging for an upgrade. I think it’s super powerful.
So now I’m going to start getting into some of the projects that you’ve made and you have made a lot of projects. So there’s a real banquet of choices here for things we could talk about. And I want to start by talking about Isla, which is a programming language that you made specifically for young kids. And it’s something that you intended to be a good first language. So, somebody’s first encounter with programming would be this language. And in one of the talks that you gave about it, I think it was in one of the talks. You mentioned that it would be a language that kids would very likely outgrow as they continued learning to programme that if they stuck with programming, that Isla would fade in relevance and they might move on to something else. And I was wondering if there were aspects of this language that you specifically designed with that in mind that you specifically designed to encourage kids to outgrow it, because that’s a really interesting design constraint, if that was something that you made intentional in any way. And so I’m wondering if that’s what you might’ve done to accomplish that.
Yeah, absolutely. The explicit thing that I did to aim for that was to not try and make it something that a child or anyone could grow with. And so I think of Isla is essentially a data definition language in the sense that you write code that defines data structures really. So you can say something like if you’re effectively, you can create an object, you can add attributes with values to that object. You can create lists of objects and so on, and that’s about it. So you’re just defining data when you write the code. And so by definition, it’s not going to be a language that you can do that much with. And I think the discipline I put on myself was to say, that’s okay. I want simplicity and accessibility to be the most important thing. And so, essentially not forcing myself to make it a powerful language, lets me double down on the simplicity and make things as easy as possible.
And the idea behind Isla in the end was that the kid would write some code which defined some data so they would create an object code with like either had the name Satin and it had an X attribute and a Y attribute that was set to some numbers. And then it had a colour attribute or something like that. So just an object with those four attributes and then they might define another object along similar lines called Mars or something. And then another one called Venus, let’s say. And they’d put them in a list. And then that would be where things ended for the kid in terms of their input. But one would be able to layer on environments that would use that data to do something interesting. So in that case, it was to create a little planetary simulation where it would use, maybe the size of the planet that you could also define to define the mass of the planet.
And then the starting positions would be the X and the Y of each of those objects. And then it would just make them all virtual sun that they could add as well and stuff. And so, the idea is that the kid would define some cool piece of data and then a more advanced programme. It would be able to set up the environment that would use that data to do something that was hopefully interesting. And so, to try and get over the somewhat interesting result out of not too much difficult code. So that was the goal. I think there were a lot of problems with all of that. I’m happy to talk about if that’s interesting or-
Yeah, that’s very interesting. I would love to hear what you see as the problems with this approach.
Yeah. The first problem, I think, is that with a number of… I don’t know. You could call them like dynamic medium projects or future of programming projects or anything that’s trying to change the way that people programme computers, basically. A number of those projects that I’ve done have a split focus between the tool that you’re building and your objectives for what that does. And then another objective around what you want to do for yourself as the creator. So for example, Isla was the first programming language that I’d ever implemented a compiler for.
And so part of my goal was just to implement a compiler for the first time. So that split focus meant that I think I did not pay as much attention to the quality of the language and if even having a language was the right move as I could have done. So for example, a few people that I know used Isla with their kids and tried it out, but as far as I know beyond that, it’s got zero kid users basically and stuff. And partly I think that’s because the design is a language and the environment themselves were compromised by my own kind of separate goals for writing a compiler if that make sense.
Yeah. That makes perfect sense. And to me it seems inevitable that if you flip the way of thinking of it on its head, instead of thinking of it as this is going to be a transformative programming environment, and I need to learn some things in order to build that. If you instead think of it as I want to build transformative programming environments, but to do that one requires practise just like they do with anything, then it seems inevitable that you’re going to end up with that split focus. And so was the problem then that you went into it thinking that it wasn’t a primarily a learning project that it was instead, primarily something that was meant to have a life of its own?
Yeah, I think I was a bit blind to it that I had this split focus between my own goals of building a variety of compiler and the goals of creating a language that would actually be fun or educational or useful for children to use. And I think I’ve noticed that as a common failure mode inside the projects that I’ve built along those lines. So I did another one called Code Lauren that I think suffered the same fate. I think you’re absolutely right that it’s like, “Okay, in order to do something like, build a language for children, then you may need to learn some stuff. And that’s just part of what you need to do and that’s fine. And I think that’s absolutely right. I think where it gets trickier, certainly in my experience is that the focus on the what of what you’re building.
So, that secondary goal of, “Oh, I want to write a compiler constrains one to that as the solution for helping children get into programming or whatever.” It’s like, Oh, there could be other ways that are better for helping children write their first code, but the solutions that have been presupposed as a compiler. And so I think that thinking inside the box is sometimes a dangerous consequence of these this kind of split focus.
That’s a 100% something that I think that I have to deal with because I’m building my own futuristic programming projects and they are visual languages because that’s the area that is my obsession. And would I write a futuristic programming project that is a compiler? Heck no. Because well, for one, I’m scared of compilers and of that whole can of worms as I see it, but it’s just not something that’s interesting to me. And so what I’m curious is now that you’ve done these projects, Isla and Code Lauren, which we will talk about in a bit. And you’ve sort of discovered that that’s an affliction that you have where you are constrained to inside the box, thinking based on what your presupposed solution to the broader problem is. What advice do you have for surely the many people out there who will hear this, who are also perhaps trapped in that box and may not yet have realised that that is something that is constraining their thinking? What have you come to having gone through that reflection?
I think that the first thing is that Isla would have been absolutely fine if I was a kid too, which obviously it’s hard to imagine, but because I would be my own user. And so I think exactly what you’re talking about with Hest my understanding is that you are building this to use yourself and to use as part of your work. And so I think that is a really good way to avoid this failure mode that I’ve talked about, because then, you can’t fool yourself by saying like, “Oh, some other people who are not me, but some other people will get value out of this.” Because you can’t tell yourself, obviously, if you’re not getting value out of it if you’ve built it for yourself.” The second piece that I think I’ve found helpful more recently is it took me a long, I don’t know if I’ve totally shaken it yet, actually, but it’s taking me a long time to shake off this idea that building software requires writing code.
And so I think in the past, I’ve ignored ways of building software that aren’t primarily writing code because they seem like, “Oh, that’s never going to be powerful enough. Or that’s always just going to be a toy.” And I feel like I’m more open in my mind to tools like that, that maybe in the past I would have looked down on is not powerful enough. And so the other thing that I’ve taken away from this I think is trying to not dismiss things just because they’re not going after a particular goal in the same way that I am. So to give a concrete example? Something like GameMaker, which I’ve been completely obsessed with recently, it’s a kind of an ID for making 2D games and it’s been used to make some very cool commercial games like Nuclear Throne is one or the first version of Spelunky.
Or Hyper Light Drifter is a favourite of mine.
Right. Exactly. Yeah, exactly. Fantastic. And so it’s definitely serious and not just a toy and it also requires some code for things, for sure, but it also does a lot of work to avoid requiring code in for many things. And so I think I’ve just tried to make myself more and more open to if you like to put in some Stuart Brand terms like low road tools like that, that maybe would have been seen as inconsequential in the past to me at any rate.
And actually one of my later questions, I think it’s going to dovetail really nicely with what you’ve just said, because I’m going to ask a little bit about your thoughts on text code and other more visual systems. Because there’s some other things that you’ve said that I think hint, that you might have some thoughts about that, or some thoughts that might’ve changed over time. So that’s something that I’m going to actually come back to in a bit, but still on the topic of Isla, you gave a talk at JS Conf in 2012, and you said the following, and I’m just going to read it verbatim, just because I found it really charming. You said, “There’s this notion of indirection, which is the reason that I think programming is so fun. I type in something to the computer here and then something happens over there.
An in-between there’s this magic and that magic is fun.” And later you expand on that by saying, “It’s like setting up a long line of dominoes and then you flick one and you watch the rest of them fall.” And depending on how you interpret or how one were to interpret that it might be seen as going against the conventional wisdom that indirection, especially in user interfaces is a bad thing. And that’s sort of where Brett Victor made a lot of great points about direct manipulation being good for some things in some cases, because of that removal of indirection. And so I’m wondering what are your thoughts on indirection? And maybe those have changed in the time since 2012 granted long time ago, but that indirection being one of the places where fun can come in. Do you see the issue there with that it might cause for some people? Or how do you wrestle with indirection? What do you think about indirection?
I think, I still think that’s true. It’s a really interesting idea about how I actually direct control and mastery of, and directness of things is a fun feeling too. I think that’s absolutely right. If there’s any delay on when you do an input to seeing the result, that can be very disempowering and energy sapping. I think I would add on to the idea of indirection that maybe now I would think about it as power or leverage where you do a thing over here and you see an effect over there, which gives you a sense of, like I say, power or control or mastery or leverage something around those words where you can have an effect that sort of distance. So I think I can imagine this being a similar thing too, if you ran a company and the feeling that you might get of knowing that there were people who were working on stuff and building it and you weren’t even aware of it.
And then they show, and it’s like, “Wow, that just happened because of the system that I set in place or whatever.” That’s really cool. And I think that might be a similar sense of fun. And so, I think to boil it down, writing a line of code and then seeing a window pop up on screen feels similar where it’s like, “Oh, I did this thing that felt reasonably small to do. And then this big thing comes out of it.” It’s like a strong sense of leverage coming out of it. And so I still think that’s why I find programming fun.
And I think emergence factors in here as well, where I’ve heard game designers talk about seeing players do things in their games that they never would have anticipated and how cool that is. And it’s like, “Yeah, I created a system or a smaller thing. And this extra consequences came out of it. That’s so cool.” Because again, it’s like that sense of indirection plus power. And I think also possibly magic, you think you have a magic trick feels related as well. Where it’s like, if you can see that the person had the ball in their other hand the whole time or whatever, that’s not magic. You just see there’s a directness there. Whereas if somehow the ball appears to have flown between from one hand to the other or the other man appears to have jumped from one box to another or something that’s like, “Wow,” there’s something literally supernatural about that. That feels powerful and almost, or inspiring, I guess.
… and sort of almost or inspiring, I guess.
So listening through that thinking I was holding in my mind the whole time that this might be one of those cases where we’re using one word in two different notions and that’s probably worth disentangling them somewhat, that the point you made very early on about latency not being good and about certain kinds of indirection where it’s sort of this artificial limitation being imposed on your expressivity. That’s no good, but there’s this other way of seeing indirection where it’s that leverage that that is good. And maybe those are two separate notions that are unfortunately tied together with one word, but then right at the end, you start talking about this magic of not wanting to see the ball go from one hand to the other of the magician. And so that to me, that’s weird.
Because I started with HyperCard and then I graduated into Flash and that was my main programming environment for many, many, many years. And so then when Flash started dying and I switched over to programming on the web, this was in the beginning of the node modules. It has the massive, a black hole era. And so there was this new tool at the time called Yeoman and you’d instal it with brew and that’s cool. I’m good with that. And you’d type, yo new angular, whatever it is to invoke it. And it has this recipe where it’s like, “Oh, okay. I’m going to bootstrap a new little angular project and download all the no dependencies. And so I typed this command in, and it prints out something like 10,000 lines of output in my console saying, “Installing this, installing this, installing this,” And I have this panic attack and I’m ripping my hair off thinking, “What the hell have I just done to my computer?”
I’m installing tens of thousands of things. This can’t be right. Where are they all going? Are they being scattered all over my hard drive? Am I going to have to go dig into my library folder and find where they all went and cleaned them up. Thinking that all of those things were each like their own traditional Mac application where they have the application lives in one folder and its data lives in another folder. And it’s got some secret data over here and there’s some cache there and I’m thinking, I’ve just made this colossal mess. How does anybody work with this? How is this seen as a, “Hey, we make it easy to get started with your first project,” kind of tool that all these people are encouraging. And so it’s that same feeling of, I just do this little tiny thing, and there’s this huge exp… This flurry of activity that happens as a result.
And that’s something where I totally see the way that, that can be a positive thing when you are first learning if it’s not something unexpected and bad as it was in my case, I’ve had the opposite experience to where it’s very delightful, but it’s something that our tools that we’re building are trying to lessen. And so I wonder if that’s one of those things where we might have to put that as one of our benchmarks that we hold dear, when we’re designing these tools, thinking about if we do get to the point where a lot of the magic is dispelled, and a lot of the inner workings are surfaced in a way that doesn’t make them seem intimidating in their complexity, maybe that loss of magic will be something that is felt by people.
Yeah. I love that. I guess I really liked the teasing apart of these two ideas of indirection leverage. And I think you’re absolutely right, that the more one understands something that one is possible that would lead to a mundaneness, a feeling of mundaneness about it. I think it’s exactly what you said, that there’s something about leverage, which is putting in a small amount of something and getting an impressive result that comes out of it. And I think that’s about just maybe it’s about one’s preconceptions of what’s impressive and what’s not. Say for example, We know when I was a kid and I could write a few lines of code to create a window on my Mac, that was just like the windows that real cool official programmes could create, then I think it’s an impressive result from something that I knew how to do.
And so I think that gives a sense of leverage or power or fun or whatever. On the other hand, I think I really love this idea. There’s almost a diametric opposite, which is that sometimes I find if I’ve dived into a complex system or seems like a complex system and learnt how it works in detail. So for example, I did this with git, and then how it works in detail. That actually also brings a similar sense of, I don’t know, power and ability and competence or whatever, even though you actually now know all the details of how it works. And so I think it’s not so much about the method that you use to get there. It’s just more about, “Wow, I can do this thing that seemed very impressive before I knew how to do it.”
And that makes me feel like a sense of fun or a sense of ownership or whatever over my thing. And that might just be just a few lines of code that produces a window, or it might be a deep dive into, I don’t know. Learning how it works in detail, but there’s a similar feeling comes out of it. And because I think with the magic trick analogy, it’s actually, you don’t know how it works and that’s the point, but that’s someone else who’s doing that magic trick. It’s not you. And so I think if we limit it to just, “Okay. Well, what can provoke you on the things that one could do oneself?” It’s sense of power can come from like, exactly what you say, both those things, a sense of leverage or a sense of deep connection and understanding.
This reminds me a bit of Ted Nelson’s Xen, Computer Lib/Dream Machines, where he talks about this like priesthood of programmers who guard their secret spells that they can invoke upon the computer and how that’s detrimental to progress as society and incorporating computers into our lives. And the magician keeping the magic secret works when the relationship is between the magician and an audience. But when you are a magician yourself and you want to learn a new trick, I think the learning how the trick is done in that context still brings with it that sense of fun rather than the sense of disappointment that might come from an audience member knowing that the trick is actually mundane, the different objectives in each case help you have a positive relationship instead of a negative one when it comes to dispelling how the magic is done.
So the other thing that made me think of is perhaps if dispelling the magic that is going on inside the computer is something that were to take the fun away to make it mundane, maybe that could be addressed and this is where we get into like, I think that this is the case, and I know that other people do not think that this is the case. You could probably address that by changing the aesthetics of your programming interface or the aesthetics of your relationship with the computer and that if there’s some mundanity there, maybe that mundanity just comes as a result of you running into a barrier where now your experience is mitigated by things that are frustrations rather than things that are joyful.
So for example, if you take away some of that magic of… I type in one command and let’s say, in my case, I ran yoe whatever to instal and it installed thousands of things, if you take away that magical nature of it and perhaps replace it with, and this is a totally contrived example, but if I had to instal those things manually one at a time, or if I had to go and populate my package.json, and then run NPM to instal it or something like that.
Like if it becomes a little bit more manual and that dispels it, then my experience with it is mediated by the fact that I’m now just waiting on the computer to do some work. Like there’s some latency here. And that’s the part that governs my experience, where if it was instantaneous or effortless, or if it were more tactically interesting. That’s the bin time on, or if it were more visually interesting then that might still keep the element of fun even after the magic has been taken away.
Because the thing that’s mundane probably isn’t the absence of magic. It’s probably just whatever is behind the surface of the magic, like a ball going from one hand to the other is mundane until you’re juggling. And then it’s exciting and much the same way. A lot of maybe the plumbing of data that we do in programming is mundane until you dress it up with graphics and sound effects and challenges, and then it’s a video game. And so it’s the thing that I feel like this idea that the magic is fun and that behind the magic is mundanity. I really like that. I think that, that’s a really fruitful line of thinking and exploring the space. I think that’s super cool.
So after Isla, you later made a project called Code Lauren, which we’ve mentioned a couple of times so far, and Code Lauren is a game programming environment for beginners. And if I get this explanation wrong, feel free to jump in and say, “No, no, no, that’s not right,” but it’s a text editor and you can type in some code.
The code that you’re typing in maybe it does some graphics like making shapes or that sort of thing. And the graphics will be rendered right into the background of the text editor. You’ve got your text code and your graphics right there in the same space. And am I correct that the language that is in Code Lauren? Is that a language you also created?
It is. Yeah.
Cool. Yeah. I didn’t recognise the syntax and I wasn’t sure if it was something obscure or something that I just haven’t seen. And did you build the language and the environment together at the same time?
Yeah, I did. Yeah. They kind of co-evolved.
And that’s what I’m curious about, and that’s what I’m hoping you’ll talk about is what was the thinking behind the project? What was it like creating a language and its editor or the other way around an editor and it’s language at the same time? How did that all go?
So way examples of that, because I knew that it would be essentially used for writing code that would produce animations or games. Then I knew that was going to be an event loop. Was he around for the whole life of the programme? And so, I made that very easy, so you could just type forever and then some phrases and then whatever was inside the phrases would be the tick of that event loop. And so that was something that was pretty simple to do for the users. So that was cool.
On the other hand, I wanted to write a VM that would let you run your code in reverse. Basically, it’s like a technical flex to be honest with you. Because it would seem impressive, I suppose. And so, those two things don’t go very well together because… So yeah, I hit the goal of, “Yes, you can run your code step by step” like a step debugger forward and you can also do it backwards but a reverse debugger actually doesn’t make that much sense for something is an event loop that’s going around maybe 60 times a second at full speed because you’re just constantly clicking back, back, back, back, back, back, back like that. And so, you click back 20 times and you would be just a fraction of a second back in this run of the programme.
So it was very difficult to find the bit that you were interested in getting back to. And so these two goals, like I say, competed with each other a bit. And I think if I was taking a more honest run at this sort of thing, I would have made something that was much closer or at least I wouldn’t have been able to make something in as good. I don’t think, but I would have been aiming at something that was focused on this person wants to make a game or they want to make an animation. What does that tell me about how the tool should work? Whereas like I say, the focus was more like, “I want to create a VM and I want to create that runs a programming language in the browser.” And so, let me let that mostly dictate things.
I think the things that I did discover that seemed helpful, where I started to kind of graft on a debugging help for the users. So some things along the lines of what Bret Victor has worked on for a while, around learnable programming. So for example, there would be help on what arguments functions that you were going to call took. For example, there would be reasonably helpful error messages when you got syntax errors, sort of things like that. And another thing that I found that I think was helpful was having one way to do everything rather than multiple ways, reduced complexity for users. And so just having one way to call a function, for example, or having the whole thing be…Oh, what was it I’m going to give example? Having one way to instantiate variables. So just simple stuff like that. That’s like why put the extra complexity on the user?
And I’m sure on the VM and the language as well, there’s a win-win there, I think.
Yeah, yeah. For sure. Exactly. Yeah. That’s absolutely right.
I have a direction that I want to go with looking at Code Lauren, but before we go in that direction, I just wanted to see if there was anything else about it that you wanted to reflect on just because the direction I’m going to take it is in quite a different direction.
Yeah. I think this could have been something that if I focused it more on what I needed to make a game which I started to do towards the end of it, it would have had been more successful because it could have been something that I would find useful which would at least put me on a useful track. And so, I wish I’d doubled down more on making it something that was useful for me to make games. And so I definitely took that away.
That’s interesting to me, because of your most recent obsession GameMaker, it seems like you… And you’ve also made games. That’s something I haven’t incorporated into any of these questions, but you also make games in addition to making tools for making games. And so before I take this in a different direction, I’m just curious if now that you’ve played with GameMaker and now that you’ve had that experience with Code Lauren where you sort of realised, “Okay, the focus needs to, perhaps be on dedicating the tool to a specific kind of project that a person wants to make, or a certain kind of problem that they want to solve.”
Code Lauren is a far more interesting tool than GameMaker in the way that it offers a very unconventional user interface capabilities, like the ability to step forward and backward through the execution. The fact that the code editor and the graphics are right in the same window in the same space. But is there anything that you’re currently chewing on that you think might be, if you could go into GameMaker and augment it in some way to leverage some of these more interesting interface ideas, what might you do with something like GameMaker to give it more of that capability? Or what might you do with Code Lauren to bring it closer to the domain specific utility that you get out of a tool like GameMaker?
Yeah, I think Code Lauren would be served well by getting a bunch of things out of code and into a GUI. And so, I think that’s one of the huge strengths of GameMaker is that a bunch of things you can just do in a GUI. So, and I think it goes down to even the most simple things that are just so beautiful. In GameMaker you, when you create a Sprite, then the creative GameMaker know that a Sprite tends to have multiple frames. And so they give you a nice little painting editor, that’s a 100% GUI to create your image. And then they let you add more frames to your Sprite and reorder them and things like that. And then, there’s an implicit assumption that when a Sprite is displayed on screen, that it will have a current frame.
And the default behaviour is for it to just cycle through those frames at top speed, which is all staff that the user does not have to programme. That there’s an algorithm there clearly, but it’s just been created with a combination of using a GUI to define that algorithm and also using some conventions to define the algorithm. And I think that’s so powerful. So I think Code Lauren, if I went back to which I don’t think I will, but if I did, I would try and lean as much as possible away from code and say, “How much can the user do without writing any code?” I think, areas where GameMaker would be… Where I would love to see this version of GameMaker. And I think it existed because in GameMaker you are always manipulating an artefact, whether that’s code or images or whatever. Essentially boxes and arrows kind of connected with wires.
You’re always been creating an artefact that is not the final output. And I would love to see a version of GameMaker where you manipulate the game itself to create the game. And I believe there is a tool called click and play that is from 25 years ago or something that takes this approach. So, you essentially arrange some objects on a canvas and then to define how they work, you can literally click and drag to define the motion of a particular Sprite or something. So it’s like, “Oh, I want this to move from here over to here.” And then you do… That drag defines that movement.
So you don’t have to type, write any code to do that. Or another example is, you can run your game and then when an event happens, like one object runs into another, then a little model will pop up saying define what should happen in response to this collision. And there’s a number of events like that, that it’s sensitive to. And so again, you’re doing it by manipulating the actual output, which I think would be a really exciting idea. I feel sure that they ran into big limitations on how powerful that that can get, but I think it would be very cool.
And it’s something where one of the objections that people might surface when they encounter that idea is if you look at an environment like HyperCard worked in this way, Flash even worked in this way a little bit at one time with a feature called Behaviours. You can have these environments where you have your actual running game or system that you’re building or playing with right there. And you can work directly with it and specify the behaviours that things should have right in that running environment.
But the environment is still, it’s not forcing you to only do things in terms of what exists in that running environment. You’re still able to, when two things collide, bring up a modal and in that modal there’s a nice dedicated interface for resolving that one particular circumstance. So it’s not necessarily like you have to give up on the indirect tools that we have available to us in a system like GameMaker in order to get that world where you are working with the actual running interface. I think that those two ways of working are very compatible and that you could probably see an environment where you have both rather than just one or the other, which would be nice.
Yeah. I completely agree. I think you’re absolutely right. I almost see it as popping up the right UI, the right editor at the right time. And if that can stick within the direct manipulation metaphor or the output metaphor, then that’s great. But exactly like you say, it doesn’t have to, and users could still totally understand what’s going on. I think that’s something that working on Airtable has taught me that the philosophy there is… I mean, I don’t want to speak for everyone, but certainly the philosophy that people talk about, not somewhat frequently inside the company is this idea that of escape patches, where Airtable is a database that you can essentially directly manipulate in your… Most of the time, 99% of the time you’re not writing any code. You’re just manipulating data in a database and defining a schema relationships between pieces of that data.
But then sometimes you need to do something a bit more sophisticated. And so, you can create something called an automation which if you ever use Zapier, it’s somewhat similar where it’s like forms based programming. So you say, “Oh, let’s say when a record is created, then I want to update that record with these values or something like that.” And you can do various other things like send email and so on. And so the automations interface is a little escape patch that lets you either automate repetitive processes, or it lets you do things automatically where you don’t have to supervise them. So you could imagine that a process of around every night at midnight or something like that. But so it’s a little bit of extra complexity, but you’re certainly not programming and actually you do that bit.
It’s like, “Oh, well, if you can’t do everything in whatever the new UI or programming paradigm is, then it’s never going to be good enough. So you might as well just stop and just do it all in code. I just think that’s exactly the wrong way round. It’s more like, “Okay, let’s do 99% in this cool much more accessible UI. And then let’s have a little escape patch way where somebody can take on the extra bit of complexity, solve that little problem and then get back into the much nicer, much easier to use environment.”
And that’s a lesson that I think I’m going to imagine here. Maybe if Andy Hertzfeld or somebody who’s in the audience, they can reach out and fill me in. But I imagine that that’s something that people often learn or that they arrive at. And I’m going to imagine a debate between people who are against escape patches and are in favour of escape patches and hopefully arrive at the point in the middle that it seems like Airtable is at. And the argument against escape patches that I hear and I often guilty of making this argument is that they will give people a way to fall back on familiar habits or fall back on the way that they’ve always done it before. So they will bring forward whatever traditional experience they have with a programming environment, into your new novel way of creating some dynamic software system.
And they’ll just do the old thing. If you give them an escape patch into a world that looks more traditional. And so the example of the reason I mentioned Andy Hertzfeld as the original Macintosh, when it first shipped had a mouse, but it had no arrow keys because on the Macintosh team, I don’t know if this was specifically Steve Jobs or if it was somebody else there, but they decided not to include arrow keys because that would encourage people to use the mouse as much as they could that if you need to move the cursor the right way to do it in the GUI paradigm is to grab your mouse and to find the spot where you want the cursor to go into click. And so by taking away the arrow keys, they were forcing people to learn and become comfortable with the new paradigm.
And not too many years later, they added arrow keys back to the keyboard because there’s so many ways that arrow keys are useful beyond just the ability to move a cursor. If you want to make a game or something like that and have directional movement, arrow keys are very natural for that. If you only want to move the cursor one or two spaces, arrow keys are really nice for that. And that, even with the arrow keys, people still manage to get used to using the mouse and get used to using the GUI. And so, it’s neat to hear that at Airtable something like the automations system, which am I correct? That’s what you worked on. Yes?
That’s right. Yeah.
Cool. We’re going to come back to that. The automation system is seen as an escape patch from what is otherwise more like a database spreadsheety style interface.
Because they won’t want to pay the cost of having to learn the new way to do it. Of course, counterargument to that is often there isn’t a way to do it in the new environment or the way to do it in the new environment is very clunky. And that’s a whole other issue, but this idea of having gradations of escape patches, I think is a really elegant way to solve that problem and to find a really nice middle ground in that sort of contentious space.
Yeah. That totally makes sense. The not so good side of escape patches. I think the distinction is that exactly like you say, with the original Macintosh, they were imagining that most of their users would be people who’d only used command line computers before. And so those old habits would be tough to break, but I think when things break out and become popular, it’s not because a bunch of people have…
It’s not because a bunch of people have, that’s a massive overgeneralization, but with an example like this, I think this is probably true, but it’s not because all of the people who used an old thing now are going to move over to this new thing, though some of them, of course, will, it’s more that the gates have been opened to a whole new group of people. And so I think that the people who use Airtable, for example, are probably not going to have those old habits necessarily always to fall back on. And so there’s no danger of them just doing it in the less efficient way with a script, when they could be doing it in a more sort of quick way without a script. And I think that the same is true for something like Game Maker, which just opens up the ability to make games to just so many more people that those escape patches are I think about enabling someone to stay in the environment, rather than having someone be able to just bring in a bunch of preconceptions and let them use this environment.
Does that make sense? So, because I think again, taking Airtable as an example, the danger is that we provide a really nice database, but then they’re like, this is great. But after a few days, there’s just a bunch of stuff that I cannot do in like vanilla round table. And so I very sadly have to go back to my internally written tool. That’s all written in code by the internal programming team or whatever it is, because Airtable just will not grow with me. And these escape patches give that ability to grow.
And that is something that absolutely, I think it rings true specifically because you made a context around this, which is that you are looking at things like the Macintosh or like Airtable, which are reaching out to people who previously were not using a computer or previously were not using this sort of dynamic software that they are new people coming into the tent. And the thing that my own personal blindness is that I’m paying way more attention to the problem of taking existing expert computer users and trying to advance the tooling that they have. And that’s sort of where my own biases creep into this, where I’m thinking about people bringing their own experiences forward is because there’s that other side of there’s the one side of, let’s get more people into the tent.
There’s a lot of startups out there right now that are thinking about the next billion programmers and I think power to them. But then there’s the other side of it, which is once we have people in the tent or once we have the people who have been in the tent for a long time, what do we do with them? Because it’s, I think there’s no argument here that the existing tools that we have are just ripe for improvement in countless ways. And I think that sort of like what you identified with Airtable and having the escape hatches in a kind of a graduated fashion, it’s sort of, the onus is on us to make sure that no matter what experience people have, and no matter what they’re bringing into this, that we give them the capability to do whatever it is that they need to do.
And so in the case of a beginner user, if they run into a limitation with Airtable and they need some programmability because entering things manually, isn’t cutting it anymore. You have an automation feature for that. And if it is an expert programmer, who’s ,they’re familiar with command lines and text languages and compilers and all of that. And you give them some kind of visual tool or some kind of novel debugger or something like that, that if you are worried about giving them the escape hatch to, for them to fall back on, which is, it’s not just an argument that I made. I hear a lot of people make that objection to escape hatches that just puts the pressure on you as the author of the tool to make whatever new contribution you’re making very, very good.
And to make it, you have to sort of do that objection handling and that I think it might almost be a, like a good benchmark to give yourself. I love benchmarks for this kind of stuff to say, I am going to put escape hatches in my thing, and that’s going to force me to make my thing good enough so that people won’t need to, or won’t want to use those escape hatches. If they’re bringing in existing familiarity with whatever world that escape hatch opens into that the new thing is so much better or so much more compelling that they won’t necessarily need to fall back on old habits. And they only do fall back on old habits, perhaps, in the case of an expert user for the sake of something like a foreign function interface, or for the sake of using an existing library or something like that. I think closure did this really well. You’re current former closurist.
Yeah. I haven’t written any for a while, but written a reasonable amount. Yeah.
Yeah. I think you’re absolutely right. And I really loved the idea of the thing just has to be way better. And then that is why people will switch to it. Cause that often seems to be the challenge with, as you said, the people in the tent and that if they’re switching, then they’ve definitely got some ingrained habits to that need to be battled with.
And is that something when you’re designing a feature like automations or when other people at Airtable are working on the product, what sort of, what’s the balance of consideration between the people who are going to be coming to sort of reconfigurable software, like Airtable for the first time, versus people who are already familiar with programming or very maybe very advanced users of something like Excel, how do you kind of split the design space for those different people coming to it?
Yeah. It’s a really interesting question. It’s something that we think about a lot. I think, first of all, that it kind of, this is an oversimplification, but there tend to be users that set up a table basis who will need to design the schema. Maybe they’ll be the ones who are more likely to create automations that run automated processes on the data and the base. And then there are the people who essentially use the base as a tool to get their work done. So you could imagine that let’s say we’re in, like, it’s somebody who runs a hair salon, they’ve created an Airtable base to keep track of their customers and their hairstylists. They employ, and let’s say their appointments and all of that stuff, probably the hair salon owner might be the one who set up the base and maintains it as a piece of software.
And then the stylists would be the ones who would actually use it to do their work to say that they’ve completed a appointment, for example, or maybe they have some availability coming up in a couple of days and would be open to appointments or whatever. And so there’s that kind of separation between making it accessible for both groups of people. And we do that in a number of ways. So for example, you can create views that are focused on, so you’ve got a table of data and then a view will show you only a subset of that data. So only a certain set of records or a certain set of fields or whatever. So that’s sort of just one example of the way that a view might be something that a user of the base uses, which is a much simpler thing that they can interact.
At the same time there’s also exactly like you say, people who have different levels of sophistication with building applications. And I would say that I don’t know in detail, but we are certainly targeting more people who are not programmers, or maybe only do a bit of programming and who are more sort of, but still need to build software basically. So I think it’s a different, not entirely different for sure, but mostly different groups. So it might be, it’s quite often people who’ve used spreadsheets to run their work in the past. And so there’s a transition there, but I think, because Airtable is better suited to the things that people are using spreadsheets for in these cases, which is basically to kind of keep track of data that has a schemer, then that transition. There’s a little bit of stuff to do there, but basically that’s not the tough problem.
I think the toughest problem is helping people learn to create schemers that are powerful for them and helping people learn to maintain those schemers. And I think the other problem that I think about a lot as somebody who works at Airtable is how you help someone understand what their processes actually are. So let’s say again take the example of a hairstylist. They might, let’s say they keeping an appointment book at the moment and that’s how they’ve been running their salon so far and that’s been working fine. But if they started to build a base, then there’s a big question of how do they translate their processes and which processes should they translate? Is there a bone on Airtable to help them build better processes for running things? So for example, let’s say that so far, if they’ve had some, let’s say they decide to open the salon on a Saturday or something like that to see which of their stylists are going to be available to take appointments that day.
Maybe they just ring round each of their stars and say, “Oh, hey, can you do Saturday?” Then they figure out what the hours are and so on. And that’s, let’s say that happens pretty frequently then would they even recognise that as a process that could be made better in Airtable? I don’t know. It depends some would some wouldn’t and maybe Airtable could help them recognise that and help them build a better, more efficient process that doesn’t involve them ringing everyone up. So, I think we’re sort of trying to push on all of these different fronts at the same time, being accessible for people who want to build the base ,accessible for people who want to use the base, and also helping people build better processes for their work.
That is fascinating because that is an institutional example of what you ran into with Isla and Code Lauren. You have this thing that you want to build and in going through it, you discover there’s actually this problem outside of the scope of that thing, that solving that problem out there would be helpful for the thing that I’m trying to build or that the problem is broader than what I’ve scoped my focus on.
Yeah. That’s exactly it. And we have a number of tools for solving that problem. So one example is that for our larger customers, we have implementation specialists who are super-duper Airtable experts, and they will help the customer design the processes that they want to put into their base. And so that’s like, you can imagine they’re almost like a consultant who is helping someone produce better processes for their work. Then within the tool itself, then we can help customers with things like having templates, for example, that target certain niches. So for example, there’s like a sales management template, or there’s like a, I don’t know, film shoot management template and things like that. That obviously are ways for us to tacitly convey these sort of hopefully useful approaches to running this type of work.
But it’s, I mean, it really like it’s so, I find it so fascinating because these problems kind of run all the way up and down the stack that we need to help customers with. So all the way up to something quite high level like that, like modelling a 10 step process that involves a bunch of complex steps in a table, kind of right at the top level, all the way down to helping a customer understand that if they’ve created a table for January, like their January sales or something like that, it’s probably not the right move to create another table for their February sales. That actually it’s a better move to create a view that will filter down what a single table of sales to show only January or only February or whatever. And so like right down to that sort of just kind of data modelling problems that we need that we also try to help customers with.
That’s fun. Like I can imagine that building a product like Airtable and running into those kind of challenges are just really invigorating for the people whose job it is to try and design their way out of them. I …
Yeah. I think that’s, that’s very cool.
We talked a little bit about Code Lauren having a virtual machine that you built in addition to the language, in addition to the editor interface, there’s a lot to this project and that lets you have this little feature up at the top that you talked about a little bit, where there are little buttons for sort of rewind, step back, step forward, and play that let you incrementally move through the execution of the code that you’ve written. And at each step, when you step back or step forward, it highlights in a little box the fragment of the code that is currently being evaluated, see. You can sort of step through like, if you’re calling a function and doing something with the return value and you are stepping forward through that, it would first highlight each argument to the function and then it would highlight the functions name to represent that the functions being looked up and then it would highlight, does it highlight the whole, the name and the arguments or just the parentheses for invocation? I can’t remember.
It will highlight the function name itself to look up the definition of the function and then it will finally highlight the parens when it actually gets invoked.
So you have to click each one of those in the right order. And it is not like, I did some computer science at university, I will say that. I did at one point know the order that all of these things were supposed to be evaluated in and playing this game revealed to me that I don’t know shit, like I did so badly at this. And it that’s interesting because, you don’t necessarily need to know that to be a programmer who goes through their programming life building programmes, but it’s one of those little things that’s going on behind the scenes that at least I found very surprising.
And going a little further back in your JS Conf talk about Isla, you had a little section of that talk where you talk about syntax and sort of going into that section and coming out of this experience with the game Step, my sort of, if you cornered me in an alley and said, “Hey, held a gun up and said, define syntax in the context of programming” I would sort of blurt out something about it being in a text-based language.
It’s just the little bits of, it’s the punctuation basically, or it’s the fact that, var is a statement and so it doesn’t have a return value. It’s not an expression, that kind of thing like that syntax is just basically like, it’s not decorative, it’s very functional, but it’s separate from the actual meaning of your code that it’s sort of like maybe conveniences for the sake of the compiler is how I might look at it. But in fact, in your talk, you described syntax as being the order that things happen. That that’s what it is. And so I went and I looked up the etymology of syntax and it literally means to put in order the syn part of syntax means together. And the tax part comes from the same root as the word tactics.
And that, I wrote this, this question out in advance before you saying that you might’ve studied English, if you didn’t study computer science. And so I was kind of in my head going, oh, interesting. Is that fair to say that, that you are very, very fond of the textyness of programming. And is that something that you’ve thought much about, or is that something that you’ve just kind of happened upon sort of naturally?
Yeah. That this is something that I’ve thought about before, it’s interesting you bring it up. I think the parts of programming that I’m good at are often linguistic. And I guess I distinguished the linguistic parts from maybe things like that. You might say the graphical parts or the, graphical by which, I mean, the graphical, a graph for example, or like data structures or whatever, because it feels like writing prose to me writing code. So I really enjoy that part of it. Because it’s like just cool stuff, like creating a closure or something like that.
Let’s say with a function that then lets you, say you create a function that returns a function and then you can access the closed overstate later when you run the inner returned function, that feels kind of very like reading a novel or something like that to me, where you can kind of create a structural link in the text from something that had happened earlier to the current moment that the person is reading. Like I said, it feels very linguistic to me. And so I really enjoy those parts a lot.
Yeah. And it’s something that catches me by surprise, just because I’ve not done a very good job of hiding the fact that I loathe the text of programming. And so to immerse myself in your work the way that I did to prepare for this interview and to discover so many ways in which it feels to me, like there’s a fondness there that you have for it that kind of seeps through in your work. Perhaps that’s the thing that is just needed to compel one to create a programming language cause you’ve helped with … So I don’t know this off hand, how many programming languages do you think you’ve made so far? How many different ones?
They’re all about trying to get at this core of how stuff works. So Step, when you click on it, I still get it wrong sometimes. So I actually went back it to a few weeks ago, coincidentally, and try to … And I messed up in several places. I could figure out why I’d messed up, “Oh yeah, of course you do that bit first.” But I think I really enjoy how precise one needs to be to get it absolutely right. All of these projects, like I say, are born out of a desire to not be confused by things, which is basically replace a fear, that things that I don’t understand make me fearful that I won’t be able to, I don’t know, might make good stuff using code or that there’ll be a bug that I can’t fix or something like that.
There’s a sort of certain sense of unease that certainly comes over me. It’s like, oh, why that’s weird. Why I thought the only saved stuff when you committed, that’s weird. You don’t really trust it and you don’t really remember it that well. Whereas by understanding what’s going on in the objects directory inside the docket directory, then I was able to understand why that was true, that git add would save a file for sort of notionally forever. And that is like gives you a feeling of, like I say, understanding, and it’s also sort of cool to, oh, that’s how that works. That’s pretty neat.
Through all of these projects, including Gitlet, Code Lauren and, and Step curse and Isla, they sort of, they all kind of, they play on this idea that when you’re working with a language like C in a very traditional editor environment, you kind of have to play compiler in your head. Like you sort of have to be familiar enough with the syntax of the language, which means the order that things go in. You have to know about any quirks of the execution environment. You have to know things about if you’re writing across multiple files, the way that they’re going to be brought together and what order they’re going to, they’re going to run in. And that’s something you can hose yourself with.
There’s all of these details that go beyond just the behaviour that you’re specifying, that you need to understand comprehensively in order to confidently build a project. And it feels like a lot of your … A lot of the tools and games and things that you’ve built over the years are trying to take some of the mechanical aspects of compilation and either play with them or surface them, or bring them out of obscurity and out of mechanisation and make them intuitive or make them interactive in a certain way. And I find that really delightful as a sort of an overarching theme of your work. And I was wondering sort of now, and this a little bit like what we talked about earlier with game maker, but if you were to keep going with that theme of getting people out of having to play compiler in their head or surfacing what the compiler is doing, where do you think you would go with that idea in future projects?
It’s all about trying to make it more accessible, trying to make it easier for people to build stuff or not be afraid of computers also, and the place I’m at now and have been for the last year sort of more recently, I guess, is sort of like I’ve touched on before getting away from writing textual code and towards a place that you still need to define programmes. So, which is to say, to define algorithms, I guess it’s how I think of it, but you don’t necessarily need to write code to define those algorithms. Trying to move towards a place where that’s not necessary which I think is, would hopefully it makes it even more accessible because taking code mostly out of the equation, I think lowers the barrier quite a lot, certainly in terms of perceive people’s perception and the fear that comes from that and also from natural challenge of it.
And so I think it gets, I don’t think there’s a way to get completely away from code forever for everyone, but I found a useful north star more recently to be no programming ever. And it’s like obviously you probably going to fail at that, but it’s a useful mental discipline. So how can I do this without programming? Because it seems highly likely if we’re still writing code, most of the software is being created by people writing code in a hundred years, than I will be very surprised. And so it seems helpful to try and run towards in that direction as quickly as possible now rather than sort of waiting for it to come in the future. And I think that part of that is about helping hopefully making things more accessible for other people. So that’s like working on Airtable for example.
And part of it’s about actually moving myself away from writing code. I mean, obviously I write code for my living at the moment, but I think that I put whatever it is, I don’t know, 20 years into learning to be a good programmer. I’m still learning tonnes, still tonnes to learn. But I think I’m a solid programmer now, but I find myself just wanting to get away from writing code more and more. So game makers, one example of that, another example is trying to …
GameMaker’s one example of that. Another example is trying to build prototype interfaces without writing any code. So by using those drawing on paper, or by using UI prototyping tools like Origami is pretty coo, or Axia is amazing. I have no idea why. And so I feel like the last six months has been me discovering tools that are 20 years old, but are totally amazing, and that I wish I’d discovered a really long time ago.
I know that feeling.
Because they’re just so much faster. It’s just like, oh God. And so essentially, I don’t want to say that programming sucks, because it obviously doesn’t. Obviously I love programming, but I do kind of find programming very slow, even though I’m quite good at it, and sort of very frustrating. And there’s just so much more joy that comes from not needing to write everything in code and being able to just move whatever it is that you’re making forwards so much faster than if you have to write code to do it.
There are some people probably in the audience who are going to get a little bit hung up on your use of the word programming. And I just wanted to give you the opportunity to sort of define what you see as the difference between coding and programming, and maybe something like using HyperCard, or maybe something like using a drawing app or something like that. Because there’s some people out there for whom drawing a picture in Photoshop is programming. They have that broad of a definition of what it means to be programming. And then there’s some people where the line is HTML. Or there’s some people where if you are writing in a dynamic language, you’re not programming, you’re scripting. So, in that you have made a very universal statement, no programming ever, I think it would be interesting to hear where you put the boundary around what it means to do programming in light of that.
Yeah, no, thanks for letting me clarify that. That’s absolutely right. I think that the rough definition that I’m using here is textual programming, as kind of a rough rule of thumb definition. I think there are some things that aren’t textual, but are still very close to it. So like Scratch, for example, you’re not writing much text, but it still feels very textual. It’s pretty close really, and so I would include things like that. I think, if one defines programming as creating something that defines an algorithm, then I think that could be helpful. And so in that lens, then writing a Word document is programming because you have types and characters in that dictate how the computer should render some text on the screen. And so that feels technically correct, but not very useful.
And so I wouldn’t go that far myself. I think what’s useful is to have a useful definition. And so I think of it as being able to define the behaviour of software without writing textual code, is a rough rule of thumb with a bunch of caveats basically. And so a good example of that, I think, is the example I gave of GameMaker of defining the sprite where you don’t have to write any code to show this set of images, go together into a sprite. That was the first part of the algorithm. The second part is that they should be shown in this order that I’ve dictated by dragging them around. And the third part is that the image should change at this speed, which you could also define graphically and so on. And so that’s definitely an algorithm that you’ve created. And so I’m not trying to get away from that type of programme. And I think that type of programming is really great.
And even if it’s not necessarily an algorithm that you’ve created like a new kind of search or something like that, you’re still doing algorithmic thinking when you’re doing it. Much in the same way you’re doing algorithmic thinking when you write a recipe or something like that.
Exactly right. It still seems to use the same parts of the brain. But I think that rearranging those images inside GameMaker to put them in the order that you want them to be displayed, yeah, it’s creating an algorithm and it requires algorithmic thinking, but I think sometimes people get confused and say, “Oh, well, if you’re doing algorithmic thinking, then you must be capable of programming.” And it’s like, yes, that’s true. But it’s so much easier to not do it in code, both for the most amazing programme, both for John Carmack and for somebody who’s just writing their first line of code. I think there are easier and harder ways of creating algorithms, and I guess I’m trying to push towards the easier ones.
And just, because it popped in my head, I bet an even harder interface than many of the programming languages that have survived to this day would be something like very complicated legal contracts, because those are algorithms that have algorithmic thinking behind them. And the interface for them is just horrendous. So it’s not as bad as it could be, but it certainly could be a lot better.
So my last question, I thought, would be a nice light-hearted kind of thing. And that is something, we’ve talked a little bit about this already, but I want to go a little deeper into this because I’m that kind of nerd. We’re both gamers. And I personally think that games have a really big role to play in both the future of programming directly, but also just in the space of thinking about what the future of programming might look like.
There’s a lot that the people making these sort of programming tools like you and I are working on, and then people listening to this show are working on, things that we could learn from looking at video games. Both video games as works of art, that inside them contain interesting ideas that we could borrow for programming, but also video games as a culture and an industry that produces software, and is wrestling with a lot of the same problems that we are wrestling with. And so I wanted to just take a moment to ask you if there are any favourite games that you have that are fun to look at through the lens of futuristic programming, or through the lens of people working on this kind of dynamic software. What do you see as interesting things that we could point to over in the games world that it might be worth us paying more attention to?
Part of my answer is around if you like the non-interface, non-expression related parts of games. So for example, the way a community conform around something like Minecraft, for example. I think future of programming people, including me for sure, can learn a lot from that. So it’s like, my understanding is super supportive community, very exploratory, lots of different people with different objectives all working in the same area, amazing community of practise around YouTube, and friendship groups sharing ideas and sharing techniques and so on. And so I think there’s a tonne of just community related stuff that we can all learn from specific video games. So that’s one part of it.
Another part of it, I think we can learn from, is that games are extremely good at conveying complex systems to people in a very short amount of time, in a very clear way. And so some of this stuff is like next level. So for example, a game like The Witness by Johnson Blow, which is, if you’re not familiar, is kind of a puzzle games. It’s 3d. You walk around solving puzzles that are on panels placed around the world. It gets more interesting and sophisticated than that. But it’s a hundred percent non-linguistic. I think there’s maybe like an X shown on the screen to say, press the X button on your PlayStation controller. But beyond that, it teaches you an extremely complex, I almost want to say language, that you need to solve these puzzles without any words. And it does it in this amazingly ramped way. It’s so cool. And so I think there’s a lot of stuff about communication and about designing systems that can be learned, and that we can learn from various games too.
I think there’s also things to be learned from really great UIs. So the example that springs to mind, when you play Mario, the original Mario one on the Nintendo, the way jumping feels as Mario is just so precisely tuned that like you hold down the button for longer and you go higher, you hold it down for the maximum amount of time and you float just a little bit, but not that much. There’s this kind of beautiful, exponential ramp up to how you move when you run, and all sorts of things that so much complexity in just a few buttons that I think it’s an incredibly expressive interface that they’ve created, and it’s just for jumping. And so I think we can all learn a lot from there.
One of my favourite genres of YouTube videos is people explaining how games are designed. So there’s a really cool one explaining how the first level of Mario works, where, for example, it almost forces you to trigger the first mushroom to jump out of the block when you jump into it, and then the mushroom is sent straight at you. So you’re like, “Oh no, it must be bad. It’s going to hurt me.” And it’s tough to avoid. And so more likely you hit it and then you end up growing and you discover mushrooms in this context are good. And just like all of these nonverbal ways of communicating things, I think, we can learn a lot about, because essentially games are learning systems, and they’re just so much better than programming systems are. It’s amazing. So I think, yeah, we can learn so much.
I love those examples. And yeah, just to fill in a bit of colour on the last two, in The Witness, an example of the language that the game teaches you non-verbally is that the panels that have the puzzles on them that are scattered around the world, they challenge you to draw a line sort of through a maze, from one corner to another corner or one side to another side, and you can draw a line that can’t intersect itself. You can figure that out very easily because you start tracing the line around the panel, and the game’s interface sort of laser focuses you on this activity. It’s the sort of thing where the game does a tremendously good job of getting you into this space where the only control that you have is, I’m moving my mouse or I’m moving my joystick, and this line is being drawn, and you can play with it and explore it and discover, okay, it can’t intersect itself. There’s this little pulsing thing that if I draw the line up to it, lets me complete the puzzle. And then that’s the most basic thing.
And then it builds on top of that. In some of the puzzles, there might be three little dots in a little square of the grid that is the maze that you’re drawing through. And you’ll figure it out through playing with it, that if you draw a line that goes past those three little dots, it won’t let you complete the puzzle. And based on the arrangement of the maze, it might kind of force you to draw a line around those three dots in a certain way. And it will do that again and again, and show you that every time you are drawing a line around those three dots and the puzzle is accepting your solution, it’s because the line that you’ve drawn goes around those three dots on three sides.
And then it will show you two dots and you have to figure out, okay, if I surround those two dots on two sides, then it lets me complete the puzzle. And so it introduces this mechanic of this symbol will have a variable number to it that is the number of sides around which that square of the grid needs to have your line go around it. And then there’ll be another symbol. And some of the symbols in the game look like Tetris pieces. They’re these little Tetris piece symbols, like a little T or a little L-shape or that sort of thing. And the game will teach you, non-verbally, that to complete a puzzle that contains one of those symbols, you have to enclose that symbol in a path that has the same shape as that Tetris-like piece. And they’re not tetranamos. They might have five segments or three segments or whatever.
But it introduces something like 10 or 12 of these different mechanics, and then puts them in all sorts of different combinations, where by the end of the game, you are like this master at this language of drawing lines that are constrained by all of these weird symbols, and the symbols are sort of forcing you into a specific course of action and you can do it incredibly quickly. And one of the very late game challenges is you have something like a minute and a half to solve this whole series of these puzzles that are incredibly complex and that are randomised every time you do them, because most of the puzzles in the game are the same every time you do them. And these puzzles at the end game are randomised and time-limited, and there’s a very juicy reward for completing that.
But it’s this masterful way of showing that the presence of an interactive system that you can have a dialogue with, will let you have a learning experience that can be radically different from what we’re used to when that learning experience is going through a medium that is human language, where it’s a direct teacher/student kind of relationship, and they’re talking about some subject or when it’s a textbook or something like that. Here is an environment where through a little bit of trial and error, but mostly just very carefully focused design, you can have a learning experience that teaches you something very complex. And of course, that complex thing was designed with the fact that it needed to be taught in mind.
And so it’s not necessarily like you could apply this to teach an arbitrary thing, but I think there’s a really good spectrum there where, from games we can see that there are so many different things that can be taught in ways that are nonverbal and in ways that are playful. That if you are looking for features to design into your futuristic programming environment, there’s a great big menu of choices you can choose from for interactions or for behaviours or for dynamic relationships between things, that we can see, and we’ve proven are very easy to teach intuitively, or that there exists a way to do a really good job of teaching it intuitively and through play rather than didactically where you have to explain it with a pop-up or a manual or something like that. So I loved that example.
Yeah. I always find it interesting what games choose to essentially give you, to just explain to you, or to give you for free, and what they want you to figure out on your own. Because clearly, if they just left you with nothing, then there would be a lot of frustration, which would be not that much fun. But at the same time, make everything really easy, that’s not that fun either. And I think a really strong example of that, that comes to mind, is a game called Into The Breach, which I also loved, which is essentially you’ve got a few little tanks that you control and you have to fight off some other tanks in a little eight by eight grid, sort of isometric view. And they spent apparently a couple of years, they kind of got the game pretty much nailed down, but they spent a couple of years working on the UI, working out how to explain the concepts to the player in a way that wouldn’t be frustrating.
So for example, they’ll make it very clear that when you have one of your tanks fire, you essentially say, “Oh, I want my tank to fire over here.” And they will show you the immediate consequences of that. So the shell will hit the enemy thing, and then that enemy will get flung back by two squares, let’s say, and then they’ll get flung into a building, which will destroy that building. And so they’ve obviously given you quite a lot for free there.
They’re showing it to you in advance of it happening.
So you get to have a preview of what is about to happen if you decide to take that action.
And what I find interesting there is like, okay, they’re kind of giving you that for free, so that they’re certainly saving you from some mistakes that you could be making, but clearly what they think is interesting is the slightly longer term ramifications that you do have to figure out for yourself. So there’s this notion of what should be told to the player or the user or whatever, that would be an emission if not told. And then what shouldn’t be told that would be like giving the game away, or maybe taking the fun out of it, or taking away that fun of exploration, whatever it is. I do not know how they decided what to put in the we’ll tell you this, and what to put in the, you have to figure this one out for yourself categories. And I think about it quite a lot. I find it fascinating.
That’s another game that I loved and another great example. And I like, specifically, that you talked about that they made the game and they made the engine and the graphics and all that sort of thing, and had this game playable, and then spent a tremendously long time tuning the UI of it. And I think that, that’s a valuable lesson, because it’s not like games are different from the kind of software tools we’re building in terms of the economics of it necessarily, or in terms of the process that you have to go through to create a thing. Game developers want to make their game and they want to ship it and they want to get it out there into the world. It’s not like all games have developers who spend that long working on the interface. That’s very unusual. A lot of games, they’ll spend the most time working on the graphics, and the world, and on populating it, and fleshing out the story, and if it’s a big AAA game, recording all the dialogue, and doing actual logic for what characters will do, and all the things that happen as you progress through the game.
There’s a tonne of things that take a tonne of work and designing the UI is often not one of them. And so what makes Into The Breach such a wonderful example is that they did take that extra time that it took to just polish the UI of this thing into something very, very special and very unique. And that aspect of it is what made this game remarkable. And personally, at least, I think it’s what made it into such a big success and such a big hit commercially, was that by taking that much time to really tune the UI. For one, they took a game style, a genre, that is normally very niche and they made it approachable to a broader audience, but they also exposed a lot of the beauty of that genre to people who previously wouldn’t know that they would be interested in that kind of a game. And they did have the luxury of getting to take that time because their previous game was sort of a surprise hit.
Yeah, it’s the sort of thing where, if you are a Mary Rose Cook and you are working on your Aila, or you’re an Ivan, and you’re working on my past projects that nobody’s heard about or cares about, it’s absolutely worth it to live with something for a long time, and to invest that energy and that effort in polish, and especially polish on the side of the user interface and on making it, like you said, Mary, deciding what to expose and what to obscure. Or in the case that they did, very specifically, they show so many things that other games hide. And other games hide them because they think that the fun will come from seeing the dominoes all fall down as a surprise.
And what these developers of Into The Breach found is, no, you can show, you can preview pretty much everything that’s going to happen. And at that point, from somebody who hasn’t played it, you’ll get the objection, well at that point, why even play the game? If I sit down and I immediately see, oh, this is everything that’s going to happen, where’s the fun in that? And they show that it can be more fun when you give people more information, because that lets you constrain the actions that the player can take in more interesting ways, more severely than you could constrain them if you didn’t show them everything.
And another example of that, that I love very much, is a game by Zack Gauge called Good Sudoku, which is a game that Zack made. And Zack does this for a lot of games. He did this with solitaire, with a lot of things, where he will make a game that is based on a very, very standard game that everybody knows and everybody’s bored to death of like solitaire or like Sudoku or something like that, where you think, “Well, really this space has been pretty well explored. What more is there to do with it?” And what Zack’s approach to the game is, is to build a very powerful user interface that lets you play the game at a higher level than the normal interfaces for that game would allow.
So most Sudoku apps will replicate the experience of playing a Sudoku on paper. And Good Sudoku will do things like it will automate all your annotations that say like, “Well, this cell can’t be a three and that cell has to be a five or a seven or whatever.” And it fills in all of those notes for you and saves you a whole tonne of manual work. And you’d think, well, doesn’t that just take the joy out of it? And it’s no, because what that enables you to do is to, A, very quickly and through a very, very good user interface that he developed for this, learn more advanced techniques for Sudoku that you might not have even known existed if you were just playing Sudoku casually.
Like if a row, which has to contain all the digits from one to nine, has two cells that are empty, you know exactly what those two cells are. They’re the only two numbers that are missing from the digits one to nine. And that’s a technique that most people might not realise unless they play Sudoku very, very seriously and get to a high level, that his interface exposes immediately. And so he can let you play Sudokus that are way, way, way harder than you’d normally ever be able to play, because the interface is so good. And in so doing, learn all these new, more advanced techniques, and see some of the beauty that that game has behind it, that is normally obscured by that interface.
And so, yeah, I think programming, we could do a tonne of that in so many different areas. I think, maybe Elms error messages are an easy thing to point to as a programming project where they put in a tremendous amount of effort to say what are some things that we can expose to the user that would normally be somewhat hidden that they’d have to do a lot of work to learn for themselves. But there’s probably many, many, many more things that we could do like that if we were each individually or collectively able to take that time and really spend it living with our interfaces and really honing the heck out of them.
Yeah, I completely agree. And I think that often hand-in-hand with the attitude of, “Oh, you’re taking the work out of it or you’re taking the fun out of it, or something like that, is this attitude that comes up in programming quite a lot of if you can’t simulate what the computer’s doing in your mind, then you’re not a proper programmer or something like that. And it’s this kind of silly, I don’t know, it’s almost sort of like a moralistic viewpoint to take that you should be doing the grunt work. You should understand everything all the way down to the ones and zeros or whatever, in order to be able to do anything. And I think it’s just ridiculous.
Agreed. Well, Mary, thank you very much for coming on the podcast, and for taking this tour through the history of your projects, and through my torment in playing with one of them and my delight in playing with all the other ones, and sharing all the interesting thoughts that you’ve built up along the way.
Yeah. Thanks so much for having me, Ivan. It’s genuinely a huge honour to be able to talk to you. So thanks so much.