In this episode, I’ll be talking to Toby Schachman, who many of you are surely familiar with thanks to an incredible string of projects he’s released over the past decade, including Recursive Drawing back in 2012, Apparatus in 2015, and most recently Cuttle which opened to the public this past week. All of these projects superficially appear to be graphics editors, but by interacting with them you actually create a program that generates graphics. Their interfaces are wildly different from both traditional programming tools and traditional graphics apps. If you are not familiar with these projects, I strongly recommend that you actually go and play them (they all run in the browser), or watch the Strange Loop talk where Toby demos Apparatus and explains the thinking behind it.
Cuttle’s team is Toby plus:
Glide are building a batteries-included app development platform where the database is Google Sheets. If you’re excited about making end-user software development a reality, go to glideapps.com/jobs and apply to join their team.
Replit are a long-time sponsor of the Future of Coding podcast transcript. They are building an online REPL with over 50 languages, where you can do everything from deploying web servers, developing games, to training ML models — all driven by the REPL.
The transcript is challenging to produce. Please know that it is unrefined at best and inaccurate at worst. But even in such rough shape, it’s valuable to many people in our community. If you’ve got the time and energy, we would love help cleaning it up.
(Preamble, continued from above.) Recursive Drawing, which was (AFAIK) Toby’s first dynamic drawing tool, was his thesis project at ITP in 2012. It’s a program that takes a very common pattern in graphic software, where typically you can draw a shape and then you can wrap that shape up as a reusable component and then make multiple copies of that shape. It sort of wonders about and then answers the question, “What would happen if you could take one of your defined shapes and put a copy of that shape inside itself?” So it’s this editing environment he’s created for creating these sorts of recursive shapes where a shape contains itself. You get these beautiful fractal-like patterns from it. In the ITP presentation he gives about it, he asks a bunch of interesting questions like, “Oh, is it possible to use this to create a drawing that represents the Fibonacci sequence?” Or in the actual a webpage you can go to for this project, there are a number of interesting prompts that sort of challenge you for different things you can try to create within this model, like the Koch snowflake, the Sierpinski triangle, or other things like that. It’s a very interesting approach to creating graphics, and to visualizing something that as programmers we work with all the time, but don’t necessarily get to work with in a very immediate, tangible, physical way.
In that sense, it’s a fascinating bit of work that he later built on in a number of other smaller projects, but that eventually culminated in a tool called Apparatus. This one is truly something special. If you have not seen this project, I implore you to go take a look at the Strange Loop presentation that Toby gives about this project and to actually play with the editor in your browser. It’s a direct manipulation drawing environment that allows you to very interactively set up these relationships between different aspects of your drawing. So you could take the rotation or the scale or the position of one shape and feed those into another shape, in such a way that the relationship between those things is always live and is always updated, that your manipulations of the one shape can then, in turn, manipulate the other shape.
This lets you do just some absolutely incredible things. For instance, some of the demos that Toby gives show him building his own UI elements for then, in turn, making new drawings. So it’s this environment where you can ladder your way up from the primitives that the environment gives you to a more sophisticated, more robust, more specialized environment for making the kind of dynamic drawings that you want to make.
Toby’s goal with Apparatus was to create an environment that was very specialized and very focused on explorable explanations or dynamic drawings — these sorts of cool pieces of multimedia that you see created by Bret Victor in some of his essays, like Up and Down the Ladder of Abstraction, or created by Nicky Case and some of their projects that teach you how to do different graphics techniques or that explore different simulation models in a very dynamic playful way. Some of my favourite examples of explorable explanations are those created by Amit Patel on his website Red Blob Games. He covers a lot of algorithms that are very common in game development and teaches those algorithms in a very visual, very interactive, very playful way.
Nowadays, Toby has actually taken a lot of the ideas that were in Apparatus and is working on a new tool that is sort of like a spiritual successor to Apparatus, called Cuttle. Whereas Apparatus was a research prototype that was designed to help you make those explorable, explanation style, interactive diagrams, Cuttle is focused on creating CAD diagrams or creating CNC instructions that you would then use with a 3D printer, or with a laser cutter, or some other tool, to take something that you’ve designed in a digital tool, but then actually manifest it into the physical world.
Because, in addition to Toby’s programming projects, he’s also done a lot of artwork over the last decade plus. That artwork explores a lot of very interesting themes that also relate to the programming work that he’s done. We’ll get into that a little bit in the interview, but this new project Cuttle is sort of a permutation of Apparatus with a very different focus. That difference in focus has led to a number of interesting design decisions that kind of contrast with what Apparatus is. So we’ll talk a lot about that in the interview as well.
Some of the other themes of Toby’s work that I think it would be worthwhile for me to just touch on here before we get into the actual discussion are that, much of his work is about wanting to get away from writing lines of text that micromanage what the computer does in a sort of a “Do this, then do this, then do this”, sort of way. Upon reflecting on programming requiring that approach, Toby has noticed that programming self-selects for certain kinds of people, typically logical sequential thinkers. The fact that that is what programming, as practised, sort of self-selects for, those people who then go on to make new approaches to programming, tend to reinforce those modes of thought. So it’s that self-selection forms a kind of a feedback cycle and so we get trapped in this mode of thinking that is very much about accounting and symbolic thinking, rather than thinking spatially and working with models and diagrams and those other modes of thinking.
Just because it’s my own personal area of interest, I found this remark that Toby made in one of his early presentations very interesting in that Toby, when looking at approaches to visual programming, people often point at things like patcher languages like Max/MSP, or VVV, Origami, and Quartz Composer, those sorts of node and wire based languages and similar. Those don’t cut it in Toby’s view because while they do give you continuous feedback, which is something that is, I think, an essential property of all of Toby’s work, but also the work that he was inspired by… While those patch languages do give you that continuous feedback that you would want from an artist tool, they aren’t actually spatial because the spatial aspect that you would want to be working with if you are trying to produce a spatial result, like some sort of interactive graphic or something like that, the spatial aspect of the work that you’re trying to produce is in no way mirrored by the spatial aspect of the code that you’re working with.
So as an example of that in one of his presentations, he shows a Pure Data patch that somebody made to control a human-shaped robot and the patch has all of its nodes arranged so that you can kind of see the shape of a person; they’ve got two hands and two legs and a torso and a head. The nodes are kind of grouped together and the wires between them kind of make it look like a stick figure, but there’s no inherent relationship between the arrangement of the code and the arrangement of the visual geometric result that is being produced. So that is what makes Toby’s work so interesting to an outside observer like me, is that he really embraced this idea of making the programming experience mirror in its spatial character, the geometric result that you’re producing. He’s done just a number of very interesting things to tie those together.
You’ll notice that I’m offering more context for his work than I’ve normally offered in my other interviews. That’s because, on this interview, I tried something a little bit different. Rather than just approaching it as a summary of Toby’s work, and reviewing the decisions that he made, and then using the interview as an opportunity for him to repeat ideas that he’d previously thought about, I approached this interview as an opportunity for me to ask Toby some questions about his thinking in these projects that would hopefully motivate some new reflections and some new thoughts and tease out some of the aspects of his work that were a little bit further below the surface.
So this interview, it’s a weird one. But I think it points to an interesting direction that this show could go in terms of how it handles exploring the work that we’re all doing in this space, and how to have a show that reflects on that work, and shares that reflection with a broader range of people so that they can incorporate that thinking into their own work. So it’s less about the particulars of what Toby’s done and more, hopefully, focused on themes and broad ideas and modes of thinking and general approaches, and hopefully, that comes through. So without further ado, here is my interview with Toby Schachman.
Do you remember what you imagined that programming was like back before you actually learned how to program?
I don’t think I ever knew that programming was its own discipline when I started programming. I played Nintendo games when I was a kid, and me and my friends would talk about like, “Oh, it would be cool if there was a game like this,” and we’d draw the levels and that kind of thing. Then I got into HyperCard, which no one told me was a programming thing. I just used it to draw pictures and make little hypermedia adventures and then started making more elaborate games with it and with my friends. So like, little SIM-like games or fighting games where you have… Because you can only move buttons around, so everything was represented as buttons that are moving around the screen, but you can… It’s a full programming environment so, I’m pretty sure that I started programming before I knew that programming was a term of art.
Yeah, and then at some point I’m sure you learned, okay, computer science is a thing and compilers are a thing and there’s this whole other world of non-interactive, non-direct manipulation programming. Do you remember at all what it felt like to go through that transition?
I don’t know when I thought about computer science as a thing; probably not until I was in college. I did HyperCard as a kid and then I got into web development just because the web was a new thing and it was fun to make webpages. I did a bunch of that in middle school and high school. Certainly by high school, I understood that you could make a living making things on the computer, because me and my friends were sort of making a living making things on the computer. Then I went to MIT and I remember wanting to go to MIT because I had seen cool research projects. I think I had seen stuff from lifelong kindergarten type groups and media lab type things and I was like, “Oh, that’s really cool.” But computer science, as a thing, probably wasn’t something that I ever really thought about until I was actually taking classes at MIT.
By that point, it’s like you’d gone on a gradual ramp up towards seeing what that style of programming, like what I would call the predominant style of programming is like, so I guess it didn’t feel particularly shocking or anything like that then.
I guess not. It seemed like everything else in the adult world, a little bit more complicated than it needed to be.
That encompasses so many feelings right there. So that’s my sort of silly, warmup question that I like, because it takes us briefly into the backstory of the person I’m interviewing, which I feel like, as an interviewer, you’re kind of expected to do that kind of thing, sort of things. So I’m trying that hat on, but I like it because it always helps us understand a little bit about what the person I’m interviewing, what their feelings about what programming is happens to be or maybe what informed that. How that kind of plays into the bulk of what the interview is going to be about, which is the work that you’re doing now.
To actually get into the thick of it, what I want to spend most of the interview talking about is the design and the conceptual thinking that goes into your projects because a lot of your work has been in pursuit of certain goals. You’ve done a project pursuing a goal, you’ve gotten the project to the point where you can actually play with it, and then you revised the goal a little bit or you come up with a different idea and then you go a little further on that. There seems to be these common themes, but that you’re constantly revising your thinking and then doing a new project, and revising your thinking and doing a new project. So, I want to explore that arc and then also where you’re at now in that process.
But the place I want to start is with one of the things that’s been a really common element in all of your tool projects. Going back to Recursive Drawing was at least the first place I saw this, which is that always have a sidebar of components on the left and, unlike most graphics tools that I’ve used, where, if there’s this component, you can just click or double-click on the component and it spawns an instance of that component somewhere on the canvas. In your tools, clicking on the component does nothing and you actually have to grab the component and drag it out onto the canvas to create an instance of that component. I’m wondering if that decision came from something specific and what it is that you like about that UI convention.
Hmm. I don’t think it’s anything specific. I agree that that shows up a lot. I did that first in Recursive Drawing and then all the other things are variations on Recursive Drawing like you were saying. It just seemed the most natural thing for Recursive Drawing. I feel like Recursive Drawing is the most pure in terms of the gestures, but I don’t know if… I wouldn’t say that there’s anything like, I’m a fundamentalist about having a left sidebar that you can drag things out of. It doesn’t really scale, the idea of sidebars in general, it doesn’t really scale beyond a screen that has sides, that are within reach. So with Dynamicland stuff, for example, the idea of a sidebar, I don’t think it really makes a lot of sense. Yeah, I can’t say that I have any insight onto why that always is, other than it’s just a carryover from Recursive Drawing.
There’s a disconnect there between how you’re doing it and how, not everybody else does it, but what’s the normal. What I like about it, and maybe what made that approach make so much sense in Recursive Drawing, is that it forces you into this sort of direct manipulation paradigm quite deeply right off the bat. Whereas the idea of, oh, I clicked something over here and something spawns over there, is already a little bit of spooky action in direction. By having to actually drag it out, it makes it feel a little bit more like it’s an actual concrete thing. I think that that’s just one of those little touches that does a lot to communicate to somebody who’s new, coming to the program for the first time, a little bit about how to think in this program.
Well, you need to be able to edit the definition in addition to being able to instantiate the definition. So there needs to be… Clicking is usually like, “Oh, I want to edit the definition,” at least with Apparatus and Cuttle and all of the above.
But there was a version of Apparatus, I think, for the Strange Loop talk that you gave, where you had to click a little pencil icon to do the edit.
Oh, yeah. Hmm.
If I remember, clicking on the sidebar didn’t do anything, but I’m not sure. But it’s one of those things where it’s different from the norm, and I like that.
I think Canva does it that way though, where you have to drag things out. Canva is actually the norm, right? That’s probably the most popular.
I suppose, yeah.
Direct manipulation things.
Though that, to me, that feels like such a different world, I often just relegate it to the back of my mind, but you’re right, yeah.
Yeah, all those. It’s the same with Cuttle. A lot of people who talk and think about these things, think that 3D printers are the norm, but actually the Cricut is the norm. That’s a way bigger community than the entire, what people call the maker community, just all the people with the vinyl cutters.
I’m so outside of that community, I don’t know what the Cricut is.
The Cricut is a vinyl cutter, or it’s knife cutter, a digitally controlled knife cutter, and it’s about the size of a printer. It costs about a printer, like maybe $300 or something. They have an aisle at Michael’s or like any of these hobby art and craft stores. It’s much bigger than the whole 3D printing community, but for some reason, the maker thing tends to ignore this huge community. It’s a different demographic; it’s mostly women who are working with Cricuts. It’s mostly on Facebook. It’s a cool community, it’s much bigger and it’s much different, and it’s sort of more like Canva that it is like, I don’t know, Figma or something.
Mm-hmm (affirmative) I’m reminded of, what was the knitting social network? Was it Ravelry? Was that the one?
Yeah, that sounds right.
Yeah. Where it’s like they were this massive social network and they had this tiny team that were amazingly productive, but nobody in the tech sector was talking about them because they were a community of women knitting and crocheting and that sort of thing and sharing patterns. It’s just interesting to learn about these blind spots that we have for massive communities that are right next door to where we’re paying all of our attention.
Totally. Mm-hmm (affirmative).
Do you know if anybody from the Cricut world is using Cuttle or has found a place for your work in that space?
Yeah, there’s one of the more small time Cricut bloggers, but she’s fantastic, has been playing with Cuttle and giving us feedback. According to her, she’s, and she’s right. She’s a few years ahead of the rest of the Cricut community, so hopefully that pattern will continue.
Cool. So I watched your ITP thesis. In the QA session after that presentation, someone asked what it would take to make Recursive Drawing into a real programming tool. You said, and I quote, “The hard part is the user interface conventions, that’s the hard part. As you discover better interface conventions, that informs how you can evolve the interface.” I’m wondering now that you’re making Cuttle, does that still hold true? Is the user interface and the user interface conventions still the hard part?
Hmm. I don’t know. It’s hard to… I’m just trying to bring to my mind the Toby of that time, which was-
It was only like a decade ago.
… more than a 10 years ago.
Yeah. You should’ve seen my interview with Miller Puckette asking him about creating the original Max Patcher back in the mid eighties. That was definitely him going, “I don’t even… Where… That was the eighties; it was a long time ago.”
Yeah. What’s the hard part? I don’t know. What is the hard part these days? I don’t know if what I said was true or if I still believe it. What I believe the hard part is these days are, first of all, being able to know that there is a better solution is hard. So just knowing that a better way exists. This is something that Bret Victor is pretty good at. His talks are good at showing you, oh yeah, there is a better way, even if it’s not a fully fleshed-out thing. If you watch one of his talks, then you can see like, oh, there are better ways. We can be doing a better job. I feel like just knowing that a better solution exists is sort of the first hard part.
Then I think also… I’m learning, but also knew that, actually making a product is a hard part. You don’t have to make a product to make a research prototype. Research prototypes are great; people should make research prototypes. But if you want to make a product that’s this huge, 10 times more work than making a research prototype or more like, in terms of… It’s like HyperCard was successful. Not just because it was a great work of software art by, essentially, one person driving the creation of it, but also because there were a hundred people making the documentation and figuring out what the intro stacks were going to be, that you would use to get started with it. All that stuff is a big part of what makes something usable by a larger audience, which is then… You sort of need that usable by a larger audience if you want to drive the ecosystem forward I think.
It’s like the research prototypes do that also but, because of HyperCard, and then because of Flash, and now because of this new crop of your Figma’s and Notions and Airtable type things, that does also, I think, drive the ecosystem forward. So then, there are more conventions that you can just rely on and people will be able to get it. Just like the idea of being able to get the idea of having a component and then instances of that component, that’s not something that is perhaps as natural as like Wiziwig. What you see is what you get, which was, at the time, like a big innovation.
As these things get invented, it’s like genre, conventions, It’s like you can make a film now, and there are all these different genre conventions you can use, or certain kinds of shots that mean certain things, and the audience will just implicitly understand what that means. But if you were at the start of film, you couldn’t do that. You didn’t have that vocabulary.
Yeah. Like people thought the train was actually coming out of the screen, like that famous film that showed a shot of a train coming towards the camera.
Right. But even subtle things like if you see a shaky camera behind some trees looking at the girl or something, then it’s like, oh…
… trees looking at the girl or something, then it’s like, “Oh, there’s a stalker or something.” Right? That’s like a convention that you can do that in like three seconds of the movie, and then you’ve established all these things with the audience just from that one shot. All those things are little words in your vocabulary and the more rich that vocabulary is sort of, the more you can do with it. I mean, people obviously made great films back in the day before there were all those words, but the more words you have, the more sophisticated a film you can make. I think the same is sort of happening with software, there are more words that get developed and then the audience becomes familiar with those words, and then you can leverage that in the software you’re making. I guess kids just grow up with it, so it’s just natural to them.
There’s an interesting parallel here, because what you’re talking about is sort of in terms of the advancement of a culture broadly, whereas I’m also used to encountering this idea in the context of individual people. The example I’ll point to is that there are certain, very artful video games that are able to create a very powerful experience for the player. For example, a game like What Remains of Edith Finch, but those games, to be so impactful, they need to be very pure in their communication, and that purity usually comes at the cost of tutorialization. They don’t have a lot of UI, they don’t do a lot of handholding, and they kind of expect you to come to the game already being familiar with the language of gameplay. So things like, if you see a ladder, you can walk to the ladder, look up and walk forward to climb the ladder. They won’t spend a lot of time coaching you on that kind of stuff, because they just want to focus on the main thrust of the point of the game.
So there’s this interesting kind of deepening familiarity with the culture on a personal level that someone can go through in their life to gradually get to more profound experiences, or in the case of tooling, more sophisticated tooling. So I think that there might be something there, not just in the need for HyperCard to Flash, to tools like Cuttle, and Bret Victor’s work kind of on a continuum over the long arc of history, but also within a person’s life there might be… I think maybe that’s a little bit of the criticism that is levied at tools like Scratch, which is sort of they’re seen as not being similar enough to “real programming.” So people don’t want to use them as a learning tool.
So I think there’s a lot there that, as an industry and as a practise, we programmers are still trying to figure out like how do we communicate that this is a program that requires people to be familiar with a certain set of conventions. At least in my case, the thing I’m really interested in seeing is, how do we get away from the idea that there is only one universal set of conventions that everybody needs to be fluent in and that is Git, and the Terminal, and C Compiler, and text editor with syntax highlighting. How do we have a broader programming culture than that and have some subset of people who are able to go through a different personal progression of becoming more and more familiar with these alternative tools, with things like instantiation of a component where it’s, “Is this a copy or is this an instance?”
If it’s an instance, I know from past experience that, that means I can go to the source component and change the source component and the instance will inherit those changes versus a copy where it will not. That’s something that if you took somebody who’s a 3D graphics person, who’s using those tools, that idea is old hat because 3D programs, all of them have been doing that forever. But if you took somebody from the world of Photoshop, that’s not an idea that is very commonly presented and reinforced there. So it’s interesting to me to kind of identify what interface paradigms, people making these experimental programming tools can depend on and which ones we still need to have a lot of teaching around and a lot of handholding and that sort of thing.
Yeah, I like your example of the ladder in the game. I would say even more than an individual thing, I think it is a cultural thing in that, I mean, certainly things like a ladder in a game is something that you can just pick up as a kid and you just get it. Then you unconsciously learn a thousand of those things as you’re playing video games, and then you’re just ready for it, then you can have these sophisticated games that leverage this cultural knowledge. Then some of the things maybe do require individual growth, I’m not sure. But I mean, Alan K always talks about reading and writing, that’s not something that you just pick up. You really do need… I mean, I don’t know, maybe people pick it up, but it does require practise and it’s not as easy as unconsciously picking up that a ladder means then you can go up and down in a video game.
Well, when you put it in that context, yeah, the reading and writing certainly has quite a bit more depth that you can go through in your development of awareness of it. Yeah.
Yeah, so I don’t know if there’s a difference in category between just these tropes and things that need to be learned in order for you to leverage them. Yeah. I mean, like the idea of like a spreadsheet or something that has data flow, is that something that you just unconsciously pick up, or is it something that requires a good amount of practise before you’re really fluent with it? Maybe my opinion might be more towards the latter versus something, I guess, drag and drop where once someone shows you that you can drag and drop one thing, you sort of get it really quickly, or some of the gestures on touch devices that notoriously kids or chimpanzees and whatever can pick up just really quickly.
Or even something like knowing that if you can select text, that means you can also copy it, like that’s an available verb anywhere that text is selectable, at least in our modern interfaces. Yeah. Perhaps this dichotomy, like you’re wondering if there’s a difference in category here between these things, perhaps if it is, it’s a little bit like with playing music where there’s sort of the difference between developing proficiency at an instrument, which is sort of about you need the practise to be able to take something that maybe you can hear in your head and to express it on your instrument versus the sort of the culture of like if you’re a drummer, or a percussionist, or something like that, knowing what rhythmic patterns will evoke certain feelings, or compliments certain styles of music, or clash, if you want to be confrontational about it. It’s the sort of the pieces that are about your own ability to be expressive versus the pieces which are what you choose to express.
It also reminds me of Doug Engelbart’s stuff was like… I think his attitude was people should just take a few hours to learn how to use the thing. Nowadays, the idea of taking a few hours to learn how to use an app is sort of… It’s funny when you say it, but it’s ridiculous in industry to require that, right?
That hurts my heart so much because it’s… Not to make this about me, but one of my shticks is that I think there’s a lot of focus in the community doing research on new programming interfaces on the beginner experience, and one of my bants is that you can improve the experience for experts in a way that also benefits beginners. It’s an easier sell given the culture that we have, if you make an improvement for an expert and pitch it to people as an improvement for an expert, but then it also quietly sneaks in and it makes it easier to learn how to do this if this is your first time. So I feel like that focus on it being a requirement that you have to pick something up and figure out how to do it without investing any time in it, that’s something we’ve got to outgrow as a culture at some point. It just seems so far from present constraints of reality that we’re subject to.
Well, I guess, there’s some related ideas and… Well, first of all, I agree with your advice. I also like to focus on making things better for experts, and then that also benefits beginners learning it. I think it’s also things that are targeted at kids. For example, I feel kids like to use the things that adults use. So, for example, I think it would be great if Cuttle was used by kids, but I feel it would be used by kids because kids see cool stuff and then they just start using it, not because it’s made for kids. I don’t know, I guess things that are made for kids seem a little patronizing to me. But then another related idea is that video games are able to teach a bunch of things, but are able to do it in such a way that you feel you’re learning the whole time, and the tutorial’s not build as a tutorial, it’s like just the first level.
At least in a good game. There’s plenty of counterexamples, like there’s your JRPG where the tutorial is four hours of clicking through text boxes. But aside from that, yeah, totally. I know exactly what you mean.
I would like to thank Replit for sponsoring the transcript of this podcast. Replit is an online REPL that gives you an immediately productive environment to get up and running with any number of different programming languages, frameworks, and tools that you can use, including Git integration. They have a Multiplayer feature so that multiple people can hop into the same REPL and work on the same project together. It’s all very easy to get started with, and easy to scale-up to much bigger projects. They’ve created this fantastic sandbox for trying out new programming ideas, trying out new languages. They have a constant stream of new things that they’re adding, new ideas, new tools. They’re absolutely firing on all cylinders when it comes to making this environment robust and productive. Even if it’s not the kind of environment that you find yourself working in frequently, it’s the kind of thing that’s useful to have in your back pocket, if you ever need to pop up something on the go, or test something out. Or if you want to, as a weekend project, stretch your legs on a new language, but not spend half the weekend installing the compiler and all the dependencies and getting your environment set up and going through all that work. So check them out. They have actually (a couple of months ago I think) changed their name — they’re no longer Repl.it, they’re just Replit. So go to replit.com to check out their programming environment and all of the tools that come with it. My thanks to Replit for sponsoring the transcript and helping bring us the future of coding.
One of the themes of your work is that the user of your tools is creating and manipulating a program that generates graphics rather than manipulating the raw graphics themselves. I’m wondering if you’ve considered the inverse where you are creating and manipulating raw graphics that generate a program?
I guess. I mean, I think that’s a valid approach. I guess, I think of that as Programming by example maybe though I’m not sure if that’s… Your question could be asking a bunch of different things, but I guess I’ll interpret it as programming by example, where you make the finished artefact and then the computer sort of figures out, well, what program generates that artefact or these series of artefacts. The way you characterise my own work, I think, is accurate, you’re still programming, but you’re using a different set of affordances to create your program rather than typing it in.
I didn’t mean for this question to imply any specific interpretation. I realise now that I worded it as kind of a yes or no question like, “Hey, have you thought about this?” “No.” “Okay.”
But the other thought that I had, and part of why I’m asking this is I’m curious, if to generate these sort of like… Because when you say you’re creating and manipulating a program that generates graphics, that’s a theme that has come up a few times on this show. For instance, Ravi Chug’s project Sketch-n-Sketch is the same way where there’s a programming textual interface on one side of the screen, and on the other side of the screen is a graphics canvas, and you can do direct manipulation. Those manipulations that you do on the graphic side update an underlying program model that generates these two views, the textual view and the graphical view. It’s that same sort of idea that you are still programming.
So, I’m curious if, is this sort of like a software eating the world kind of thing where it’s all programming? Photoshop is programming, Excel is programming, HTML is programming. Or, is there even another side to this coin? The thing that I thought of was there’s an ISO lang, “Hey, future Ivan here, the ISO lang I’m thinking of is named Piet after Piet Mondrian, the Dutch artist famous for his abstract paintings of rectangular shapes in bright colours.” The ISO lang takes its name from that artist because the programs you create end up looking somewhat like one of Mondrian’s paintings. You create a PNG that has different colours and a sort of a pattern of squares, and it makes an image that looks like an artwork.
When you evaluate that program, the evaluator actually looks at the pixel values and executes that. Even though in that case, the interface that you’re working with is more about the graphics, you still have to do that computational thinking about it. You still have to think systemically about it. So I was curious if this is something that’s come up in your exploration where you found like, oh, here’s an idea for an approach where you can do programming, but get even further away from having to think computationally or think systemically?
I kind of feel you have to think systemically to program. I mean, I guess maybe some of this new machine learning stuff, you’re thinking a little bit not systemically, I guess. I don’t know, I’m not an expert in that stuff, but I can imagine some sort of machine learning paradigm where you’re training the computer, and so you’re not really thinking systemically. You’re just sort of rolling with it in some sort of way, which I don’t think that really exists yet, but I can imagine that existing at some point. I think you need to think systemically, even if you’re not really thinking in so much on the symbolic spectrum. So like status programming, you’re typing all this stuff, and that’s like leveraging your linguistic capabilities and you’re symbol processing capabilities. If you have the right kind of programming environment, you can draw flowcharts, or you can draw timelines, or there are all sorts of things that you can do that aren’t quite so heavily symbolic, but I think they are still pretty like systems. The system thinking seems like a more essential part than the symbolic thinking.
I guess, thinking about some of the approaches to programming by demonstration/ example that I’ve seen are the systems where the computer will present you a number of choices for… like you’ll make a change and the system will generate a program that would have produced that change, but there might be several ways to go about it. So it will say, “Oh, which of these examples of me extrapolating from that is the one that you meant?”
Then in that case, it’s kind of like your machine learning example: you’re not thinking so much about the how, you’re just thinking about the what of it, maybe.
Well, it’s different than the machine learning. The machine learning one is like its own mess, because that’s just all in some weird probability space or whatever. But what you’re saying about the programming by examples where it’s sort of doing some of the system thinking for you, and it’s certainly doing some of the corner case thinking for you that’s such a hallmark of programmer mentality, that it’s… That’s probably a big deal, I can imagine, because so much of your mind as a programmer, or at least when I reflect on my own experience or the experience of people I’m working with, it’s like so much of your mind is just thinking about, “Oh, what corner cases am I missing? Like here, here and here.” You’re like always trying to think about the corner cases.
Okay. There’s whole disciplines devoted to like, we’re going to use stricter and stricter type systems and TD, BDD to try and make sure that we’re always catching all those corner cases and failure modes and all that.
I can imagine the computer being able to assist in that in such a way that you don’t have to be thinking of the corner cases so much, or you can be… Like there’s a really good Takeo Igarashi work, who’s great. He’s the one who did Teddy, if you’re familiar.
No, I don’t know that one.
Well, the one I’m talking about is one called Pegasus. You draw with a stylus or something, and then it says, “Oh, did you mean to draw a line that’s perpendicular to this other line, or did you mean to draw a line that’s something else.” Then you can say which constraints you want basically, but you don’t have to think about what the constraint is before you draw the line. You just draw it and then it tries to figure out what could you have possibly meant by that, and then give you the option to choose what you meant. So I feel that’s like that paradigm of, you don’t have to be thinking about it at a very low level all the time.
You’re thinking more about what your intent is for the result and less about what the mechanics of achieving that result would be.
Yeah. I mean, your intent was to draw a perpendicular line, but you just didn’t phrase it to yourself that way.
Right. Or phrase it to the computer, yeah.
Right. Yeah. You didn’t have to remember the word perpendicular.
Yeah, that’s neat. So I’m going to keep moving through questions because I’ve got a… Then I’ve got a lightning round at the end, so we’re going to go through a bunch of sort of fuzzy philosophical sort of like, “What were you possibly thinking” kind of questions. Then we’ll get down to some ones that are like, “This pixel here in this spot, tell me about that pixel”, that kind of thing.
This is going to be another one of those questions where if the answer is just, “Ah, I don’t know”, that’s fine, but I figure it’s worth exploring. This is kind of a long question. So your artwork, and especially your later artwork, focuses a lot on mirrors and mylar reflectors, and you did a mirror-hacking workshop, and a lot of projects with kaleidoscopes. It feels to me like this is a good fit with your programming projects, like recursive drawing and even apparatus because an apparatus, they’re sort of the recursive drawing feature in there too, where a shape can contain itself. The results of the things that you create in recursive drawing, and an apparatus, and to some extent in Cuttle, also have that fractal-y kaleidoscopic kind of patterning to them.
Then you also did a project where you created sort of a prototype for what an interactive textbook might be like. This prototype was focused on pixel shaders, which are the things used by a… Just for the listeners’ sake… They’re the things you used by a graphics processor, a GPU, to actually colour all of the pixels in an image. So, instead of a normal computer program where you might have a single thread that goes pixel, by pixel, by pixel and sets them all to whatever colour they need to be in a graphics card, the GPU will take the same program and run it massively in parallel on a whole tonne of pixels all at once. So, you might have like 3000 different execution units, all running the same program on 3000 different pixels. For that kind of job, it’s a much quicker way to get a result.
In that project, you said that traditional programs, like those on a CPU, are usually quite long, and the challenge is in understanding how the computer steps through the program. Where, GPU programs are often fiendishly short and the challenge is in understanding how a simple program, when performed thousands of times in parallel, can produce a powerful effect. To me, that is yet another example of the theme in your art and with recursive drawing and the other drawing tools that can have that kind of fractal-y pattern or this mirror-like kaleidoscopic kind of effect, which is it’s about little pieces repeated again and again, with slight transformations, building up to a much bigger result.
So kind of realizing that theme in your work made me wonder that all of these examples, and especially mirrors, are about creating more of something in that they double their input. The mirror of one thing and the reflection is a copy of that thing, now you have to have it. The more mirrors you add, the more doublings you get. Same with recursive drawing, you put something inside of itself and you get a whole tonne of that thing. You get an infinite amount of that thing. That these ways of thinking and of working are all additive. So I was wondering if you’ve thought about the additive nature of this kind of work? If this is something you thought about, or even if it’s not something you’ve thought about, I’m curious if there’s a way to do a conceptually subtractive relationship in a sort of a direct manipulation or spatial way?
I don’t mean just like Boolean subtraction, where you have one shape and then another smaller shape, and you punch out like a hole from a square or something like that, where you’re subtracting one shape from the other. Because to me, that’s still additive in a way where you’ve got two shapes that are being combined to produce a certain effect. I’m wondering if this sort of additive nature of creating things in all of your projects is something that you’ve thought about and if you’ve run into limits with that, and maybe there’s like another side to that that you’ve done any exploring of?
I like the question a lot and I like that you found that connection between my mirror sculptures and the kinds of stuff that I do on the computer. I hadn’t ever, I guess, thought about the mirror as being a thing that adds. I try to think of it as something that folds, so it folds the space. That’s how I think about it. GPU shader also, in that if you’re sort of working in the grain of the medium, you’re like folding space rather than painting. But in terms of this thing about, it’s always adding, I see what you mean, and I could see it potentially as being a weakness of sort of the approach that I’ve been taking. That it’s like… I don’t know. It can be kind of Baroque, I guess, like it adds too much a fine detail or something like always adding. I agree that a Boolean difference is also adding.
So how could you do something that would subtract? I think it’s a really good question. I don’t have an answer right now, but it’s something that I’m going to think about.
Do you feel like for spitballing on it a little bit.
So, because I wrote this question, a few hours ago, and I’ve had a little tiny bit of time to think about it, the only thing that I came up with as an example of where this additive quality might manifest in your work as a limitation, where maybe something subtractive, conceptually speaking, would fit. This might just be my own, like not knowing how to use Cuttle and see other related tools you’ve made. But it feels sort of like these relationships that you build in something like Cuttle where you’re making a-
… relationships that you build in something like Cuttle, where you’re making a component and then you’re reusing that component inside other things and in a sort of a nested way, and you’re kind of building up these… It’s a very sort of tree-like structure of relationships between components. Is that fair to say that it is like you’re always creating a sort of a tree-like structure, or are there ways to use Cuttle so that the result of putting components together is more like a graph, where at some point there might be like a narrowing down of things in the hierarchy?
I think Cuttle is like, at least right now, it’s sort of the same amount of tree-ness as a typical status quo programming language thing. Usually when you’re programming, you’ve got tree-like stuff, because you have your conditionals and your loops and things, and your functions that can call other functions, but you can also, in your call graph… That’s a graph, right? You can call functions. So I feel like a typical Cuttle design is… I mean, it’s sort of modelled after programming. Oh, and then parentheses, obviously. We use a lot of parenthesis in programming. That’s also a tree-like thing.
I mean, if you have self reference and conditionals, then maybe there’s a way to make it less tree-like and more graph-like. I think it’s that conditional-ness of it. In Cuttle, is there a way to say that you have a component that contains two sub-components and then there’s some kind of parameterization of the parent component that’s like “in some circumstances show the one component child; in other circumstances, show the other component child”? A way to kind of-
Right. There’s nothing built in, but you could probably make that yourself. It’s the kind of thing we would consider building in. I’ve seen that pattern.
Yeah. Because it makes me kind of wonder… I think that’s where this thought came from for me, is it’s sort of like the thing that you focus on in your work is the parts of the problem space that are about building up things, and the parts that you haven’t focused on as much are the tools for like winnowing down. That’s where conceptually, to me, it kind of gets into a bit of a murky soup. Because is the addition of some kind of conditional logic conceptually subtractive, or is it conceptually additive, or does that distinction even matter?
I think it’s additive. I think if you’re putting more logic, even if it’s taking away in some dimension, you’re adding in another dimension. I guess the place I would go with it is meditation. I don’t know what the implications of that is, but that’s the only thing I can think of where it’s like you’re really… That discipline is sort of all about subtracting, I guess. There’s not really many disciplines that are all about subtracting, that I can think of.
Yeah. It comes up in aesthetics perhaps, like minimalism, or the very norm joke about jazz, about it being the notes you’re not playing. There’s some places where-
Yeah. Yeah, I guess like design in general. Subtraction is a very important part of becoming fluent at design, like taking away all the things that are unnecessary to achieve your result is part of what makes a design good.
It’s interesting to me to just think about: Is there a category of… Not even features, but a category of concepts in programming that are about that sort of narrowing down and taking away? Because whatever that category might be, it feels like your work doesn’t focus on that, and instead focuses very much on… It makes sense to me why you would be so focused on addition, because you’re focused on giving people expressive power in a new way and about letting someone create a very impactful result very fluidly. I think the focus on that experience drives you towards building tools that can produce a big impact, a big result, very quickly. Whereas the other side of it, the, “Actually I want to have less of a result. I want to spend a lot of time honing things and about reducing stuff,” that seems maybe complimentary, but definitely to the side of what you’re focusing on, at least in my sort of headcanon.
Are there things in the computer world that are on that side of the spectrum? I guess I think of most things in the computer world as being very additive.
Yeah. That’s a good question. Just as an aside, something I’ve wanted to do with this show is have more of these kinds of conversations where it’s sort of exploring a theoretical space or a design space together rather than just strictly doing an interview about a person’s work and their backstories. Doing a little bit of thinking together on the show. The thing that I’m realizing now, trying this with you, is that dead air is hard. Especially because we don’t see each other. We just have the sound of my squeaking chair and that sort of thing, and it’s sort of… I don’t know what you’re feeling, but I feel like there’s a pressure to kind of keep the momentum of the show going. So that’s an interesting…
But it’s thematic to have dead air if we’re talking about the opposite of adding.
Oh, there you go. Yeah. All right. There you go.
I don’t know where I heard this, but there’s some tip for if you’re trying to sort of lead a conversation in a grounded way that, before you say something, you take a breath. It forces you to slow down the conversation. That feels like a subtractive technique. It’s not a computer thing, but those are the kinds of things that I’m thinking about when you’re like, “Huh, all these things are additive.” Then I’m like, “Oh, you’re right.” That’s actually a big problem with sort of modernity.
I very briefly went to art school, and by briefly, I mean about nine weeks, and then dropped out of college and went off on a different adventure. One of the things that my very charming, very grumpy, grizzled drawing instructor said is that human beings are meaning-making machines, and that one of the functions that we can’t help ourselves but do is interpret and make things meaningful in some way. It sort of feels like that push against entropy is so inherent to the human experience, that (a) it makes sense that meditation is something we have to do as a subtractive practise, as a push back against that meaning-making nature that we have. But then, it also makes sense that something like computers, which are so modelled in our own mental image so to speak, are so much about creating more and more and more and not giving us tools to do less.
That’s a big problem with computers, I think. A lot of people experience that in their lives. Not programmers, just everyone with the technology. It’s so much adding and not enough dead air.
Yeah. Maybe a tool for CNC… That’s Cuttles’ stated purpose, at least in its current form, that was more about subtraction might do something like help you take an existing design and simplify it rather than starting from nothing and building a new design. It would be like an experience guided towards: How can you reduce the number of parts that are in this thing or reduce the amount of material that it takes to make this thing? You could use current Cuttle in that way, but I don’t think that’s a goal of the project, per se.
Yeah. I mean, the best answer I can give you towards this end would be that a goal of Cuttle is to get you off of the screen as quickly as possible, and working with the actual physical thing that you’re prototyping and making. So if we can make it faster for you to not be on the computer, that’s what we want. A lot of that is being able to quickly sort of sketch your idea without having to do unnecessary steps, and then also quickly iterate your idea without having to repeat a bunch of steps, which is sort of the status quo of design tools.
I like the idea of something that can simplify a design, though, by actually removing. It’s like if you wrote a program, could you have the computer just eliminate lines of code or write it in a more elegant way? I mean, maybe you could do it. That’d be a cool research project.
Well, that’s Haskell, right? Like Haskell’s whole thing about being lazy. That’s the point of laziness in Haskell is so that you can write something that is-
That’s more about the evaluation model.
Like I want to simplify your mental model.
Oh, right. Yeah. How do you envision that working maybe?
Well, when I worked with Brett, he would spend days doing that. He’d print out his code on four pages and spend a few days trying to make it three pages. He would say that he thought the experience was like writing poetry, which I don’t really do, but I think he does, where you’re trying to say the same thing but in a more economical way. Writing poetry, I don’t know if that’s a thing that a computer does very well. Certainly not in the subtractive sense, I would think. You could make a computer write a lot of poetry, but that’s not what I’m going for. It’s like, how do you make a computer write really elegant poetry?
It’s something that comes into computing culturally, in that there’s a culture around like Code Gulf or something like that, which is a different but related thing. I guess there’s also a culture around optimization. Can we make this process use less memory or finish faster, burn less battery life, that kind of thing? But that’s different than what you said about reducing the conceptual model, which I think that’s really something. If we could find an approach to programming that was about that, that would be, I think, very interesting.
Yeah, maybe like a automated proof assistance or something, I think. I think there are some results where sometimes the computer can come up with some elegant proof that no one had thought about before that does it in fewer moves, so to speak.
Mm-hmm (affirmative). Mm-hmm (affirmative). Though, it seems like, and maybe this isn’t with proof assistance so much, but with the kind of related field of programs that are for deriving formula, it seems like a situation that arises quite often is that they’ll come up with a formula, maybe to explain some phenomenon in physics or something like that, and the formula will be so fiendishly complex that there’s no way for a human to verify it. It goes beyond our capacity for… It’s not tractable by humans. You just have to kind of trust that, “Well, the program that we built has a robust model and it’s telling us that this formula holds, so we have to assume that it holds.”
Sounds like an overfit.
Yeah, maybe. Though, that might only be the case if there’s some sort of inherent truth to the idea that there’s a sort of a pure simplicity to reality. That just because all the fundamental formulas that we’ve discovered for acceleration and thermodynamics and so on and so forth have gradually trended towards simplicity over time, perhaps. Like principle of least action. Like the principle of least action. If that’s conceptually simpler than the things that it explains, just because that’s the direction that we’re sort of trending in our discoveries, doesn’t inherently mean that that’s true of everything in the universe. Maybe there is some aspect of physics out there that is truly thorny beyond… Like maybe Grand Unified Theory or something like that. Maybe it’s an equation with 200 quintillion terms or something like that. I don’t know if there’s an essential reason that it needs to be conceptually simple.
Yeah. I mean, that’s a philosophical debate.
I’m going to take a break-
Yeah, yeah. Go for it.
… for two seconds and I’ll come right back and then we continue?
Our second sponsor today is Glide. I introduced them last episode. That was the first time they appeared here on the show as a sponsor, and so we’re back for round two. I’ll give you a really quick review of what they’re doing, and then this time I wanted to share what I find particularly interesting about what they’re doing. Glide’s mission is to create a billion software developers by 2030, by making software dramatically easier to build. We all marvel at how successful spreadsheets have been at letting non-programmers build complex software, but spreadsheets are a terrible way to distribute software. They’re an IDE and the software built in it rolled into one, and you cannot separate the two. One way to think about Glide is as a spreadsheet programming model, but with a separable frontend and distribution mechanism. The way it works (and here we get to the part that I like) is that you pick a Google Sheet, and Glide builds a basic mobile app from the data in the Sheet.
This is great, for example, if you want to build a directory at your company, to record all of the people and their different positions and responsibilities. That’d make it really easy to find a person who’s the responsible party for a particular function in your organisation. That sort of app is a perfect fit for Glide, because the database that you use is Google Sheets. You have a really nice interface for working with tabular data in Google Sheets — that’s what it’s all about. You can use that interface to create and update and work with the data, and get all of that information into the shape that you want it to be in, and continue to modify it over time. Glide will pull the data out of that sheet and populate it into the app. The app’s interface is dynamically generated based on the spreadsheet that you present to it. They do offer some control over what the app is like as a result, and that’s something that they want to improve in the future. They already do a lot of work so that you’re focused more on the data in your app and less on what it takes to actually turn it into working software. So that big focus on the “what” of the programming rather than the “how” of the programming, which I think is huge. So here’s why Glide is sponsoring the podcast: They want Glide to become a lot more powerful. It needs to have much more flexibility and much more capability. Its declarative computation system has to support many more use cases without becoming yet another formula language. Its imperative actions don’t even have a concept of loops yet, or of transactions. Glide needs to integrate with tons of data sources, and scale up to handle much more data. To do all of that, Glide needs your help. If you are excited about making end-user software development a reality, go to glideapps.com/jobs and apply to join the team. My thanks to Glide for sponsoring the show and helping bring us the future of coding.
I’ve got three more kind of broader questions. Not nearly as broad as the last one.
So the first one being that Cuttle is meant to be a practical tool subject to real world use rather than more of an experimental prototype like Apparatus or Recursive Drawing, and you’ve said in early introductory post that you wrote about Cuttle that you’re walking back some of the more experimental ideas from those previous tools, and I’m just curious if you could talk a little bit about what those ideas are that you’re choosing to not pursue.
A couple of the ideas in Apparatus… I think they’re ultimately good ideas, but the just weren’t ready for me to have that many balls in the air, I guess. I mean, the thing that I’m keeping is the idea of you can directly manipulate your literals on the canvas and type them in in the inspector in the symbolic way, and I’m keeping the idea of components and instances. I’m actually adding the idea of modifiers, which Apparatus didn’t have. Apparatus had no way to refer to geometry that had already been created. But then, there were a bunch of ideas from Apparatus then I’m getting rid of. Not because they’re not good ideas, but just because they’re too tricky to handle all at once. So probably the first big idea would be: Apparatus didn’t have parameterized functions.
Instead, it had this sort of strange prototypical inheritance where you’d make a thing. Then any instance of that would be like a variant and you could change anything about that, and if you go back to the original definition and change it, those changes would propagate to the variance, except in places where you’ve overridden it. So for example, if you have a definition that was a shape consisting of a circle, a square, and a star, and then you made another instance of that, well, it starts as the circle, square, and star, but then maybe you could delete the star or make the star have 10 sides instead of five sides or whatever, and then if you change the original, those would propagate through, except if you change the colour of the star while the star isn’t there anymore in the instance… It’s actually based on Jonathan Edwards’ work Subtext. So the first Subtext, I believe, is the one where he has this thing where it’s like you sort of copy the whole tree, and then you can make modifications to that tree, and then the propagations go through. It doesn’t make sense in textual programming. It just becomes way too hard to deal with all the bookkeeping. But it does sort of make sense if you have a very sophisticated IDE, so to speak. So that’s a good idea, I think. But instead, I was like, “No, we’ll just do… They’re like functions. You have parameters that are sort of at the top level and you can reference those parameters.” In general, I’m trying to, where I can, leverage things that are tried-and-true from status quo programming. So like the idea of a function that has a couple of parameters, and then you can use those parameters within the function. That form seems to work. It scales to pretty complex stuff, so it’s like, I’m not going to reinvent that right now, because I’m trying to reinvent other things.
The removal of inheritance from Apparatus, is that kind of the main thing that you’ve done away with, or are there other things you could also point to where it was an idea present in your previous work that is deliberately absent in Cuttle?
I also got rid of spreads from Apparatus, which I also think is a great idea and I kind of miss them. I’ll probably end up having something more like a traditional for-loop in Cuttle. Cuttle doesn’t currently have anything like that that has these modifiers that you can sort of use in that way, but it’d be good to have a sort of first-class for-loop type mechanism. Apparatus had these spreads, which I borrowed from VVV. Anywhere that you can, say, pass a number, you could instead pass a spread of numbers, which is like a list. Then, everything that comes out is the list. So I guess it’s sort of like a list monad, but it just comes for free. That idea was in Apparatus and it was really powerful, and I don’t have that in Cuttle. But yeah, it’s just like, if you have that, then there’s a lot of extra implementation things you need to make that work, and UI things.
Is that… Because when you talked about taking out inheritance… Which was something I’m curious to talk a little more about, because that was something that I ran into in playing around with Cuttle. You mentioned that taking out inheritance was just sort of to, at least for now, simplify the development work that you need to do. I’m wondering, in taking these things out, or in taking out spreads in particular, I guess, is that motivated mostly by wanting to simplify the interface or wanting to simplify the conceptual model inside of Cuttle? What sort of motivates the removal in that case?
Well, with the inheritance, it’s definitely to simplify the conceptual model. Because I could never get it quite right in Apparatus. It was never really clear to me, in using it, what you were doing. Especially if you ever try to make a really recursive thing in Apparatus, it’s really tricky to understand sort of what level you’re changing a definition. Whereas, it’s like recursion is not easy with traditional functions, but at least it’s sort of a solved problem and there are known patterns and we don’t really have to think about it, because it’s something that we’ve been doing. By “we,” I guess I mean programmers have been doing for a while.
Yeah. Especially like quite experienced programmers. Like sure, recursion can be tricky when you first encounter it, but once you’ve practised and come to a level of sophistication with it, it’s a well-established pattern that we’ve done a lot of work to codify.
Yeah. I mean, so if you have a function in a traditional programming language, you declare your parameters at the top of the thing, and then those are the only parameters you have to think about. But I guess you could… Imagine if you wrote a function that you didn’t declare any parameters, but every single literal in your function became a parameter. That would be an abstraction mechanism, and it’s more powerful than declaring only some of the things are parameters. But it’s also a very different way of working, and that was sort of more like how Apparatus was. It’s like all your literals are actually parameters because when you instantiate a thing, you can change any of those things, and it’s just changing the default values of those particular parameters. It’s more powerful and it sort of made sense in Apparatus. It was a great experiment, but Subtext, I thought did it…
It was interesting because it had that feature, but it’s inventing all of the UI conventions around that is its own year or two or five of work. So we’re just going to have to suffer with parameters that you have to explicitly say, “Oh, this is a parameter of the thing,” but it’s a known way of doing abstraction. That’s the thing. It’s like, because there’s this whole landscape of abstraction mechanisms, how do you pick where you’re going to stand on that landscape and trust that it’ll give you the power that you need. So, unlike Apparatus where I was experimenting with exploring that landscape, with Cuttle, I’ve actually picked a spot which is sort of status quo programming, where you have functions with parameters. I know that that’ll give the community most of the power that we need, but may not be a perfect match. There’s probably something better that exists, but it would take a lot of research figure out how to find that sweet spot.
Yeah, because at least in this case, you know what the possibilities and the limitations are. Whereas if you did something more like Apparatus, there’s probably emergent properties of that design that would be very hard to create a good, reliable experience around without a lot more work. So, that sort of, I think, ties into my next question. The first thing that I tried to do when I started using Cuttle was make generative art, like the kind of thing I would do in Processing or one of those kind of tools. I’m not coming to Cuttle to try and make a tool path for a CNC machine to cut out a piece for my bike or for something that I’m building. I’m like, to me-
… for my bike or for something that I’m building, I’m like, to me, this is a drawing tool, there’s a canvas, there’s shapes, there’s colours. What can I do with that? What surprised me was even though that’s not really the initial focus of Cuttle, there was so much there that I could already do that made it really good for generative art. I found it charming that the limit I ran into was not about how expressive I could be with shapes and patterns and forms and geometric relationships, because that was incredibly robust, but that I couldn’t do even something like gradients. When it came to actually making things have colour, the tools that were most apparent were the first level you’d get from SVG or something like that, where you have strokes and fills of solid colours.
Doing something more than that required doing it at the level of geometry. If you want a gradient, you can do a repeating pattern of shapes and change the colour on each repeating shape a little bit. If you have enough of them, a roll your own spread gradient approach. So, I’m kind of wondering, Cuttle feels to me like something where you are quite focused in your initial design, but that there’s a lot of potential here to grow beyond the initial focus on CAD. I’m wondering if that’s a long-term goal, and if so, how we can go about doing that.
I think I do want to keep focusing on CAD rather than adding gradients. I am just really interested in the idea of making physical objects, so that’s what Cuttle is going to be about. But I think some of the broader ideas in Cuttle could be used for other domains. Like, you could make a whole animation system or something for visual art or something for sound or music using a lot of the same ideas of you have these primitives that are directly manipulable, but you could also parameterize them, and you have the idea of modifiers and components, which give you ways of combining and abstracting. It’s like taking a lot of the things that you get for free, so to speak, when you’re programming, and bringing them to other domains and making environments that are really top-notch for working in that domain. But the domain that we’re focused on is right now cutting, cutting tools.
Some of the best advice that I’ve heard for succeeding in making these novel interfaces gain traction in the world is to be super focused on a domain. So, it is cool to hear that you are happy with the domain that you’ve picked and are keen to just stay really, really focused on that. If someone else were to make a general purpose programming environment that was inspired by Cuttle and Apparatus and your other projects, what do you personally think that kind of a tool would look like, and in what ways would it be different from what you’re doing with Cuttle?
I don’t know. I feel like it has to be like you’re always implicitly choosing a domain that the tool is good at making things within. I don’t buy the idea of Python is a general purpose programming language. I don’t know. I don’t really buy that. It’s like every medium has things that it’s good at. You can look at something like HyperCard or something, it’s general purpose, but it’s also very domain restricted to certain kinds of things. I feel like all you can do is pick a domain and then get good at that, I guess. I don’t know, or you can, I don’t know, do what Brett’s doing with real talk, and that’s more of a systems level operating system kind of project with similar kinds of values and approaches. But I guess I’m not buying the premise of the question.
I love it. I think what I meant by it, not to try to constrain or invalidate or anything like that your response, but what I think about when I think about that is that there might be a notion of openness or closeness to a design, maybe in terms of a degree of focus in building something and in imagining how it might be used, and then in finding people and evaluating how they use it and seeing what they make with it, you can see the limitations that they run into. That’s just where that curiosity came from. You specifically said, “Hey, we’re probably not going to add gradients to Cuttle.”
In hearing you say that, it’s like, of course, because the goal is to make something that gets you away from the computer, and adding gradients doesn’t help you get away from the computer and back to working with something that you can hold in your hands necessarily. I guess the only other thing that I’d be curious to hear if you have more thoughts about is, is there something that Cuttle is doing that’s there to focus it on this domain of CAD and CNC, where maybe taking that thing away from Cuttle would broaden its usability?
So, for example, with Cuttle, you can make a modifier that rounds all the corners, but often in CNCing, you’re like, “Well, I want to round all the corners except for this corner.” It’s like, well, what’s the abstraction mechanism for that? We don’t actually have a great answer to that right now, but that’s a big thing that we think about. So, it’s like that’s driving the abstraction mechanisms in a certain way that, well, how can you do a thing, but have some exceptions to that thing? That’s different than the things that are driving Apparatus.
In Apparatus, there’s this cool idea that we don’t use in Cuttle, that there’s this back solving mechanism where you can write a formula that says, “Well, I want this…” say I’m making a planet going around the sun diagram and I have a formula that based on the time the planet’s going to be at this location, then you can also do this back solving thing where you make the planet an interactive object that you can then drag, and then that changes the time. So, rather than time driving the position, position can drive the time. That’s actually really cool because it lets you make generic, drag and drop user interfaces.
I think in the Apparatus demo, I make sliders, and then you can do all these things with sliders, but you can make all these other kinds of UI things. So, I think depending on what domain you’re in, that determines what you’re trying to invent to make that domain more… just make a better experience to be working in that domain. Then it’s a co-evolutionary process. Apparatus is probably more general purpose because the things that interactive diagrams take you to is sort of what you need to be making arbitrary screen interfaces. It’s more general purpose if you’re trying to get towards more screen interfaces.
If you’re trying to get towards more interfaces, because there’s, I think, a sort of an essentiality that people feel about general purpose programming, where their success criteria is can they use it to make more so-called general purpose programming. If you step away from that for a second, it’s like actually you haven’t made something that is truly general purpose. You’ve just kind of defined general purpose as being able to make more software.
That’s a powerful thing to be able to make more software, but it’s not the be all end all of things.
There’s a couple of other features that were in Apparatus that I think fall into that bucket where it’s like these are features that were very clearly made because Apparatus was pursuing that goal of being useful for making explorable explanations. To me as somebody who was very fond of Apparatus and goes to Cuttle and not having any participation in that maker community, I look at Cuttle and I just see the absence of these capabilities, because I’m that person who I’m thinking about making more software, and so I see those features, like in apparatus, the control that you have next to certain properties where there’s a little circle you can click, and then that means that whenever you drag on the symbol on the canvas, you’re going to be manipulating that property and it triggers the back solving. That’s not in Cuttle because that’s not what Cuttle is for.
Or even in the strange loop talk about Apparatus, you mentioned that one of the future potential goals for it was for the interface of Apparatus to be bootstrapped. I can imagine that that is absolutely not a goal for Cuttle.
Yeah. It’s not something that is high priority, I guess. On the other hand, we do a lot of bootstrapping in Cuttle, at least for sketching. Whenever there’s a geometry algorithm that we’re trying to figure out, we’ll often just open up Cuttle and start making it within Cuttle, because it’s so easy to live program, but then also make geometry appear and then algorithms, like say you have a Bézier curve, right? Like a cubic segment, and you want to be able to drag on the… it’s so hard to do a podcast where I can’t gesture and draw.
Drag on the middle of control point kind of thing?
Yeah, but not a control point, just like you’re stretching it like it’s a string.
Oh, yeah, yeah.
Right? Which is a common gesture, and it’s in Cuttle, you hold option and then you can stretch on the thing like it’s a string. There’s not a single solution to that mathematically. The curve could do all sorts of things and it would be correct, so to speak, but you’re trying to find an algorithm that feels good, that like feels like you’re pulling on it, and what do you want out of that algorithm? So, when we we’re prototyping that, we just did it all within Cuttle.
So, it’s still, even though it’s focused on CNC and in CAD, it’s still useful as a tool for thinking about geometry, because it’s a tool for geometry. Yeah. That’s an interesting interpretation of the term bootstrapping in that the impression that I got was if you were to bootstrap Apparatus, it would be that the user interface was created using the same components and the same back solving and inheritance and that that you would be using as a user of Apparatus. Maybe there’s some way to pop the hood on that and to go in and reconfigure the interface. Is that-
That is a powerful notion of what bootstrapping is, but, yeah, I guess I was thinking of it more in a mental model sense. An analogy would be like when you’re saying that, “Oh, Haskell has lazy evaluation.” I’m like, “Well, that’s just the evaluation model.” It’s the mental model.
It’s like bootstrapping the mental model using the tool.
That’s a theme throughout our conversation is that my initial thoughts about something are, “Yeah, but how do you implement it in computers?” Your thinking is more, “Yeah, but how do you implement it in the idea space of the person doing the thinking?” I love that. That is a valuable lesson for me to take away from this conversation. I had promised a lightning round at the end, but we ended up actually talking about and answering all of the questions that I had down there except for two. All right. The first one is in Apparatus, you can drag an attribute, and it’s kind of like it makes a little bubble into the expression for another attribute as a way of setting up a relationship there. Is that possible in Cuttle, or is that something that you decided not to do, or is that something that’s just not there yet?
I can imagine it not being there yet, but it is true. Apparatus has, the way it’s implemented, references aren’t done using names. They’re done using unique identifiers, and then the name is just a label on the identifier. I think there are some other… Subutex had that. The name is just… it’s like the difference between a name being meaningful to the computer and the name being a label that’s only meaningful to humans. I thought it was important for Cuttle to be both approachable by existing programmers and also for people who are not programmers, they can at least, if they start learning Cuttle, then it’s not too far removed from what their real world programming experience, so to speak, would be not.
Does that mean if you declare a variable at the project level and then you use it in a couple of places, if you rename that variable, does that renaming propagate to the places it’s used, or is it just-
It does, but only in the sense of the ID is smart enough to understand where you referenced it, and it changes it, but you can shadow variables, things like that.
Yeah. Yeah. That’s something I guess you couldn’t do in the Apparatus model, is do that kind of shadowing.
Yeah. Well, that’s the thing. It’s in that unexplored abstraction world, can you do shadowing? Maybe. I haven’t explored it enough.
Yeah, totally. Yeah. I don’t know if this is a good way of thinking about it or not, but to me, Cuttle feels a lot like the relationships that exist between things in Cuttle feel very much like the relationships you get when you’re building code, versus in Apparatus, it felt more like the relationships that existed were more data relationships. In the inheritance model, for instance, or in this example of variables, it sort of feels like in Cuttle, in the way that you’ve described it in the analogies you’ve used, you’ve talked about it being very much about you’re trying to make the things that you’re building in Cuttle feel like code. To me, that’s not how I would describe how I as an outsider felt about how Apparatus worked.
Yeah. Yeah. It’s very intentional. That’s because Apparatus was designed in a context of I was at this research group and the imperative was to try to push the limits and explore more, so that’s why it took all these interesting experiments. But with Cuttle, the idea is both to be practical, which means being familiar, even if it’s not quite the optimal thing, the familiarity counts for some points. Also, I guess, it’s more conservative because it’s the same structure as how you would code up your geometry if you were doing it purely with code.
ITP, interactive telecommunications program.
Yes. There we go. It took me so long to find what that acronym means. It’s not anywhere on their website, but, yeah.
Well, everyone thinks that stands for interactive technology program, but apparently it stands for interactive telecommunications program. It’s a wonderful program, by the way, for anyone who’s looking at something like that.
It seems super cool. I’m going to include a link to that presentation video that you gave. When you were introducing recursive drawing there, one of the inspirations that you talked about was the movie Inception, in that it allows you to have… yeah, it’s been awhile. That movie is about having shared dreams, and that may be one of the long-term goals that you had… I won’t say you had, that 10 years ago Toby had, a, was this sort of multiplayer, multi-programmer kind of collaborative sort of vision for working together on these shared creations. In asking this question, now, I can once again hear myself falling into the trap of thinking, “Ah, that means the answer to this is Figma style. We’re going to make it so that multiple people can move their mouse cursors and you see wherever everybody’s mouse is going.”
Not to answer this question for you, but I bet you’re going to say something about it’s about getting away from the computer into the shared world with other people, but to now to give you the chance to answer this question, is that still a goal of yours, and in what ways do you think that you’re still pursuing that goal?
Well, I think that was the shared goal that we had for Dynamic Land. Brett is still working on Dynamic Land, and I hope to still be orbiting and engaged with that project in the years to come. I mean, it’s supposed to be a 50 year plus project. So, I would say that Cuttle, it’s a different, I don’t know, it’s its own project, I guess. I would hope that it allows us to create things that lead to shared experiences. I mean, the reason that I would make anything laser cut is because I’m trying to share it with people in physical space together, whether that’s art installations or sculptures.
Something with the pandemic from last year is because there haven’t been any events, whether that’s festivals or gallery shows or anything like that, my art practise has been withering because I guess I can only do art when I have a venue, when I understand what’s the space and what’s the context. So, hopefully I can bring some of that energy back with things starting to reopen.
Well, that’s, I think, a beautiful sentiment to end on. Before we go, I just wanted to give you one last chance if there was anything you wanted to refer the audience to, or point to as something that people might be interested in seeing but that didn’t come up during our conversation today?
Well, I mean, the Takeo Igarashi reference, both Pegasus, but all of that group’s work is definitely worth looking at. They have sort of, I feel, a different perspective on how to create creative, powerful, but playful interactions. That’s something else I thought of when we were talking about, you were saying how everything that I’ve been making is an interface for creating a program that generates an artefact. I really like the forward to the essentials of programming languages, which is this book. The book itself, I haven’t really been able to crack too much, but I really like the forward, which is written by Hal Ableson, I believe, about how one of the great ideas you can have as a programmer is when you learn that everything is a program and you can write an interpreter for that program, and it’s just a really powerful perspective to have, and something that I think I was doing at the time. But reading it there sort of cemented it as, oh, yeah, that is what I do all the time. It’s just being able to write interpreters I think is a good skill to have.
Toby, thanks so much for letting me interview you today. I think this was definitely one of the most out there interviews that I have done so far, so thank you for going on that journey with me.
I had a good time. Thank you.
That brings us to the end of the episode. That was my interview with Toby Schachman. Like I said at the top, kind of a different vibe than the other interviews that I’ve done so far, but just to kind of speak extemporaneously for a minute here, as I am wont to do, I kind of like this. It’s something that I’m going to need to hone in on a little bit more in the future interviews to try and strike the right balance between, as we remarked on, dead air and actual things that are said. But I think that the podcasting medium is still very young, and it doesn’t need to mirror the format and the norms that have become common in things like television interviews or radio interviews, where they have a sort of a fixed schedule that they have to hit, and they have to be approachable to a very wide range of demographics.
I think that with a podcast like this, where it’s speaking in a really particular way to people who are doing a really particular kind of work and have a lot of shared background and shared culture, but also we’re gradually taking new people out of the mainstream crop of programmers and bringing them into these new modes of thinking and these new approaches to making software, et cetera, I think that there’s room for the podcast to just stretch a little bit away from being a more traditional interview format and getting a little bit more into collaborative thinking and about exploring subjects in a way that’s hopefully reaching a little bit deeper, even if that’s a risky thing to do.
So, hopefully you will stick with me as we continue to explore what this medium can be, and as I personally grow as an interviewer, because this is a new skill that I’m working on that I have just a tiny bit of practise with, but hopefully through many episodes to come, we’ll find a rhythm and an approach that really sings no matter who the guest is, and really, really puts the spotlight on them and their thoughts. Thanks so much for listening. Thanks again to Toby for coming the show, and I will see you in the future.