45 • Devine Lu Linvega • Orca
2020-03-01
Listen in your podcast player by searching for Future of Coding, or via Apple Podcasts | Overcast | RSS
Orca is a visual programming environment for making music. Except it’s not graphical, it’s just text arranged in a grid. Except it doesn’t actually make music, it just silently emits digital events across time. When you first see it, it’s utterly alien. When you start to learn how it works and why, the logic of it all snaps into place, and it becomes a thrilling case study for authors of live programming environments and interactive media tools.
Devine Lu Linvega, Orca’s creator, struck a wonderful balance between flashy style and significant utility. Orca is typically encountered as an inky black and seafoam green alphabet soup, pulsating to some species of broody electronic industrial throb. But it is also a forgiving learning environment that doesn’t crash, puts code and data together in the same space, lets you directly manipulate code and data interchangeably, allows generous recovery from mistakes, and supports discovery through freeform play.
I invited Devine to come on the show specifically to brain dump about the design process of Orca, how he started the project and built it up to what it is today. During our three-hour conversation we wound up talking a lot about all the other tools he’s created, and you can hear that discussion on last month’s episode. This time it’s all Orca — inspirations, execution model, operators, interface, system design, ports & reimplementations, interactions with other tools, and the community.
This episode contains many snippets of music, as examples of what you can make using Orca. All of it was created by Devine, and is available on his Youtube channel. If you like a particular piece and want to hear the full thing — and see exactly how Devine made it — they are all linked down below in the transcript at the point that they appear in the show. So just scroll and skim, or search this page for some phrase that neighbours the song you want to find.
Quote of the show: “It’s for children. The documentation fits in a tweet, basically.”
Videos
Here are some videos of Orca that do a wonderful job showing the variety of ways you can structure your code.
Links
Devine Lu Linvega is our guest. He and his partner Rekka funnel their lives and creativity into Hundred Rabbits. Devine has created countless tools, but Orca is the focus of today’s episode. He also appeared on the previous episode. Support them on Patreon, so they can keep making amazing things like Orca.
At the dawn of time, Devine was inspired to make a game by misunderstanding Autechre’s music video Gantz Graf. And, why not, here’s my favourite song of theirs. Yes, that’s one song. Put on some big headphones and play it loud while you read, debug, sleep, drive, trip, what have you.
In the theme of creation through misunderstanding, Orca was inspired by a misunderstanding of Tidal.
It’s not mentioned in the episode, but I wanted to link to this Tidal remix (By Lil Data, aka FoC community member Jack Armitage) of a song by Charli XCX. This remix slaps, but… you can’t really feel what the music is going to do based on the code, hey?
Rami Ismail hosted a year long game jam, for which Devine and a friend created a little block-based puzzle game named Pico, which would eventually become Orca.
Sam Aaron created the music coding tool Sonic Pi, which is included by default with Raspbian. It reminded Devine a little bit of Processing without the compile time, and seemed similar to Xcode’s Playgrounds.
Dwarf Fortress, ADOM (Ancient Domains of Mystery), and other Roguelike games are precursors to the 2D character grid of Orca. The code structures you create resemble the patterns in Game of Life.
Learning how to read Orca code is like learning to read the code in The Matrix.
Orca’s traveling N E S W operators are likened to Rube Goldberg machines, rolling ball sculptures, and the Incredible Machine.
Orca is a language that uses “bangs”, a concept borrowed from Max/MSP and Pure Data. Devine also made a similar looking flow-based web framework called Riven.
Generative music arguably went mainstream with In C by Terry Riley. Here is the definitive recording, and here is one of my favourite renditions. While you can make generative music with Max/MSP, or Ableton Live, Orca offers a much richer, easier approach.
The Chrome version of Orca is easy to get up and running with no dependencies, thanks to web platform features like WebMIDI and WebAudio — much easier than tools like Tidal or Extempore, especially if you use Orca’s companion synthesizer app Pilot.
Orca is so simple that it’s been ported to Lua and C. The C version runs nicely on the Norns, which is a little sound computer by Monome.
Ivan recently listened to a fantastic interview with Miller Puckette (creator of Max and Pure Data), which sparked curiosity about realtime scheduling for live-coded music tools.
A friend of Devine’s used the C version of Orca to directly generate audio by sending bytes to the soundcard.
The transcript for this episode was sponsored by Repl.it. They’re amazing, and seeing stories like this just melts my heart. Email jobs@repl.it if you’d like to work on the future of coding and, hey, help kids discover the joy of computing.
Orca’s Euclid operator U was inspired by the Euclidean Circles synth module.
The community around Orca largely grew out of the “lines” community, a forum started by Monome. They make a lot of pieces you can use as part of a modular synthesizer rig — you know, one of those giant cabled monsters used by the likes of Tangerine Dream in the 70s. People still do that, and it’s better than ever.
It seems like all node-and-wire visual programming languages, like Origami and Node-RED, are perpetuating certain conventions borrowed from modular synthesis without any awareness of that history and the limitations it imposes. This makes your humble host a touch grumpy.
The THX deep note was an early example of the wild polyphony afforded by computer-synthesized audio, as opposed to the limited polyphony or even monophony of analog synthesizers.
You can use Orca to control Unity, which is neat. You can use it to play QWOP, which is nuts. Speaking of QWOP, it’s part of a whole genre of hard-to-control games like Surgeon Simulator, Octodad, I Am Bread.
Devine has used Kdenlive and Blender to edit videos, since they’re both really good (for an open source programs). Better than editing just with FFmpeg.
Remember when Jack Rusher said “Orcl”? Yeah, good times.
Full Transcript
Transcript sponsored by Repl.it
Corrections to this transcript are much appreciated!
Music:
Ivan:
Welcome to the Future of Coding. I'm Ivan Reese.
Ivan:
Orca is a visual programming language. It is ostensibly a tool for music, but as you'll hear in this interview there's nothing strictly musical about it. It is simply a tool for generating patterns of events in time.
Ivan:
In this interview, Devine shows us Orca from many different angles. We'll learn about Devine's philosophy of creation through misunderstanding; Orca's origins as a simple video game, and how it evolved into a tool for music; the way Orca found a community, and how that community pushed the design of Orca in new directions; and of course, we'll spend quite a bit of time reflecting on language model itself — single character operators, base 36 numbers, no abstraction, all code and all data visible at all times.
Ivan:
Now, if you have never before seen Orca, you have two choices.
Ivan:
I recommend pausing this podcast and going to the show notes, which are at futureofcoding.org/episodes/45. There will be screenshots of Orca, as well as links to videos of people using Orca to live code music.
Ivan:
Your other choice is to just keep listening. Throughout the conversation, we will occasionally stop to describe how Orca looks or how it works, assuming that some people might need a mental image to understand what we're talking about. Podcasting is an audio medium, which makes it tricky to cover a visual programming language like Orca, but we did our best.
Ivan:
On the other hand, Orca is also a tool for music. I placed musical interludes at the natural corners in our conversation where we changed topic. All of the music featured in these interludes, and the music at the end of the show, and the music you're hearing now — all of it was created by Devine, using Orca. He has these songs and many more on his YouTube channel, and you'll find links to every song in the show notes.
Ivan:
Now, a word of warning. Normally, I like to listen to my podcasts sped up a fair bit — usually just a hair under 2x speed. If you're anything like me, you might want to start listening to this show a little slower than others. Unlike most podcasts, this show is edited, so you'll find there's less dead air and awkward pauses. This episode in particular warrants this warning because it has music. I specifically tried to select songs that would still be comprehensible, if not enjoyable, when sped up some amount — but if you go too fast, well, you're on your own.
Ivan:
If you missed the previous episode, it's worth going back to. It's the other half of my interview with Devine, where we talk about all his work except Orca. Each of these episodes is standalone, so don't feel like you're missing any context if you haven't heard it.
Ivan:
One last programming note: stay tuned to the very end of the episode if you want to hear who next month's guest will be. I'm pretty excited about that one.
Ivan:
Alright, on with the show. Devine is going to start us off by telling us where Orca came from, in a philosophical sense.
Devine:
Orca was me misunderstanding how Tidal actually worked. Sometimes you see something and then you're like, "Wow, this is brilliant." Then when you actually figure out how it works, you're kind of like, "Huh, that's not really how I interpreted it." Well, my life is full of those.
Devine:
I remember when I first saw the Autechre music video, I was like, "Wow, this is a great game. I really must find this engine," but then I realized it was all like-
Ivan:
Like hand-keyframed, pre-rendered.
Devine:
Yeah, exactly. But that inspired me to actually build what I thought it was. One of the first somewhat programming-y projects I attempted was something like it. I was like, "Oh, I'm just going make the Autechre music video audio react-y thing." I thought it already existed but I didn’t have access to it. Later, I learned that it just didn't exist at all, and it was just me projecting that it was a thing.
Devine:
Tidal was kind of like that. I saw people using Tidal on YouTube, and I was like, "Hey, I can do that." But then when I actually learned later how… it's hard to explain now looking back because I don't have that sort of innocent look on it. I thought what I was looking at might have generated this specific sound I was hearing but, I think, in the end it wasn't. With Tidal, and these kind of languages, it's sometimes kind of hard to inspect how the sounds are generated, or at least where they come from, especially in sequences. There's an emacs thing that lets you interpret some passage of Tidal code in realtime and you can kind of see the line highlights. Later, I kind of understood how that worked, and it wasn't that the music was making these lines blink, but it was the person just interpreting these specific lines on the beat.
Music:
Ivan:
For any of the listeners who haven't seen Tidal, Tidal is a live music programming environment. But it's structured much like traditional text-based code where you have lines of instructions that proceed in a certain order. It's sort of like a REPL where you can evaluate a certain statement, and that will start triggering some behavior in the music.
Devine:
Yeah, you'll pick your sequence that you want — because it works in kind of like sequence things, starting with d — and then you can be like, "I want to want to play this one," and then you can pause it, and you can also affect the others by whatever's playing at the moment.
Ivan:
It's all happening in a text coding environment that looks very similar to what you would be coding in if you were using C or Java or JavaScript or one of those kind of languages.
Devine:
Tidal is based on Haskell, but when you actually write Tidal code, it doesn't really look so much like Haskell but it has all these sort of like arrows and double semicolon and this kind of stuff.
Ivan:
The special syntax to mean things in a musical sense, right?
Devine:
Yeah. The Tidal language is actually really nice. It has a lot of things that I can appreciate more now, looking back. Orca is, I think, a good way to get into that sphere but it doesn't extend really well. If you want to break the mold of Orca it's tricky, or at least I haven't found ways that are elegant. When I look at Tidal now, I'm like, "Oh, yeah!" It being more verbose gives you more power. The challenges are different in Orca. Estate is really important. You get quickly cramped in your program in Orca. That's not something that you have so much with Tidal. But the trade-off is that with Orca, it's really focused on the idea that you can always see what is happening.
Devine:
One thing that I started to do is — like the show in Berlin — I would write some code that, even though it doesn't play just yet, people can expect it to play in a certain amount of frames, and people can see it on the screen. There's like this number counting down, and when that number's counting down, it sends a projectile and you can kind of like, follow it.
Ivan:
Watch it move across the screen.
Devine:
Yeah, and then you're like, "Oh, what's going to happen when it hits that thing?" That, in a sense, creates a sort of build-up and expectation. I thought that was kind of like a nice power to have for a crowd. Every single thing that generated the sound, you could see it. You can inspect everywhere the sounds come from at all times. That was kind of like one of the important things.
Ivan:
So that was, sort of, like your goal in starting it — to make an environment where, as a member of the audience watching somebody live coding, you can actually see things happening in the programming environment that match up with sounds in the music that you're listening to.
Devine:
Kind of. At this point, I was maybe halfway into the timeline of Orca because that's not really where it came from, but that's where it became a sort of live coding language. The origin of Orca is much more...
Music:
Devine:
With a friend of mine, we made a game for a game jam called Pico. It was a little puzzle game where you had generators and combiners, and you would send a red block and a blue block into a combiner, and then you'll get a purple block. The goal was to make some kind of shade of teal. You would have to solve this little puzzle by using these little machines, and it was in 3D space. That was maybe five years ago or more, a while back.
Devine:
Maybe two or three years ago, Rami Ismail was doing a sort of a year-long game jam, where there would be like one developer every single day of the year making one game. Mine was like, "Oh, I'm just going to make a demake of that color game that we made, and it's going to be in HTML. It's just going to be like moving little cells around." It had no sense at that point that it would ever make music. It was just kind of like… you could send v and the ^-sign, and >, <. You could send these little arrows around and solve little puzzles. I thought it was kind of neat. But then I saw people on Twitter use that to trigger MIDI events, and I was like, "Oh my god, this is so cool." So we're going to add more operators. We're going to add one that deflects, and one that reflects, and one that just splits. In a sense, the first few letters that ended up being in Orca were N S E W. Eventually, just adding little operators, I ended up having 26, like a whole family of them. Then I was like, "Well that's it." It's like a little puzzle game that lets you play music. I started to think that maybe I could use that to play a show. And then that's kind of like, "Oh, well, okay, so let's look at what other live coding languages are doing," and then I did it.
Devine:
I contacted Sam Aaron and I was like, "Okay, so, what can you tell me about?" I have a bunch of Raspberry Pi computers that I use for doing various kinds of work on the boat. When you install Raspbian, it comes with Sonic Pi. I was like, "Oh, cool." It reminded me a little bit of Processing without the compile time, and you can kind of like change things. It was also a bit similar to how Xcode had the Playgrounds, maybe it still does, but you could change things in realtime. I thought this was neat, but I never considered it for music. So I did some research, and then the operators for Orca started to stabilize a little bit. Then I was like, "Oh, okay, so most people need a quick way of writing sequences," and so on.
Devine:
The more I worked on it, the more I realized that maybe Orca was not a bad design for this. The first few shows I ended up watching of live coding, there were two camps. There were people starting from a blank page, which I liked, but it was very verbose. You end up typing a lot of stuff to start getting any sounds out. So the first thing I would do with Sonic Pi was make libraries of single-character functions, just like helpers and this kind of stuff. How that translated into Orca was that… well, every single function is one character already. Instead of moving up and down and scrolling up and down on a text file, you have the two-dimensional thing where one line can affect the line underneath without variable assignment, which I thought was tedious in realtime to do in front of an audience.
Music:
Ivan:
When you are looking at somebody using Orca to perform in a live setting, what you will be looking at is a text editor-like screen, but it is spatial on a 2D grid.
Devine:
It's more like Dwarf Fortress.
Ivan:
Yeah, it's like Dwarf Fortress or ADOM (Ancient Domains of Mystery) or any classic Roguelike game, where it's like a big grid of characters, it's two dimensional so you go up, down, left, right. It's not like lines of code that are written like prose or poetry or something like that. It's more like a map that you are looking down on from above. You have your cursor that can move around freely, and you can enter a character under the spot where your cursor is. Each letter of the alphabet corresponds to a certain function, or a certain "operator" as it's called.
Music:
Ivan:
Those operators can do things like trigger a bang every n ticks. You could put that next to something that, on a bang, will emit a certain MIDI note. That MIDI note will go to a different program, because Orca on its own makes no sound.
Ivan:
It will send a MIDI note to a different program, and that other program will have some synthesizer or a sequencer or something that will actually generate the sound.
Ivan:
So in Orca, what you're doing is you're building these little layouts of letters and numbers. They look a little bit like the shapes you see in a cellular automata, like a Game of Life simulation. You get shapes that look kind of like that have different letters and numbers that will generate MIDI notes, or they can also generate other kinds of signals to other programs — you can use OSC, or you can use UDP. People have used that to hook it up to game engines to do graphics for live visuals in their shows and performances.
Ivan:
What you are seeing, when you were looking at the code running, is the actual code — which is all the letters to have a function… that does some action… that affects some other function… that does some action… that affects some other things. You compose them together spatially. Then, there's all the data that those functions or operators are acting on. So it could be: this is a sequence of letters, so it could be like ABCDEFGABC for the musical notes, and they're all in a row. Then you can have an operator that takes that list of musical notes and cycles through them one at a time, and plays that note on the piano. Or it could cycle through them in a random order.
Music:
[The music here is just something Ivan whipped up to accompany the description.]
Devine:
I found that there's not anything like it that really helps explain it. A lot of the time, people just have to make the trek to find a video of it. Then everyone's like, "Oh, I see. That's very simple. There's only 26 letters that you can use, and that's it." Yeah. It's for children. The documentation fits in a tweet, basically.
Ivan:
What I find interesting about it is: you only need to know the tiniest little bit about how it works to go and look at the music that people are making using it in a very different way. When I first heard about it and was first looking at it, I couldn't make heads or tails of what was going on. It looks very neat. It looks cool. It has a very distinct visual style to it. It looks different than pretty much any other programming environment that I've ever seen. But then after learning the tiniest bit about what's going on — what things are data and what things are code, what things are changing over time and what things are remaining the same — you can suddenly start to see the clusters of letters and numbers almost like they're little machines in an assembly line. Each one of them is doing its own little part of the job. As you're listening to the music that's being generated, you can go, "Oh, that part down there is making that sound and that part up there is making that sound," and you can follow along with what the code is doing, because it's giving you so much information as you are watching it run. It's giving you visual feedback, because different things are lighting up and moving.
Devine:
The best metaphor that I heard about this was like a Cypher reference where someone was like… the first time you look at it, it looks like the screensaver in The Matrix. Once you've learned what North, West, East, and South do, and Clock, and Track… then all you're seeing is blondes and redheads.
Ivan:
Let's talk about those. It was fascinating for me to learn just now that one of the starting points for Orca was that N E S W, North, East, South, West — which are those letters of the alphabet that when you put them down in Orca, they will, every tick or every frame, move one space in that direction; so North moves up, "Tick, tick, tick, tick, tick" up the screen; south moves down — those things are bangs. So when they run into some other operator, they will activate that operator. It's very neat because you don't see them in all Orca programs, but you do see them used in a whole bunch of different ways. There are people who use them just scattered all over the place randomly to make glitchy sounds or you see people making waves out of them that travel up and down and up and down and bang different notes, sort of like if you're running your fingers up and down the notes on a piano. That part of the language to me at least feels like it's trying to break out of the constraints of text, like it feels very much like this could be extended to particles in 3D or something like that very easily, and it would come across really well.
Devine:
There's a name for that kind of machine, right? It's something like… Rube Goldberg! It's kind of like a Rube Goldberg machine where you'll send a little E, it'll bang, and then it'll push a little marble down — another Incredible Machine).
Ivan:
Yeah, exactly. For me personally, in all of the research that I've done and the work that I'm doing, working on programming tools that are sort of a little bit off the beaten path, one of the things that I keep coming back to is: you can make systems that have the charm and whimsy of a Rube Goldberg machine, but that can actually be useful for doing serious programming work. That's what makes Orca so cool to me. People are using it to make really, really good music in a bunch of different styles. The code that they're writing to do it has this very charming, playful character to it.
Ivan:
I think what we're going to do now in the show is kind of dig in to the different aspects of that design, because that's something that I want to try and get closer to an understanding of. Like, how all the pieces of the design came about, how they fit together, and how they contribute to that sort of overall feeling.
Devine:
I don't know. When you said like, "Oh, I'm curious to learn more about the design of how things came to be," I just had a flood of images of me fighting on forums and chatrooms to get operators changed. I guess… just go ahead and ask me, and I'll do my best to answer. But just know that it's not all my doing. It was a collaborative effort of a whole bunch of people, most of which are way more intelligent than I, and some of the operators that are in Orca I can't really explain properly. So let's do this.
Ivan:
Well, I think that's interesting. The audience for this show is mostly people who are working on their own projects. A lot of the people in the community have a tendency to want to retreat into their own private space and chip away at a design for years and years to try and make something perfect. I think hearing a success story of the person who's implementing the language being collaborative with the people who were using it through the development process — that's a story that I often hear going wrong. I think it's good to see an example of where that went right, and led to a more interesting result than what would have happened had you just made it on your own.
Music:
Ivan:
So this is a language that uses bangs. You have symbols that will emit a bang. A bang is… I know it as terminology from Max/MSP).
Devine:
I know it from Pure Data.
Ivan:
They're visual languages. They're some of my favorite languages. But this idea of a programming language where… it's live executing. In a traditional programming language, like a C or something like that, you don't need a construct that represents, "Make something happen now." In live programming, I think you really do need something that means like, "Make something happen now." I'm wondering, Devine, at what point did bangs come in? How did you arrive at things banging other things? Did you ever consider any alternatives other than that approach?
Devine:
Well, no. I think that aspect was there right from the beginning. But even before building Orca, my website used a framework I made called Riven which looks pretty much like some of the work you're doing, some kind of like flow-based web framework.
Ivan:
Yeah, it's like nodes-and-lines kind of visual environment.
Devine:
Yeah, and that already had bangs as well. It's kind of a concept that I used to think a lot. When there's interaction involved, I find that works really well. Also, inspecting flow-based concepts and tools, I find really hard without the concept of bang.
Ivan:
…and without being able to like see the bangs happening.
Devine:
Yeah, so Orca had to have something like this. I didn’t want to have just compare. There's three bang operators in Orca. I just can't think how I would do most of the things I'm doing today without this sort of thing.
Ivan:
I think the bangs, that's where the comparison to Rube Goldbergs really come from. Because when you look at a Rube Goldberg machine (I meant to say rolling ball sculpture), it's got marbles traveling around on these tracks, and the marbles will go and physically hit into something, and that will cause some action. Whenever I look at a language that has bangs in it, it feels to me like those are the marbles, those are the things that are pushing the action of the machine forwards. As opposed to like, in the engine of your car, there are no marbles. There's things directly connected to other things and directly driving them, but you don't get to see the sense of anticipation. "Oh, that thing's heading over there, and in a moment, it's going to cause something to happen." So bangs were there from the start.
Devine:
Yes.
Ivan:
When you did the N E S W operators that travel around, those were always bangs?
Devine:
Yeah. That's the first thing I did, basically.
Ivan:
What were the first things that they were running into?
Devine:
The first few operators… I had a whole bunch of special characters at first. Like the slash would be a sort of like a deflect, and a pipe was a sort of mirror, the tilde was a rotation. I actually didn't really use any letters, it was just special characters. Even the E W N S, I was a using > < ^ v to make little arrows. But at one point, I thought all the special characters were getting a bit hard to scale. I figured 26 would be a really nice limit to the scope of the project. A lot of people will jump into this sort of like creative programming stuff, and there's no scoping involved. It's kind of like open-ended.
Ivan:
Yeah, you don't know how deep you're going to go.
Devine:
If I put my foot down now and say, "Well, the limit is as many letters that we have," and every single extra characters are just IO, ways of communicating outside of software-
Ivan:
Yeah, we'll get to that.
Devine:
-that gives us something to work with. I absolutely love working with limitations, and this was just a beautiful limitation. We had to optimize for the fewest amount of things that would allow for the maximum amount of possibilities. That aligned with my idea of easy to learn and hard to master.
Devine:
Like, so, you want to learn Haskell? Well, alright, here's the Haskell book and good luck. But with Orca, there's 26 things to learn, so how much time can you waste just going through the whole alphabet like writing down, "Q, what does Q mean?" and so on.
Ivan:
They're like pieces of a chess set. There's only so many pieces. You only have to learn for each one of them how it moves. But then you spend forever and ever and ever learning how to combine them in really powerful ways.
Devine:
Yes.
Ivan:
You decided on 26 pretty much right as soon as you moved from having symbols (> <) to letters, you thought, "Let's go with 26."
Devine:
Yeah. So, suddenly I had 20 spaces to fill, because I already had maybe six operators down. I was like, "Okay, so what can each do," and then that brought the concepts of, like, iterating clocks, and all these little other operators that became the foundation for what Orca was to become. The whole time, there was this sort of like… I had to thread carefully between a tool that you can use to make music, and conceptual programming. I mean the people who use Orca, there's like two camps: people who find it a nice toy to think about programming and explore different programming concepts, who are all pushing for like, "Oh, we need greater-than," all these sorts of logic operators…
Ivan:
More data types.
Devine:
Yeah, exactly. That kind of stuff. On the other hand, people who are not programming- or computer-savvy so much, but are really into music, are pushing for things like, "Oh, I want more ways of writing atonal music," this kind of stuff. In the middle, there are people who can kind of see both sides. I found some really, really interesting people who came up and said, like, "Okay, so, I just found this paper that talks about how we can compress musical notation. I think there's a few things in there that you can use." We would kind of look over this and maybe make some changes. We basically implemented some papers in Orca. Because Orca is a way of writing music in a very, very dense way. I could write, like, an hour-long piece in just a handful of characters.
Ivan:
Yeah, it's extremely generative in that sense.
Devine:
Yeah.
Ivan:
For anybody who's not familiar with the history of generative music: there's this breaking point that happened around the 1960s. Music went from being something that is either purely improvised and not really notated, or something that is strictly written down in sheet music notation or various other notations that is played quite literally, to…
Ivan:
I think one of the first pieces was In C by Terry Riley, where you had just one page, a letter sized page of music, with something like 53 or so little snippets of melodies. The players of the piece were instructed to all start at the same point, and play that first melody as many times as they wanted as long as they wanted, and then move to the next one when they felt like it. Everybody would start, and then some people would move forward and other people would stay behind. The other constraint was, if you hear people are getting too far ahead of you, you should probably speed up. If you're getting too far ahead, you should probably slow down. What this led to is this one piece of written music, this one page, has versions of it that are played with every different combination of instruments you can imagine. Some of them take five minutes or less; some of them take five hours or more. It was the beginning of… finding ways to make music where there's a little bit of control by the author creating some constraints, and a little bit of a system, and then the performers have more leeway and more room for interpreting how to execute that piece.
Ivan:
Orca is also interesting in the history of music because it is a very approachable system for somebody to make that kind of generative music. Prior to Orca, the ways that you could go about building a generative system are a little bit less accessible. Like, there is Max/MSP, and there are some other musical environments. You can do a little bit of generative music using some of the randomization features in tools like Ableton Live. But to do generative music on computers has often required people to be quite expert at programming. I find Orca interesting in that respect, also. I think it borrows some of the approachability of video games and makes that available to people who want to make music.
Devine:
Well, beyond the tool itself, I think there's something to be said about the state of these tools today. Usually, in Tidal workshops, a third of the time involved is setting it up. I'm in a strange position where I thought it'd be kind of nice if it worked without dependencies, and anyone could just run it in the browser. It took me a while to get there. But now we have a version that's basically like 100 kilobytes. If you can install Chrome, it'll will work on your computer. It just uses the web tools, like WebMIDI, WebAudio. This approach is kind of nice. It's wasteful in other ways, but I find it's a good introduction for people who don't come directly from programming. Even me, I'm a programmer, and I had such a hard time setting up Extempore. That's not accessible for most people, I think. If the first step of your thing is like, "Oh, yeah, we'll install Haskell, and install emacs," that limits a whole bunch of people who might be interested in the concepts behind the tool.
Devine:
I was always kind of aiming, I guess, at children. I was like, if you can just open the page and put that in front of a kid, could they figure it out? It wouldn’t take that many keystrokes until they figure out which… like E will start moving, and through the act of playing they'll find their way without having to read the documentation.
Ivan:
It's an environment that is — unlike programming where you have an interpreter or a compiler, and you have to write just the right syntax and just the right way or else you get a syntax error, or maybe you're using a structured editor and so you don't get syntax errors but you still have to know the right way of combining all of the pieces of the language to get a result to show up — Orca is much more approachable in a playful way. If you know only four or five characters, you can put them down on the screen in a bunch of different combinations and there'll already be things moving around and things changing.
Ivan:
If you have something like the other companion app you made, Pilot — which is a very simple app, you download it, you open it and it's just a bunch of synthesizer banks ready to go, so that Orca can send messages to it and you can immediately start hearing the result, you don't have to do any configuration or anything like that — it's very easy to go from having never used it before, and having never done programming at all, to having written legitimate code — like this is actual computer code in a programming environment — that is generating music in a way that you understand and can start playing with and start building your way towards complexity, piece by piece.
Devine:
I'm not 100% sure, but… I don't think you can crash Orca.
Ivan:
Oh, that sounds like a challenge. All right, so, listeners: go ahead and play with Orca regardless, but while you're doing it-
Devine:
Send me the crash reports.
Ivan:
Yeah!
Devine:
One example is that you can go on any website, and just copy paste the whole frigging text content and paste it in Orca and something will happen.
Ivan:
Well, have you seen anybody hook up Orca to a Game of Life simulation, or hook it up to like a roguelike game or something like that?
Devine:
Well, Orca doesn't work like Game of Life. Game of Life, it just looks at the whole grid, counts the number of neighbors, and assigns a value to that cell. It'll look at one cell and look at like, "Okay, so for sides, and we have three neighbors, that cell is worth three." Once this is done, then it kind of like does the operation of moving on to the next frame. Orca doesn't work like that at all. Orca wouldn't work as a Game of Life ecosystem because it starts from the top left and goes to the bottom right, and it just runs that cell, and then moves on to the next. In theory, it's not allowed to look ahead.
Ivan:
Yeah, but that's what makes Orca super cool (or that's yet another thing that makes Orca very cool) is that you have put in special commands that can read in input and write it into the code grid, or send out things that are in the code grid elsewhere. What I'm imagining is you could make a little space in your Orca grid that is just a bunch of sequences of characters that are used for melodies or rhythms or something like that, and then take that chunk of the grid and send it to another program that's running a Game of Life sim. The Game of Life sim is what is generating your musical patterns.
Devine:
Oh, I see.
Ivan:
Orca is using these patterns to trigger the MIDI events, and then you have a synthesizer that's doing something with those. I think most of your tools work this way, where the tools have a very, very narrow range of responsibilities — like that Unix philosophy of do one job and do it really well. That forced you to have really, really good ways of communicating in each program with the outside world.
Devine:
Oh yeah! The I/O is super important for Orca because it really doesn't do much on its own. It's just a way of automating things. One person fed their webcam feed into Orca as letters — like, you know you can convert an image to some ASCII thing — and that was kind of neat. If you have enough ; characters, you'll end up with sounds and things. That was noisy and interesting.
Music:
Ivan:
I know our audience is really interested in how to implement programming environments. I think to a certain extent, it's interesting to look at: how did you go from zero to Orca? Because a lot of them are on their own journey of going from zero to a system of their own.
Devine:
That's a super good question, actually. All right, so like I said before, Orca had to be very accessible. I made the mistake, I guess, of tagging the repo as "programming language" because I thought Orca is kind of like a programming language in the same way that Brainfuck is a programming language. It's like, "Oh, esolang." But then that attracts a certain kind of people. If you said, "Oh yeah, I implemented it in JavaScript," these people would not have it. No matter how interesting your language or your project might be, if it runs inside Electron, it's garbage. I had my share of this.
Devine:
I was super fortunate to know someone who could help me with this. A friend of mine in Japan was like, "Well, we could make a super simple version in vanilla C and that would shut these people up," and we did. It was actually kind of nice because that opened up Orca to be available on super low-power devices. The C version of Orca I use every single day basically right now, because you can run Orca on Raspberry Pi to trigger a hardware synth with a super simple setup, written in a way that satisfies the need of the people who are very interested in the design of programming languages and see Orca as a programming playground more than a tool to write music. Since it's a very general programming thing, there are no specific tools in Orca that are designed just to make music. A lot people don't even see it as a music writing tool at all. That's probably how you've seen it, because on Twitter that's what's most viral, I guess. A lot of people just use it to animate some sprites in games or whatever. It has a sort of like general aspect that that reaches out to different kind of people. Having this the C version really shielded me from this sort of criticism, which I liked.
Devine:
What came out of this is that somebody ported that in Lua. A lot of musicians use Lua. It's kind of similar to Ruby, in a way. The implementation of Orca is so simple. I'm pretty sure you can make Orca client in less than 500 lines of code now. But the Lua port made it available to a whole new crowd of people. Like, it runs on the Monome Norns, which is a little sound computer. I really liked that since the scope is very small, it's really easy to port into other languages and that also addresses specific needs of these people much better. If I had only the Electron version, or just the Lua version, or just the C version… The C version, setting it up, is complicated. None of the musician people would be able to use it. The JavaScript version, it runs in the browser, and most of the people who see Orca as a programming playground don't run JavaScript in their browser. A lot of musicians don't like to be facing a computer when they're writing music — so for them, the Lua running on Norns is the perfect thing. Just being able to move across these different platforms easily, because the scope of the thing is so small, was totally a game changer for the project.
Ivan:
That's something that I know a lot of people in our community are wrestling with — the question of what environment to implement their prototypes in. That's a really strong testimony in favor of keeping your core ideas simple enough that you can do multiple implementations.
Music:
Ivan:
Going back to that idea of the tools being small and doing specific things, Orca not making sound of its own — I listened to an interview with Miller Puckette creator of Max/MSP and Pure data. He said that when he first started making Max, the thing that he was most excited about was the scheduler inside the engine that he wrote. Thinking of this program that he had made — which is ostensibly a tool for music — thinking about it in terms of just a way of organizing time and a way of getting the computer to create a notion of time that was useful for live programming in. You talked a little bit about the execution model for Orca. When you were building it, how much were you thinking about the way that it handles time, as opposed to just getting something that works so that you could be making music with it?
Devine:
I feel bad just saying not at all. Orca could be way more optimized than it is. It could be way more clever with time as well, but it's not. It basically waits for the frame to be finished, to be rendered, to do the next one after a specific amount of time. But it doesn't know how long it took to render that frame, and it doesn't compensate. Even though it will wait exactly the same amount of time, every time, no matter how long it took to render the previous frame. Which will definitely throw the time right out of the window. But I decided to keep it. I decided to keep it like this, because it's really hard to break the sort of the super-precise time signature of electronic music. The C version, you can run it at 90,000 frames per second, and it will run fine. It's very imperceptible. But for the JavaScript version, maybe you lose, like, six milliseconds, nine milliseconds every frame, and I was okay with this.
Ivan:
That's on par with the amount of drift that a human musician will have, it's on the order of ±10 milliseconds generally. Because Orca is not generating sound — it's just generating instructions that get sent to another program and that other program generates the sound — I don't know that I would have noticed that it had that time drift, even though I'm a musician and I'm used to listening to music in a very critical way. That's interesting to learn. I'm not hearing pops and glitches and that sort of thing that you normally hear when an audio program is struggling to keep up.
Devine:
Just like you said, the synth might do these kind of pops and things but Orca will just send events so you won't notice.
Ivan:
I guess what that means is another angle of approach to making glitchy music with Orca (which is really fun) would be to run it in Chrome using the developer tools that say, "Only give this tab 1/20th of the available CPU," something like that.
Devine:
Well, inversely, a friend of mine, she used Orca to do DSP, basically. The way she did it is sending UDP packets directly to the sound card at 44,000 frames per second.
Ivan:
Oh, from the C version or?
Devine:
Yeah! You can basically send bytes of sound to the sound card and you'll get sound out, and you can modulate this in your Orca program.
Ivan:
That's very interesting. Where they live programming it or did they sort of-
Devine:
Yep.
Ivan:
Oh! Okay. Since you had decided up front to have 26 operators, so you knew there was an upper limit on how much room you had in the design of the language… Another thing that you had said elsewhere was — or at least you alluded to this — that backwards compatibility was really important. It seems like you-
Devine:
No, it wasn't!
Ivan:
No?
Devine:
Did I say that?
Ivan:
Well, here's the sense that I got: you had said somewhere that once the 26 operators were decided that they were pretty much finished and that you weren't thinking that you would ever change those.
Devine:
Oh, no. I meant more like… that's the limit of what is available, but it's still subject to change. Whenever I say, we're never going to change the scope again, you know, two weeks later…
Ivan:
Right! Okay. That's interesting, because I read that totally the opposite way. I looked at that statement like, "Wow, you're making a commitment to making sure that people's Orca programs from a year ago will still run in the current thing." I was going to ask, how does that square with some of the things that I've seen changed? It looks like, to me, at one point you moved from having operators all have their arguments on one side to having their arguments kind of on either side. Is that something you did?
Devine:
Yeah, that's a massive breaking change, and broke it for everyone. Broke it for me. I had hundreds of hours of patches that just suddenly stopped working.
Ivan:
That's another thing that I think is going to be interesting to our audience. How did that go? How did you decide that it was worth it to make a change that broke all of the existing code that everybody was using? How did you get people over that hump? Why did you make that change?
Devine:
Well, I can't assume… I'm learning about all of this. I'm a bad programmer, and I'm still learning about this. I still discover things from the '80s that just blow my mind and suddenly I want to implement in Orca. When I started, it was like a little game that you could send bangs around. I didn't really understand the ramification of changing one operator for the other; I didn’t care so much. Nowadays, there's a whole bunch of people using it, so I have to be more mindful and respectful of the work that people have put in. There's no reason to do any change unless somebody comes up to me with something that just blows me away. Once in a while, somebody will show up and be like, "Oh, my God, I tried to do something in Orca. I couldn't. But, instead, I implemented this." The code is so simple to change that usually what happens is people will make the change and then they'll show me. It's like, "Oh, I've modified your X operator and then suddenly it does this. Look at what I can do." Once in a while, I am just floored, like, "Wow, how was that not in Orca?"
Devine:
This happened a lot in the beginning, and less so now. I know that whenever this happens, it just pisses most people off. I'm okay with that. I have to deal with the community. But usually what I do is not just sort of like change it, then don't tell anyone. I use the forum and the chatroom to communicate with users. We usually talk it over. Usually, some people will see some edge cases that I didn't see. Also, since now, there's three versions in three different languages, everyone has to be on board, because they're going to have to implement them. Usually, I don't touch the C version so much because I'm not as knowledgeable with C that — I mean, I was going to say I was good at JavaScript, but I'm not — I'm just kind of moving the pieces around. But, usually, people make their implementations of things. But, yes, everyone has to be on board, basically. Some people will have some angles that I couldn't see. Usually, that's what's happening. I get super excited about one thing, and then I'm like, "Oh, my God, we're moving toward this," and somebody was like, "Oh, but you can already replicate this behavior with this and that operator."
Devine:
Usually, for the past few changes that we did, they were super justified and the operatives that are now in place, I would miss them if they were not in. We also replaced some operators that were not used so much. I look at all the videos that people share on Twitter, and it's always the same eight operators people use, T C D U NESW X O, and that's it. The other ones, they're kind of like more esoteric and more kind of weird and harder to use and not as straightforward and have weird names and so on. So these ones, I feel more comfortable changing them.
Devine:
Recently, we changed three behaviors that are pretty core like the F which is if. That one, we basically reverted to what was in the beginning, but somebody convinced me to change it. After using it for like six months, we realized that it was much better before so we changed it back. The L which was kind of a placeholder, we never really knew who knew what to do with that one, so we just tried something and it was kind of breaking patterns in Orca. Now, it's just less than, which is actually really useful to do more computer playground type things.
Devine:
The other one was B. The behavior of B changed a little bit. It's more like, on itself, it won't count, but it will take a counter input and output like a bounce. The behavior changed enough that it's kind of like a different thing now, but the idea is that you can bounce outside of being on the clock. Now, you can take an iterate operator and feed its output into the bounce thing and you will get a bounce outside of the clock. You can do all sorts of things like this, which is much more useful.
Sponsor:
The transcript for this episode of the Future of Coding is sponsored by Repl.it. They are an online REPL for over 30 languages… (actually, it was 30 the last time I checked… but let me see how many it is now… 1, 2, 3, 4, 5, 6, 7, 8, 9…) 50 languages! It started out as a code playground, but it's now scaled up to a full development environment where you can do everything from deploying web servers to training ML models, all driven by the REPL. They're a small startup in San Francisco, but they reach millions of programmers, students, and teachers. They're looking for hackers interested in the future of coding and making software tools more enjoyable and accessible. Email jobs@repl.it if you are interested in learning more.
Sponsor:
Something I've noticed is that striking popularity among high school students, so much so that there are some high schools that I have heard of that have blocked Repl.it because kids were using it to create games in the browser — which is, as far as I'm concerned, about the highest endorsement that you can make of a programming tool, that it was blocked for creative use and kids were having too much fun using it.
Sponsor:
Thanks again to Repl.it for sponsoring the transcript.
Ivan:
Some of the operators act when they receive a bang, and some of them act automatically on every tick or on a certain interval. How do you decide which things should serve as a clock source?
Devine:
For the longest time, we had one operator that was simple. It was just "bang at x frame", the D operator. Then someone showed up in the forum saying like, "Oh, have you seen the modular synth module Euclidean Circles?" I was like, "No, I'd never seen this." So I looked, and it was based on a paper that encodes binary rhythm using a very simple formula that had two inputs and then an output. That was like just the best operator to have in Orca. It allows for all sorts of really interesting repeated sequences that you can use for snares and all these kinds of things. I didn't come up with it, I didn't do the implementation, but I use it every single day.
Ivan:
That led to the Euclid operator U?
Devine:
Exactly, yeah.
Ivan:
I was going to ask about that one, because that's one that I haven't played with yet. Looking at it in the list of operators, it seems awfully specific. Whereas everything else seems very simple, very similar to what you'd see in other programming languages. This one seems different than what I've seen elsewhere. That's cool to learn, that that's how that came about.
Devine:
At first, we were like, "Okay, let's try to implement it in Orca using the Orca operators." Usually, that's what we do. It's like, "Oh, let's try to do a multiply with Orca operators," and so on, but that one was just a monster. It was so big and there was no way that I would just, during a live set, try to rebuild this whole thing.
Ivan:
Because there's no way to make functions in Orca. There's no way to say, "Here's a submodule. It's hidden. It's running some complexity internally. You can open it up and edit it if you need to. But otherwise it's abstracted away." There's none of that.
Devine:
You cannot define anything except variables.
Ivan:
Variables will just store a value. You can't store a complex structure in a variable.
Devine:
No.
Ivan:
That's interesting that that's the criteria. Because that puts a lot of pressure on you to make really good use out of these 26 slots that you've got.
Devine:
Yeah, but there's… If you go Github help find these pages that explain every single programming pattern that is out there, there's not that many. A lot of them are just combinations of two or three others. Right now, when I look at the Orca specs, it covers most of these sites that just list all the patterns in programming. There's a few things that we omit by design that don't apply to the things that we want to solve with Orca. But in general, when you get to basic logic gates and that kind of stuff, you basically have something that is close to Turing complete and gives you enough space for iterating. That is just a matter of UX. We had to make it so like… There are some cases where, obviously, a third input port on an operator would open a whole bunch of new avenues and possibilities, but it's also another port and that-
Ivan:
Takes up space in the grid.
Devine:
Yeah, in Orca, your most valued thing is estate. You really want to save up on all this space because it operates on a 2D grid.
Ivan:
There's a there's a real benefit to keeping the operators spatially compact.
Devine:
Yes.
Ivan:
You mentioned earlier that there are some people in the musician side of the community who want to expand Orca to do things with microtonal music. The side that I immediately fell onto was wanting to have more control over, like, micro-rhythms and sort of more flexibility in terms of rhythmic composition, get off the rational divisions that are available. How do you respond to the people who want that? I think this is going to be interesting to our programming community because these are the people who are on the domain side of the problem rather than on the programmer side of the problem. I'm curious how you balance their needs against keeping the model of Orca that you have simple and pure the way it is right now.
Devine:
Well, that kind of feature I get a lot requested. It's like the OSC people. Maybe 1% of the people who will use Orca want this specific thing. So, I've implemented something that will that would kind of serve that. I mean, it's always a good request, "I would really like to be able to just use a micro… or add swing to Orca, or something."
Ivan:
Yeah!
Devine:
I was like, "I would use it. Yeah, sure, okay." So I started to look into it. Instead of changing the specs of Orca, I'm changing the client, because the client keeps track of time. I didn't feel like there should be an operator that messes with this. On the client side, I can do all sorts of crazy changes that basically… obviously, it locks people into that platform, but it's not so problematic. I can just add a client function that would do, like, add some swing, or change the BPM, or-
Ivan:
When you say client, just to be clear, you're talking about the GUI that is running the Orca engine inside it, and not, say, like a companion app like Pilot or something like that that's actually doing the synthesis.
Devine:
Yeah, so the C version has some specificities to it.
Ivan:
Right, or the Lua version.
Devine:
Yeah, exactly.
Ivan:
In addition to the 26 basic operators and things like # (comment), and some of the I/O operators like : and % and ;, there's this $ operator that is for special commands. It seems like that's sort of a pressure relief valve, or something like that, for adding things to the language. How do you decide what commands to add and when to add a command? I imagine it could be very easy to just go overboard on that and add tons and tons and tons of commands. But you've also done a pretty good job of keeping that tight. It's like: play, stop, evaluate one frame, set the beats per minute — it's like configuring what the Orca engine internally is going to be doing.
Devine:
There's not that many things. The inject command is just one command, but the ramifications of that command are huge!
Ivan:
Are huge!
Devine:
I was like, "Well, if I can spend 15 minutes on this one command, and then it just opens the gates for like…" well, actually, when I do shows, I can't imagine not using this.
Ivan:
For listeners, that's a command that lets you take another Orca file on your hard drive and load that into your current running Orca environment. It's a way of getting a little bit like a subpatcher or a function or a subroutine sort of behavior.
Devine:
Yeah, exactly. There's not many things that people requested that cannot be done with what is in place now. I'm always open. But usually, 90% of the time somebody would come up with like, "Oh, I really wish you could do this," and I'm like, "Well, you can. You can combine this one and is other operator," and then they're like, "Ah, ah. I didn't see this way. I didn't understand the behavior of that one." That's why I'm so active in the forums, and I actually love this. Imagine waking up every single day to questions about how to do something in the language you invented. It was my dream. I'm super happy. And, often, I have no idea. Somebody will ask me how to do something, and I'm like, "Well, let me think about this. I'm not sure I've done something like this before." So I'll spend a whole day doing design, but design that is on top of what's already there. I won't change the language to answer a question. I'll just try to figure out how to do it with the language that I've made. A lot of people think I'm the best person using Orca. I'm not. There's a whole bunch of people that I know that are way better at solving Orca problems than me. Usually I'll just ask them how to do this or that. Or I don't remember how to do, like, greater than. I always forget the silliest things.
Ivan:
The little patterns that you can make.
Devine:
I have a file on my computer of all the patterns that people have given me that I can use and also reuse, and some things are less obvious. A lot of people coming from programming, they're used to doing, like, paper computers, or adding binary numbers. That is a skill that is really useful in Orca. Orca is base 36 but there's a lot of math stuff that just goes over my head. But then I'm like, "Oh, how do you do again, multiplying with addition?" And then someone's like, "Oh, you have to make this and that," and "Ah yeah, obviously!" It's really not straightforward.
Ivan:
In case anybody missed that: Orca uses a number system that is base 36. I'll leave that as an exercise to the listener to figure out why that is. But it's, I think, another pretty neat part of the design.
Music:
Ivan:
While you say that you are not the best Orca programmer, and you're not designing this language… you're not like the benevolent dictator sort of person saying, "Here is Perl 6, accept it as it was made over the last 15 years." You're not doing that kind of process… What things do you think you've done that have really allowed the community to flourish in the way that it has?
Devine:
Well, I know for a fact that I'm really not good at this. So, I just kind of like… I trust everyone… well, I mean, I'm very easy to convince. I don't really hold so strongly on some ideas or some preconceptions of things. If at any point, I'll be like, "Well, the Orca design is absolutely perfect. There's no way this could be any better," then, of course, it would be harder for anyone to show up and be like, "Oh, well, I really wish I could do this, which I can't." But now I'm just kind of like I let myself be convinced. If someone is interested in Orca enough…
Devine:
I'm super humbled by the fact that people spend time playing with this at all. I made this for myself. But whenever I see people doing workshops and shows and things like this with the tool, I'm just like, "Wow, there must be other people like me out there," because, usually, I just feel like nobody makes stuff for me. Like everything is made for some other fictional human being that I am not, because I can't relate or use any of the things that are found in the mainstream. So I'm glad to see that there's some other people who appreciate these kind of lean designs.
Ivan:
To what extent does the other work that you're doing as part of Hundred Rabbits play into the community around the tools that you're building? You've got quite a following through that. It's sort of neat to see how each new thing that you make — whether it's your Inktober series that led to the creation of some new art tools, or whether its Pilot that came about so that you could have a plug-and-play, ready-to-go synthesizer for Orca — they all sort of build off of one another. Do you see a lot of people coming to Orca from the community of people who were already following your Hundred Rabbits work? Or is it something that… that community came from elsewhere?
Devine:
Well, in the case of Orca, it totally came from elsewhere. Actually, most of the Orca community comes from the Monome forum. Do you know what Monome is?
Ivan:
Yeah. For the listener, Monome is a company that makes a bunch of boutique hardware for musicians. Each of their different pieces of hardware have a very pure visual and material aesthetic. They're very simple. They're very elegant. They all sort of connect to modular synthesizers, which… If you've ever seen big a synthesizer that, like, Tangerine Dream, or somebody from the '70s, with all the cables running here in there and everywhere — that sort of synthesis is still going today. It's still very popular. So they make a lot of pieces that you can connect into one of those big synthesizer rigs.
Ivan:
Some of their tools are just like, "This is a keyboard that has a bunch of unlabeled buttons on it, and you can configure what you want those buttons to do." Or, they have a module that is called the Norns, that is just a little box that will run some computer code that you feed it. But it's a very simple, minimal computer. It doesn't do very much. Because a lot of people who are doing modular synthesis, they don't want to use a computer. They want to get away from the computer and go strictly with traditional electronics. The Norns is a way of introducing the tiniest little bit of the power of computation, and the dynamic nature that that brings, back into this process, without having a laptop and a screen and an operating system and a Slack client and notifications and all that kind of stuff being a part of your live performance. That community is a very neat place, sort of at the boundary between people making music with computers and people making music without computers.
Devine:
It's not so hard to see that how the Unix philosophy translates into music is via modular synthesizers, because every single module targets one task and just does that. It talks really well with the same protocol to all these other machines.
Ivan:
In fact, people familiar with patcher languages like Max/MSP, those are inspired by modular synthesis. In modular synthesis, each of these modules is a node and the wires between them are physical wires. You're plugging in patch cables from one to the other. Not to get off on a tangent, but I think a lot of the limitations in the node-and-wire visual programming languages we have come from their faithful adherence to the way that modular synthesis works. You see that influence all the way through to today. I think that a lot of programmers who have never seen modular synthesis have a bit of a blind spot to the fact that a lot of the design decisions underneath came from that world and the limitations of that world. They're sort of cargo culted into every new… like Origami and Node-RED, all of those programs just carry on with these designs without really thinking, "Oh, yeah, the only reason we're doing it that way is because, once upon a time, people were making electronic synthesizers connected with physical wires meant for performance on stage or in a recording studio."
Music:
Devine:
Imagine my position, where I was working on Orca and then someone was like, "There's a thread on Orca on that forum," and then suddenly I could access this crowd of people. I felt so much at home there. I was like, "Oh, my god!" It was in my peripheral vision. I could not see this demographic at all. Once I discovered this, I was like, "Wow, this is a place where I feel really at home." It's the interconnection of programming and music, and people are aware of how their tools are working, and it's a big open source mentality there. Like, Monome builds lean, beautifully crafted, open source hardware. As Hundred Rabbits, our studio was trying to address this via software. I was like, "Whoa, okay, so this is going to be my home now." This is basically where I spend my days. Nowadays, it's just looking at what people are building with Orca, with the Norns, or whatever they show on the forum but also now it's like a big source of inspiration, which I kind of wish I had stumbled upon way, way before.
Ivan:
Me too. Their community is it's one of those little corners of the internet. They haven't exactly found their in-group. They all have that overlapping passion, and there's just a lot of energy there that feels really, really good. So that's where you feel like most of the Orca community came from, was out of the Monome community.
Devine:
I think the time was perfect. But also if you look at Orca closely enough, it looks like kind of like it's a bit Lisp-y, it's a bit like a flow-based language. I think people who come from the modular synth scene, they look at this and they're like, "Yeah, okay, I see. I get it."
Ivan:
I think that's what drew me into it. You can look at Orca and see echoes of so many other interesting styles of programming that we've had in the past. The fact that they're brought together in service of making music, for me, that's just an added pleasure because I enjoy music and making music. But I think you don't even need that to appreciate the little details that it's borrowed from everywhere else, synthesized into a new kind of thing.
Ivan:
When I play with Orca, and when I look at the code that other people write in Orca, I sometimes feel like it's kind of fragile. I think that the premium that you place on the real estate — the amount of space that you have to build things — that makes it feel a little bit like… if I typed the wrong thing in the wrong spot, it sort of causes this cascading failure, or things can kind of go haywire. I haven't had that happen too badly, but it always feels like it could happen. Is that something that you also feel? Is that something that people feel when they're new but in practice it's never a problem? Have you ever seen somebody accidentally send their whole Orca patch way off the rails?
Devine:
I certainly have. There's not that many failsafes to stop this. I mean, you can undo. They way undo works kind of keeps the time. I mean, I messed up in front of an audience and nobody noticed. But sometimes, they are very destructive things that you can do. You can accidentally paste over things that you didn't want to erase. Usually, I assign a lot of the variables up top and… this is like a no-go. I have to be very careful not to go there, and change things because it'll just break everything. Once in a while I do, but I can always undo.
Devine:
I made one kind of… I guess I call it a sort of helper or like… A way of minimizing mistakes. Usually when I do shows I use the monophonic operator, unless I really need to do long release notes. Little things like this, that makes it harder to fail, are useful. Nothing stops you from, like, accidentally sending the letter Z on like the fourth octave and you get like the highest pitch.
Ivan:
Yeah, high piercing noise. Just for the listeners, the monophonic operator — the way MIDI and synthesizers work is that, normally, at least these days now that we have nice technology, when you're playing a melody, if you push two notes at the same time, they will both play. If you push one note and then let it go, and that note has a bit of release on it, it will gradually fade out. Then if you push another note, it will play. All of the notes are independent. But when you run a synthesizer that is monophonic, each new note will stop the previous note from playing. So when Devine says he would use the monophonic operator as a safety net, the reason that that works is: if something screws up, he doesn't accidentally send like 1000 MIDI notes all at the same time out to his synthesizer, which would be cacophonous. It would just send the very last MIDI note, because that last one would stop the one before it, which stops the one before it.
Devine:
Exactly. It's a safer way of doing this. And... I just want to stop you there because you said, "Now, we have good technology."
Ivan:
[Laughs]
Devine:
I think monophonic synths can be, by design, made today. All my synths are monophonic. It's something that I actually look for.
Ivan:
I guess what I mean is like, once upon a time all synths had to be monophonic because there was only one oscillator. You could only make that oscillator vibrate at a certain single frequency. The way we got from only being monophonic to being polyphonic is by adding more and more individual oscillator pieces in hardware. For a while, if you got a fancy synthesizer, it would let you play like eight notes — up to eight notes — because it had eight physical oscillators in it. Then when things went computerized, then it was like, "Okay, now we can do the THX deep note and have 5000 oscillators all running in parallel because they're virtual. That's what I meant by better technology. But I agree with you that monophonic synths are almost a different category of thing unto themselves, and they're useful and beautiful in their own way.
Devine:
Do you think really it's because they added more oscillators? Like, it's not something I know about that much…
Ivan:
Oh, yeah. Back in the in the '60s and '70s, when they were first making synthesizers — and even before that, like the theremin originally only had one oscillator in it. An oscillator is a is a fairly complex component as far as I understand it. You have to tune them individually. If you had multiple oscillators, for a while there, you could tune them to be in the same center of what note they would play. So they're all tuned to, like, when you press A, it's going to be 440 Hz. But if you took that synthesizer outside and put it in your car, and then drove to a show that you were going to play, and you took it out of your car, and brought it in, the temperature change would have thrown all of your oscillators out of tune.
Devine:
What?!
Ivan:
Yeah. If you listen to recordings of old synth music, a lot of times, they would have to retune their oscillators throughout the show because the heat generated by playing the synthesizers would throw the oscillators out of whack.
Devine:
That's amazing.
Ivan:
There was a while there were it was a very physical limitation, so mono was just simpler. You didn't have to worry about your oscillators going out of tune with respect to one another, when you did want to tune it was easier to do, and it didn't cost as much. That was one thing where, music going computerized, that was an early selling point. Even though the quality of synthesis was much poorer, the ability to have as many oscillators as you wanted — because you would pre-compute all of the audio, you would set up your sound system and say, "Render," and then it would render your audio out using code, it let you effectively have as many oscillators as you wanted. That's part of how music went from being done on physical electronic synthesizers to computer synthesis.
Music:
Ivan:
What are the sort of weird things that people have done with Orca that defy the purpose that you built it for?
Devine:
There was one girl in France who was using Orca in her class. She was in robotics. I guess it was like a sort of like an esoteric programming robotic class or something, but they were all using different kind of… someone else was using Sonic Pi for robotics, somehow. Anyway, I ended up in a position where I was answering questions in how to move a robotic arm via UDP in Orca. Orca doesn't have that much granularity in its… I mean, it doesn't have any sense of floats. There's 36 numbers, so that gives you, at most, 10-degree rotation angles. But that was a nice limitation that this person want to play with. So I played along. That was kind of a weird reason to use Orca. I remember, at the time, we didn't have the multiply operator, which made things a bit harder.
Devine:
What other weird things? Well, one of my favorite Orca artists uses Orca control to Tidal, which I think is amazing because sequencing in Tidal, I find, is a bit hard. Well, no, sequencing in Tidal is fine. Building tracks — entire tracks — and mixing and navigating this music I find in Tidal doesn’t come naturally to me, and I have a hard time getting a sense of what is happening when I'm writing Tidal code. But Tidal is really, really good at synthesis. You can make really interesting granular synth stuff and glitch and things, and it has a whole bunch of interesting effects you can use. I found that the combination of sequencing in Orca and synthesis Tidal is like a really good match. That's been one thing. It's not super weird, but it's super interesting.
Ivan:
I see in the README, that there's information about hooking Orca up to Unity. What have you seen people doing with that integration?
Devine:
A friend of mine used it to animate a character. It's a nice way of doing automation in general. So you could imagine… You know the game QWOP?
Ivan:
Yeah, Q-W-O-P?
Devine:
Could you imagine playing QWOP with Orca?
Ivan:
Oh, wow. So, for the listener, QWOP is a game where you have character kind of like a stick figure. Your goal is to run a hundred-meter sprint, and you control it using the letters on the keyboard, Q W with the left hand and O P with the right hand. When you push one of the letters down, it sort of bends a joint on your body and, when will you let it go, it unbends the joint. So you have to use the letters… You type them in a very careful rhythm to make your character run. It was one of the first games that sort of led to the movement in games… of games where they are very hard to control. There's like Surgeon Simulator where you're a surgeon, but it's very hard to control the surgery, and so you end up accidentally dropping all sorts of scalpels and that sort of thing into the patient. There's a game called Octodad, where you play as an octopus who has to do a bunch of parenting activities. There's a game called I Am Bread where you play as a piece of bread and your goal is to get into the toaster, and you have to kind of climb across the level. There's this whole genre of games where the fun comes from… you know what you need to do, and what you want to do, and getting the game to do it is very hard. So, imagining somebody using Orca to try and drive one of those games… like, that, to me, is a very charming, entertaining idea.
Devine:
That was very interesting to watch.
Ivan:
Did it work?
Devine:
No! Well, I mean… but does it matter?
Ivan:
No!
Devine:
I can't even play QWOP, I'm terrible at QWOP.
Ivan:
Yeah, nobody can.
Devine:
Who cares? It's just my interest and entertaining.
Ivan:
Well, it feels like the sort of thing that, in theory, if you could do it just right, it would work, because it's a game of getting the right rhythm.
Devine:
I know! It's like, fine-tuning the release is what it's about, but can you even get there?
Ivan:
That's very cool.
Music:
Ivan:
That's one of the things I'm going to be most excited to see in the future: what other curious uses of Orca people come up with. It's a tool that is so… it's interesting to look at, and it's interesting to write in, and it feels very fresh and different. I think that inspires people to go, "Oh, what can I use this for?" That's immediately where my mind went after seeing it, was like, "I want to hook it up to Game of Life somehow," or "I want to hook it up to other kinds of things, like a roguelike — a game that uses the character grid in the same way, but to a very different end." So I'm very stoked to see what other people come up with for that.
Devine:
Do you remember when Jack was on the podcast and he was saying, like, "I'm not sure if it's pronounced Orca or Orcl?"
Ivan:
Yeah… [laughing]
Devine:
[laughing] I just remembered that.
Ivan:
Because of the Λ (lambda) as the A.
Devine:
Yeah. Well, if you do make something weird with Orca, send it over and use the hashtag #ORCΛ.
Ivan:
That kind of ties into my second last question here, which is: what tools do you want to make that you haven't started making yet, or that you've started working on but haven't finished? What's the next itch that you feel like you need to scratch?
Devine:
In the perfect world, there would be no more tools for me to make, because I just make them because I can't find them or I can't use them. Right now, I'm somewhat content with how I'm doing computers. So I don't feel like I have to build everything.
Ivan:
Have you done a video tool? Because you make a lot of video work.
Devine:
Yeah, actually, I use Kdenlive. It is excellent. I am always the first one to complain about how open source programming looks like shit, but Kdenlive looks a little bit terrible, but it works really, really well. So I'm super satisfied with this. Actually, we had to edit the video, and all we ever knew was iMovie.
Ivan:
…Yeah.
Devine:
[laughs] That's the tone of voice I have in my head when I think back.
Devine:
We had to migrate our whole workflow. We tried to edit in Blender, and that was not so terrible. There's always a way. Sometimes, it really helps to be in a position where everything stops working. I really had to install Linux on bare metal to be exposed to the new challenges that I would be facing, these past few months. If you run Linux on a VM, well, you can always be like, "Well, I'm just going use the Windows version because I'm running it on Windows," or something. But being in that position, I frigging love it when things are kind of hard.
Devine:
Video is one thing that was daunting. I was like, "Oh, my God, what am I getting myself into? I'm really not in a position where I can do web-based JavaScript video editing. I didn't want to edit all the future videos with FFmpeg in the console. Yeah, Kdenlive worked really well. Blender's an amazing 3D… Actually, like, getting away from OSX forced me into Blender 2.8, which is by now my favorite 3D modeling software by far.
Ivan:
Blender's amazing.
Devine:
Yes. I guess the convenience, the comfort that I thought I had was kind of illusory. Now, I can see that actually I was just being sheltered from the things that I really was looking for. Recently, I was having drinks and thinking about this sort of idea. You know, when you go from, I guess, omnivore to vegetarian or vegan, you think you lose something. You're like, all the things that I was eating, I'm going to lose. But no! You can be sure that you're going to discover way more ingredients than the one that you're going to stop eating. When you have to make up for things, with the convenience of having most of your nutrients from red meat or something, to do meal planning and learn about how nutrition works a bit better, then you discover all these ingredients that were completely out of your vision before. That's a stupid metaphor, but this is exactly how I feel today. It took me way too long to get into Linux. Now, I'm like, all the things that I thought I cared about, actually, now I learned how to make them and they work way better.
Ivan:
It's a little bit like you have a new set of creative constraints that you have to work within.
Devine:
Yeah, and I'm exposed to a lot more of the history of computer culture. This, in return, it feeds into how I do things today. I don't have a specific tool to solve the challenges I have today, but I do have a lot of searching and trying the tools that are already out there, see which sticks. One of my absolute latest passions, I guess, for the best few months, has been Lisp. Now, everything has to be running Lisp and everything has to include some kind of Lisp or Scheme. But there's like 30 years of history to go through, and 30 years of tools that people made to go through. I'm super excited to have this now, this completely, like, whole new field of interest, or whole new field of things that existed all along. You know, sometimes I can think back to moments when I whined about something that didn't work properly, and I just wasn't exposed to, maybe, something somebody made in the '90s that exactly tackled that problem.
Ivan:
I feel that a lot when I'm learning more about programming, and I discover a new kind of data structure or an abstraction or research paper or something like that, that just lets you reframe the problem in a way where the solution becomes apparent. I really love that, that feeling. Like, the change in perspective giving you the opportunity to see solutions that you couldn't see before.
Devine:
Yeah. I think like for language development, language programming… the first time you make it parser and it's lightning fast, and then you realize that… instead of adapting how I work with programming, I can make something that just follows the way I think. Having this reversal of how you would do programming or computing, I find is super empowering.
Ivan:
Yeah. If your only experience with programming has come from Xcode or Visual Studio, you're really missing out on what it feels like to use tools that are designed for simplicity rather than being everything and the kitchen sink.
Devine:
OMG, if you've never experienced writing your own interpreter, then you're really missing out, because you've only ever used other people's way of doing things. It's kind of nice to be reflecting on what your intuition would want. Nowadays, I feel it's getting more and more accessible for even beginner programmers. Like, I know how to program but I don't come from a very strong programming background or anything like it. I mostly come from arts. But it's crazy that I can write an interpreter today and make a programming language the way I would expect programming languages to work. Orca comes from a misunderstanding of Tidal. Well, this is a pattern in my career that I see everywhere. I made a language that I thought was Lisp. You look at Lisp code, and you're like, "Oh, well, I see what's going on." I made an interpreter that works exactly like this and I use it all the time. I used it today. I can't really call it Lisp because it just looks like it, and it's like a misinterpretation of what Lisp is and how it works.
Devine:
I was at Jack Rusher's place in Berlin the other day. I was like, "Oh, can you look over my Lisp-y thing?" Then he would just, like, point out all the ways that I'm breaking patterns in Lisp, and how it's kind of like a weird twist of what Lisp is. But that's kind of how my mind interpreted it, and I guess how I reflect, and it's kind of nice that I was able to build it so quickly.
Ivan:
That's beautiful. There's so many things that have been made as a result of… Somebody mishears the melody for a song and it gets stuck in their head, and then they write a song based on it, and then they go and listen to the original and go, "Oh, that's not at all what I thought it was."
Devine:
I've done my share of that… I made covers of songs just based on memory, and then later I was like, "Oh, that's way off."
Ivan:
But then it led you somewhere new.
Devine:
Oh, yeah. It has value, just different.
Ivan:
Doing the expedition into the history of programming and computing — that's what makes it so exciting. There's no way you're going to learn the cultural context that led Smalltalk to being exactly what it was. So you might pick up on the ideas of Smalltalk and misunderstand them and make something and then that turns into… well, this is a bad example because that turned into Java and Java-style OO.
Devine:
Wait, Smalltalk turned into Java?
Ivan:
Well, object-oriented programming — the term and the ideas — came from Smalltalk. But what we call object-oriented these days, where it's… like, you make a class and you instantiate the class, and the class has methods, and all that kind of stuff… that's a pretty bad misunderstanding of what Smalltalk did for OO. So when you get into it, there's all the people who say, "Oh, are you talking about Smalltalk-style OO? Or are you talking about Java (etc.) -style OO? There's this new generation of people like us who are discovering that history for the first time. I think it's going to lead to some pretty interesting results where people misunderstand what came before and do something very different.
Devine:
Well, hopefully somebody looks at Orca, and misunderstands it, and does something that's amazing.
Ivan:
That is a beautiful note to end on. Devine, thanks for coming on the show and talking to me. I'm really excited to hear what people think of this episode. I think we went to some pretty "out there" places, compared to where most people in the programming research community are spending their time. As we established at the very beginning, I think there's tremendous value in looking a fair bit abroad from whatever starting point you have. So thank you for taking us there.
Devine:
Thank you so much for having me.
Music:
Ivan:
That's our show for today. Thank you again to Devine for making these past two episode with me, and especially for letting me use your music in this episode. I think it was a very fun way to capture some of the essence of Orca and share it with the audience here.
Ivan:
The next episode of the show is going to be a small episode recapping the survey that we ran on the Future of Coding Slack. I'll go through some of the survey results, and some of the answers suggesting things we should do with the community and with the podcast over the coming year. The feedback was tremendous, and I think it's really interesting as a barometer for where our community is at. That'll just be a little mini-episode about halfway through the month.
Ivan:
The next full episode, which will come out about a month from now, well… here's your hint: [excerpt from earlier in the episode] Going back to that idea of the tools being small and doing specific things, Orca not making sound of its own — I listened to an interview with Miller Puckette creator of Max/MSP and Pure data. He said that when he first started making Max, the thing that he was most excited about was the scheduler inside the engine that he wrote.
Ivan:
Join the community at futureofcoding.org/community, show notes are at futureofcoding.org/episodes/45, and I will see you in the future.