Future of Coding

Steve Krouse Research Statement

May 3rd, 2019

End-User & End-Community Collaborative Programming

It is a missed opportunity every time someone wants to customize the software they use but cannot. It is incredibly confining to be unable to fluidly mix and match pieces of functionality to better suit our purpose. It is a unnecessary drudgery when someone wants to leverage automation but cannot. It’s so limiting that the only way to justify the inordinate cost of software features is mass adoption. It is so stifling to have to use cookie-cutter collaboration tools within our teams and communities. It is a daily tragedy that millions of ideas to improve our individual and collective virtual worlds aren’t even conceived: people don’t waste time thinking up improvements they have no possibility of causing.

As impactful as computers have been, they have so much unrealized potential. As Seymour Papert says, “The computer is the Proteus of machines. Its essence is its universality, its power to simulate. Because it can take on a thousand forms and can serve a thousand functions, it can appeal to a thousand tastes.” Yet we have less agency over the infinitely fungible pixels of our screens than the material atoms of our physical world. We have to content ourselves with lowest-common-denominator software on a device that is flexible enough to run any system we can dream up. As Alan Kay laments, “the real computer revolution hasn’t happened yet.”

My aim is to make software fully malleable, empowering all users of computation to better augment themselves and their communities with high-quality,bespoke tools, while at the same time leveraging deeper levels of open-source collaboration to share the best innovations as widely as possible. As a secondary effect, I hope that empowering people to customize their virtual worlds will manifest as a Precisionland (a la Papert’s Mathland) that will be a natural place for billions of people to assimilate powerful ideas and other augmented thinking abilities.

Spurred by the recent death of Google Inbox (deemed a failure for attracting millions, not billions of users), I can imagine one of the most initial uses of this system would be for people to build highly-customized email clients for themselves. Most people would likely start by building off of another’s email app as a template and start customizing from there. After messing with the look-and-feel (colors, layout), users could mix and match their favorite common email features that other users have created, such as tags, filters, snooze, or open tracking. They would be able to try out others’ email innovations, such as a gamification system. If they are creative, a user might implement their own email workflow ideas, such as automatically suggesting from one of various template responses based on machine learning of past responses.

Some of the biggest gains of a malleable computing medium are to be found by blurring the lines between tools that are now walled gardens. For example, my favorite part of Google Inbox was that it was a combination of an email client and a todo list. In a malleable software medium, one could improve on this idea by combining their favorite aspects of task management software and email clients. For example, I am a big fan of Workflowy’s infinitely nestable bulleted list structure. I would like the ability to fully embed live email conversations within such a structure. Why shouldn’t we be able to take an interactive email conversation and paste it deep into another structure? I can even imagine drafting emails right from within my todo list. But of course, I’d want to pick my own WYSIWYG text editor for the email drafting. I’d likely also add my calendar into this system so I can fluidly drag tasks over to schedule them.

But there’s only so much malleability to be had by building on top of rigid structures, even open protocols like email. A fully malleable software medium would enable a community to continuously innovate on their shared virtual space, while allowing for individual personalization. We want to empower individuals within a community with as much customization power as possible within the confines of the group structure. For example, customizing look-and-feel of a single person’s interface to a group chat app should be possible without affecting anyone else’s interface – unless of course they choose to adopt the changes voluntarily. More fundamental changes could require group coordination. For example, a user of a group chat app could propose the ability to send a message in reply to an existing message, so that messages could nest arbitrarily deeply. However some users might prefer a more linear conversational structure and object to the change. A compromise might be reached by adding in the reply-to-message feature at the data-level, and allowing for entirely different views on the data. Those in favor of a linear, time-based flow of messages could collapse the hierarchy to a single stream of messages with links added to nested messages so they can get the context if they wish.

Theory of Change

While it’d be a mistake to too closely pattern myself on the successes of the past, it’s also important to be explicit about my theory of change, or what lever I plan to pull, and how I imagine its effect taking shape. Large, positive innovations in technology have come in all shapes and sizes:

Academia Unbundled

My current strategy is to do an “unbundled” version of academia, mixed with a strong collaborative circle on the Internet. Down the line, I see possibilities to either spin-off left into a non-profit or right into a startup. I don’t have any plans to obtain military, NSF, government, or other grant funding. I’d definitely be open to working at an R&D lab, like Protocol Labs, but that’s not currently on the table.

What I mean by “academia unbundled” is copying many of the academy’s practices but without formally enrolling anywhere. I plan to copy:

By doing academia in this self-styled way, we avoid:

But also have to do without:

5-Year Goal

The goal of my “PhD thesis” is to prototype and demo a system that plausibly points towards a vision of malleable, empowering, and collaborative computing. Like Douglas Engelbart did at “The Mother of all Demos”, I aim to “deal lightening from both hands.” The demo will show:

After watching this demo, people should be inspired to think up all the ways they could customize or build their own tools to better suit themselves and their communities. They should return to “normal life” dissatisfied with a newfound sense of disenfranchisement. They will yearn for the ability to put their newfound continuous stream of ideas into practice. They will also be awed by how much better and cheaper software could become if we could unleash full power of collaboration this vision points to (akin to the wave of innovation powered by open-source software).

As long as we’re dreaming here… This demo would sit at the top of Hacker News, r/programming, and other tech sites for the day, getting thousands of upvotes and comments. The paper that goes along with this demo, co-authored with Jonathan Edwards, would get accepted to a reasonably prestigious conference, such at Onward!. It would inspire other work and be cited well. A true marker of success would be if people “stole” the work and incorporated it into their systems. Of course the dream would be that eventually such a system would eventually shift from a ‘neat demo’ into a living system that eats the world.

This dream is closely related to the field of end-user programming (Lieberman), including the most successful end-user programming environment the spreadsheet (Forms/3, Nardi), HyperCard, and the Smalltalk language and operating system. Recently, Ink & Switch published a survey of this space; their Farm has similar goals to personalize software. Recent projects such as Webstrates and TiddlyWiki are also relevant.

Additionally, there has been a lot of work in industry on “app builders” such as Bubble and DropSource, website builders such as Webflow, prototyping tools such as Facebook Origami, BaaS platforms such as Firebase and Parse, and workflow automation tools such as IFTTT and Zapier. While democratizing various parts of software creation, these tools mostly focus on personal as opposed to collaborative creation.

Modularity

The key is allowing innovations to be shared as self-contained units that can be added a la carte to existing systems. Systems built of interchangeable parts, such as the car, lend themselves to piecemeal modifications over time. If you want a faster car, you can install a faster engine instead of buying a whole new car. Because’s today’s software isn’t created to be piecemeal replaceable, we are stuck either hand-tuning the engine (creating a personal fork of the whole project, which is a genealogical dead end as far as innovation is concerned) or buying an entire new car (switching to another project). The more piecemeal a system is, the better it is able to leverage a wider swath of innovation. We speed up innovation by parallelizing it on independent parts: instead of buying a new engine, install a higher flowing exhaust. Piecemeal systems also allow for the greatest amount of personal preferences to be brought to bear upon the design trade-offs: instead of choosing between a couple hundred models of car, mix-and-match the 30,000 parts of a car to create a virtually infinite array to choose from.

The functional programming community has shown how we can use various language features, such as higher-order functions and lazy-evaluation, to modularize programs into smaller reusable pieces. It seems like the key is maintaining equivalence relations. Thus the importance to remove assignment statements which kill our ability reason about equivalences.

Functional Reactive Programming has shown how we can recast an irreducible nest of callbacks mutating global state into a self-contained, declarative entities.

Multi-tier Programming

Proper modularization of multi-node software must allow for individual modules that cross tiers. For example, a collaborative counter button that counts the click of any user on any computer could be moved to another location while retaining its two-way connection to other users’ counters. At the same time, there has to be the ability to copy the behavior of a counter button but disconnect it from the past and future clicks of the original copy’s users.

PX of Programming

There are a number of “programmer experience” features that will enable a larger number of people to happily use such a system to its fullest advantage. For example, it’s important that the system be always running, with no distinction between edit mode and run mode.

Next, one could imagine a system where pieces of an interface could be replace with other pieces the expose the same type. For example, a color-picker would expose a Behavior Color, so you could in theory replace this part of the interface with an expression, such as constant t -> blue or dynamic color t -> rgb t t*2 t*4. It seems like there’s a interesting mirror between GUIs and expressions. For example, a pattern match expression could instead be a GUI. A number literal could be a scrubber.

TODO

Draft 1 Feedback (JE) and my responses

This is a great vision statement! To turn it into a research proposal you need to drill down a level to problems and approaches. What are specific problems you need to solve to achieve your vision?

The vision is composability, so my approach is to build upon the successes of the FP community that augmented our ability to compose programs, such as managed memory, first-class functions, lazy evaluation, etc, along with its avoidance of mutable state and delicate handling of all side-effects. I will use the phrase denotative programming as a synonym for the original vision of FP, which prioritized this composability.

What new approaches do you want to propose to solving those problems? An approach is not a complete solution, it is an insight or general direction that could lead towards a solution, which hasn’t been done before. Note that it is OK to not yet have approaches to all your problems - finding them is part of the research. But you need to show that you have at least one new take. That’s hard! Writing this proposal should feel like you are pushing yourself to a higher level.

One way to elucidate your problems and approaches is in comparison to others: why haven’t we already attained your vision? What is missing from existing solutions? Why haven’t the visionaries you cite (Engelbart, Kay) achieved it? Another thing that might help is to pick one problem as your first priority and drill deepest into that one.

Past approaches haven’t sufficiently focused on composability, and had the successes of the FP community to build upon.

Question: isn’t malleability a subset of creativity? Focusing on the former suggests you are going for some sort of adaptation layer rather than the full ability to create from scratch. If you think indeed malleability is essential to your approach then explain why.

“If you wish to make an apple pie from scratch, you must first invent the universe.” ― Carl Sagan, Cosmos

There is no creating. From the first valid expression you put together, it’s modifying from there on out. At bottom there are primitives, but most often you are going to be creating by combining and customizing non-primitives, pre-exsiting parts created by the community. Why create from scratch when you can use and customize community-created pieces of functionality?

You mention FRP almost in passing, yet it seems central to your thinking so far. If that is still the case you should make a fuller argument about how it is an approach to your problems.

Questions

Let’s build a community chat app. Let’s start as simply as possible:

  1. Displaying a single message Message -> HTML
  2. Displaying a list of messages, sorted by time sent [Message] -> HTML
  3. An input box to draft a message, where the enter key sends it HTML -> Event Message

A few perspectives to consider:

Distributed computing

  1. How do messages get sent from computer to computer? Do we imagine the algorithm from the perspective of one node or the whole system? Or both?

1a. Here’s a wacky idea: the program is a distributed system. Anyone who wants to become a node can allocate an amount of hardware (memory, network, cpu, keyboard, etc) to the program. We then use distributed systems theory to get all the properties we need for the program. In other words, the world is one big computer (distributed) with a bunch of scattered hardware (I/O) devices. The advantage of such a scheme is composability in the sense that a program is a horizonal slice. You can put it wherever you want and it will figure out the communication and how to leverage the hardware given it in various places.

  1. How do we model the amount of time it takes for a message to be received?

2a. The simplest answer is: don’t. You can get an event from a remote piece of hardware as an event. If you want local time as well for some reason, you can add that in to the event. (We can elucidate this by rendering both times in the message list.)

Variants

  1. What if one user wanted to customize something locally? What if they wanted to share the customization with another? What if the both further customized and wanted to merge?

  2. What if some or all participants wanted to modify the type of message to include the sender’s name or other data model changes?

  3. Could there be a place users could browse through the styles of chat apps that could be applied to theirs? Could we do this even if much of the underlying architecture between apps is quite different?

  4. Could users import data model changes from other chat apps?

  5. Could a user separate the app into functioning pieces? For example, what if a user wanted to create a personal dashboard that included a window showing certain subset of message without the input box for sending messages?

Immutability

  1. Can we edit code fully immutably? That is, without defining terms to as mutable mappings of strings to expressions? For example, can an expression allow for arbitrary or limited changes to itself?