Structural editor

This is short document about idea called “structural editor” (or whatever it should be called). In this document I will try describe basic concepts of my vision. This contains some core concepts and ideas about some advanced features.

Motivation

Traditionally we are expressed many more complex things with text. Programming language syntaxes are optimized to be user friendly as possible while they express complex data models. Cost of nice syntax is that actual data behind syntax is much harder to understand by software. Case with modern programming languages is that usually only software that understand structure of syntax is the compiler itself. It also mostly act as black box in conversion between text representation and final assembly completely hiding machine understandable actual structure of language.

One of the most fundamental thing in this project is threat data-structure as first-class citizen over text (syntax). Text is just one single representation of that real data behind it. It’s general principle on software development to separate data from view. This project tries to apply that basic principle to developers own tools.

With improved machine friendliness we can improve lot of our UX experience especially for use cases which are naturally complex. It’s important to understand that structural editing is not about alternative for text-based or visual. It’s superset of both of them. Therefore we have currently forced to make unnecessary tradeoffs when choosing best tool for the job.

Architecture

In center of everything we have data structure. This structured data is our first class citizen. Good editor should be like our body extension to interacting with that abstract thing on memory.

Back end data model

Traditionally many used data structure models (especially serializable) are hierarchical / tree structures. Example: Json / Xml / Html / etc.

Hierarchies may be good for visualization and organizes things by single context. However most real-world use cases are more complex and therefore require some sort of graphs to be modeled more accurately. This can be seen very clearly by observing HTML model. While our UI model on web is indeed hierarchical it still has some graph properties.

• id attribute is just one to many relationship
• class is many to many relationship

However these relationships aren’t matching to our basic hierarchical model of HTML elements. Therefore we are talking about graphs and not hierarchies. We probably want of course view HTML structure as it’s normal hierarchal structure but these other relationships still need to be known by editor.

This basically gives two options: either structure need to support graphs natively or we should be able to know which value properties should be considered as reference on well defined scope. Basically native references should be a primary option over value equality. If editor won’t really understand relation correctly, it should be considered as an error.

It’s also recommended to have some sort of identification (UUID / hash?) for every object in data. This offers:

• Better diff analyze for changed data
• Better syncing
• Talking about data spots -> metadata

Supporting multiple formats

To allow support for multiple formats/languages/domains we need to define some common general model how to handle data. This mean that we have to accept some data model to as our main model which we use to express all our domains.

Component which allows working with some language is called language driver. Language driver is component which minimum requirement is to offer functionality to import/export between original representation and data format of editor. Usually we are talking about textual languages but of course source model can also be what ever like Google Protocol Buffers.

Language driver should be able to express what operations in terms of main model are legal. This can be done witch schema or maybe custom logic. It’s should be noticed that some data manipulations may become over complex if we require model to be completely valid at any point of time. Editor should allow temporal invalidity for domain rules but in terms of main model data should be always valid (although on UI level even this can be compromised).

Microeditors

Microeditor (naming inspired by microservices) is idea that instead of single “monolithic” editor which shows all the data at once we break visualization to small pieces. Single microeditor shows (usually) just small piece of structure at once. This buys us some interesting possibilities:

• We can create reusable general purpose editors like Unix tools
• We can mix many different style of viewing and modifying data. This is useful if data touches multiple domains.
• It makes easier possible to incrementally cover more use cases by adding new editors.
• 3rd party can develop (more or less) easily their own editors
• While not being actual goal, editor can act as UI platform

When saving work editors should be able to save their own metadata about data. Example:

• Box sizes and positions for diagram editors
• Potential invalid inputs for raw text editors

Working with microeditors

Since whole usage of editor is about microeditors, choosing right microeditors make huge impact for usability of final editor. One core idea behind microeditors is that editors can be chosen by user on fly. So user can define how he/she wants to view every piece of data: “I want view THAT piece of data as table and THAT as diagram”. Master editor should offer only editors which can manage given data.

In practice language driver should be able to offer some defaults for domain it’s made for. Also some microeditors may prefer some specific microeditors as their sub editor. Editor setup could be also distributed as separated configuration (with data). Still user should have a last word.

Basic procedure of microeditor

Microeditor will receive their piece of data as parameter. Microeditor offer functionalities:

• Resolve does it support given data (could be done centralized with schema)
• Render data
• Expose data points which are out of scope for editor

Table editor would expose cell data for example. These exposed data points will potentially be rendered as sub microeditors and that’s how editor nesting happens. These points should be also connected to rendering result so that subeditors can be placed to final result.

It’s recommended that exposing data points and rendering is separated and rendering merging is placed to outside from microeditor. This makes possible to use either top-down or bottom-down rendering and disable rendering of middle components (shallow collapse).

Visual editors

Raw text (json/XML) editor

Should be relative easy to implement. This could be very nice for demonstrations purposes for example. This clearly shows that structural approach isn’t exclusive for text editing. However if this is only option we have just over complex text-editor.

DOM editor-like

General structure browser. This probably will be our primary goal.

Spreadsheet like table

Quite universal for certain type of data.

Diagram style

Unreal BluePrints

Flow based programming - NoFlo

Shader forge

Hyperbolic graphs

May not be very practical but in some sense quite interesting way of graph navigation.


HTTPS://www.youtube.com/watch?v=xHvAqDuWG2M&t=6m27s
HTTPS://www.youtube.com/watch?v=pwpze3RF55o&list=PLvxNfzGYILaSAnN7G9gYO_T3PfjhIrCfM

Other

For inspiration:
Bret Victor demos (just in case if missed) http://worrydream.com/
* Stop Drawing Dead Fish HTTPS://vimeo.com/64895205
* Drawing Dynamic Visualization HTTPS://vimeo.com/66085662
* Media for Thinking the Unthinkable HTTPS://vimeo.com/67076984

Apparatus HTTPS://www.youtube.com/watch?v=i3Xack9ufYk

Functional micro editors

Editors may also expose not direct points from original data but something that they are computed from it. This make possible to make editors that change data model to fit for other editors.

• Data duplication (exposing data twice? Not actually computation) which makes possible two different editor observe same data at same time
• Drill down limits, when drilling down deep data structure, master editor need to have way to limit UI nesting
• Instance duplicate notification, since data graph can contain same piece of data under multiple paths, instance duplication on screen should be made clear
• On demand loading
• Collapse controls, collapses can be to bottom (collapse descendants), to top (“editor to fullscreen”) or middle area (collapse some range)
• Conversion between native references and value equality relations
• Data filtering
• etc.

Editor as end-user UI

Microeditors can be also tough as widgets which build traditional user-interfaces. In this approach software can be developed with get simple user interface near free. Basic microeditor set not of course offer user experience expected today even nearly but hand crafted micro editors “widgets” can be incrementally implemented.

Main editor itself as part of data structure

“Eat your own dog food”

At some point editor internal UI could be also built with same technology. This approach is quite reasonable especially for settings which are quite clear data.

Some shallow feature ideas

Version control

Version control/history features? Checkout version control system darcs: http://darcs.net/

Remote data and collaborative working

If we sync data over internet -> We have some sort of web browser.
If we can merge changes from multiple sources -> We support collaborative working.

Using external editors

What if we could at runtime choose to use editor by URI?

Technical specification to do

Coming soon