Standardize and you simplify lives: everyone learns the system only once. But don’t standardize too soon; you may be locked into a primitive technology, or you may have introduced rules that turn out to be grossly inefficient, even error-inducing.

Don Norman, The Design of Everyday Things

If you set off to design a language today, you probably wouldn’t ask, “Why text?”
It’s the implicit default. For good reason too—all our tools are now based around it. If you change this, you’d have to reinvent editors, compilers, debuggers, version control, and a lot more. But making a new language without considering its medium is like making a submarine without considering water.

Here's an analogy—walkie-talkies send and receive on the same frequency. Only one device can transmit at a time—you can't listen while you're talking. This affects communication in two ways. First, users transmit short messages to minimize use of the shared frequency. Second, they perform communication control themselves, like saying "over" to indicate that the frequency is open again. You don't need to do this in phone calls because everyone can talk at the same time.

A language lives within the constraints of its medium. The medium changes the way we communicate. And here, I want to do the inverse—we know our communication patterns in text. Let's use that to design the perfect medium for programming.

I'm going to look at text from the perspective of humans and machines. To make my points I have used examples from languages I’ve used a lot (C, Python, Go). I don’t mean to imply that these languages are better or worse than others. For my cause, the choice of language doesn’t change the conclusion. CSVs and C++ both inherit the same properties from their communication medium.

Also, this isn’t leading up to visual programming. That’s not where this is going.

Token identity

Escape characters are crucial for parsers. We need these because data and syntax live in the same space and parsers can't tell them apart. Parsers detect token boundaries by a change in character sets. On the other side of this are characters that you can't use in your identifiers—mostly whitespace and operators. Whitespace looks like a token separator to parsers, not data for the identifier. A variable containing an operator in its name would make parsing ambiguous.

We could get rid of these restrictions completely if we stored token boundaries. This would create an obvious distinction between data and syntax for the parser. This is important because you should have the ability to use unconventional naming if it makes the most sense for your application. Whether you use this ability is a decision that should be made by you and your team, not the parser.

Most languagesdon't allow the first character of an identifier to be a number. The parser assumes that a numeric literal is coming up when it sees a number. Reserved words can't be used as identifiers either. It's easier for the parser if keywords have a fixed meaning. In all these cases, notice how you have to manage your tokens for the convenience of parsers. You end up using underscores (or some other convention) to avoid breaking their rules.

As with token boundaries, we could lift these restrictions if token types were explicitly encoded into the source too. It's weird that we give parsers partial information and ask them tofill in the gaps later. Token boundaries and types—you have this information when writing code, but you can't explicitly give this to the parser. Doing so in text would clutter it with token lengths and types. Instead, we pass token identities implicitly using grammar and accept the tradeoffs.

Explicit token identities would also solve code injections everywhere by default, but that's for another time.

Aesthetics

In Python, docstrings and triple-quoted strings have similar syntax, but different behavior.

There is a conflict—the whitespace can belong to either the code block (as indentation) or the string literal (as data); one interpretation isn’t more correct than the other. The root of this conflict is that the whitespace performs two functions—data, and aesthetics. How do you pick the correct one?

A common workaround is to remove whitespace at runtime. You improperly store a string and spend CPU cycles to clean it up at runtime. Worse, if this string is in a loop or function— you either repeat the clean-up, or initialize the string outside the block. A syntax problem forces you to change program logic. Just to be clear—the performance penalty isn't my main issue. I want you to see how an aesthetics problem gets solved by logic, when these two shouldn't be linked at all.

This isn’t exclusively a Python problem though. It’s a text problem. Multiline strings in Go, Javascript, and many others work the same way. Consider YAML multiline strings where this problem is solved with more syntax. You have six ways to handle newlines and an option to set indentation level explicitly in case your string starts with spaces.

Whitespace comes with a lot of cognitive load. I just talked about interpretation of whitespace. Line endings differ across platforms. Style guides like PEP-8 have extensive suggestions on whitespace use. Look at the feature requests of any language and you will find one asking for support for someone's preferred style.

Encoding program structure unambiguously for parsers while looking aesthetic and readable—these are inherently conflicting goals. More syntax generally means more information about the program structure. Readability generally implies less syntax. Any amount of syntax that you choose for your language will be a tradeoff between these two goals.

Moreover, to work together, we only require a consensus on program structure, not style. Users have a variety of hardware, software, and (subjective) personal preferences. One style cannot satisfy all of them. We enforce formatting rules because text doesn't allow style to be separated from structure.

In the previous section, I talked about the boundaries between parts of the language (tokens). Here, I'm talking about separating program and data, and how there's ambiguity in that boundary (which sometimes comes from style). Take CSVs for example. You can choose a field separator other than a comma. Some tools can guess what that separator might be (to resolve ambiguity). The fields themselves might be escaped and padded, so the program then translates them into the correct form. These extra steps wouldn't be required if the program and data were unambiguously separated, and writing tools would become much easier. The machines don't care about aesthetics at all. They have to filter through aesthetics to get to the data.

Metadata

Comments are metadata. There are many uses for them. We have conventions for tagging them, and tools like ESLint, Pylint, and Javadoc define their own structure. Every language has comments. Comments can store any amount of metadata, encoded in any format, and can be placed anywhere in code (at least the block variants).
However, comments cannot be mapped to language entities. For humans, this relationship is often apparent through visual cues, but this information does not exist in the data. Tools have to find their own way to associate comments with code.

Unlike comments, Python docstrings can be mapped to a module, function, class, or method definition. This relationship is encoded in the program. However, docstrings and comments are disconnected from the code they document in some ways. If a variable or function parameter changes, its reference in the documentation has to be updated manually.

Attributes in C can be unambiguously mapped to almost anything, but these are meant for one-word tags. They are suitable for deprecation warnings and compiler flags, but you wouldn't use them for longer content like documentation, or even TODOs.

Identifiers carry metadata in some languages. Capitalized names are public in Go. Double-underscored names are reserved in C.Underscores for name mangling in Python. Names with an underscore prefix are private, reserved, or ignored, depending on the language. This pattern, where names carry extra information, is used to make syntax less apparent. In Go, this was preferred over access modifiers.

Metadata comes in many forms, but there is no generic way to encode arbitrary metadata into code. We've seen tools extend comments, so this is clearly a necessity that languages do not meet. Another problem with comments and other metadata syntax is that they are entangled with program logic. Even when editors style them differently, they can create clutter in code.

Files

Implementation

A weighted directed graph (hereafter: graph) is the perfect substrate for program structure. Language entities like tokens, functions, and classes are stored as vertices. Vertices have types and boundaries, which solves the token identity problem completely.
Edge weights indicate the relative order of successors. These are used to store ordered collections like struct fields and statements within functions.
Edges with null weights represent unordered relationships like metadata, classes inside modules, and functions inside classes.

The graph is pure data—it has no aesthetics. Editors are free to render them however they want. Users can pick the style they want without affecting others. Style and aesthetics can evolve independently of program structure. Something I want to experiment with is levels of detail for syntax. Type casts, namespaces, access modifiers, and metadata aren’t relevant when I just want to understand the high level program logic. What I’m looking for when reading code varies and one syntax can’t possibly serve all needs. This kind of design is only possible when the data is disjointed from style.

Edges signify a "has" or "contains" relationship. This is perfect for arbitrary metadata that tools can associate with language entities. Edges exist in their own space, which means that you can add any amount of metadata without affecting other parts of the program, like logic. A function vertex can have documentation, linter declarations, and deprecation tags, without ever getting in the way of its body.

Hierarchies are stored using edges too. This lets me remove files and all issues that come with file linearity. Instead of files, there are vertices for modules, classes, functions, etc. Vertices can be fetched on demand and there's no need for an indexing pass to resolve references. Parsers can start parsing from anywhere. If you start from main(), you wouldn't even parse dead code.

Edges create references to vertices. This makes diffs more meanigful, because you only need to change the vertex in one place to update all its references. Think about how all calls to the function have to change when you rename a function in text. You wouldn't need to do this in a graph. More broadly, the graph replaces this reference-by-name pattern in text that you see in linker scripts, YAML anchors, and documentation comments, where you have to update all references when the source changes.

Another great thing about the graph is that edges partition grammar rules. Errors in one part of the graph are isolated to that sub-graph; parsing is never obstructed, and you can add new features to the language without affecting existing ones. Remember most vexing parse in C++?

Every app that reads or modifies the graph is a client. Editors, compilers, linters, language intelligence tools, etc. all get access to the program structure by performing CRUD operations on the vertices and edges of the graph. Clients can ask the server to notify them through events when vertices or edges of specific types are modified. This is useful for a documentation generator that only needs to do something when a documentation sub-graph has been modified. Or for a compiler that doesn't need to rebuild if only the documentation has changed.

Working together

The graph acts like a network-shared virtual filesystem in the userspace.

This is great for setting up a remote development environment. Compilers, language intelligence, and other tools reside in one place that all your machines access through the graph server. VS Code Remote Development uses SSH to make this work, but change notifications on network filesytems have their own difficulties, especially across operating systems. The graph is platform-independent—clients get a consistent environment on every host, and change events work the same on all OS. A Linux client will receive change events even when a Windows or a Mac client modifies the graph. The graph gives them a way to work together seamlessly.

Add an authentication layer and you can share this environment with your team for some real-time collaboration. The graph's event channel can be used to mark the cursor positions of all contributors, and change notifications tell editors when code has changed externally. VS Code Live Share and Zed's multiplayer editing offer this in their editors, but the graph is perfectly positioned for real-time collaboration to be a part of the protocol. Unlike text, vertices are isolated from each other in some sense. For example, when you add or delete text, everything that follows moves, and the cursor has to be shifted accordingly. Vertices don't share the same physical space, so this adjustment isn't necessary.

Grand Unified Data Bus

Graphs can represent data in Blender, KiCad, Godot, digital audio workstations, word processors, and many more. At one level of abstraction, these apps are no different from programming.

The Graph Model presents an opportunity for interoperability—you make one generic tool that works everywhere. The same issue tracker that you use for programming works with your PCB designs and video projects. The same documention tool is compatible with game engines and 3D software. Notably, these apps can add scripting support by exposing their internal structures. Blender, for instance, wouldn't need to embed Python—it could integrate with it via the graph. KiCad is already implementing a similar architecture with their IPC API.

In Working together, I talked about how users can share their work. Those benefits extend to all apps that adopt the model. The server only sees vertices and edges. You make the collaboration system once, and all apps can use it. LibreOffice, for example, could let your team share documents and collaborate in real-time over the graph.

Lastly—version control. Many of these apps save state in binary files. Some use text files that may as well be binary. They'd be easier to version control on a graph, where binary data can be broken down into smaller, more meaningful chunks.

The Graph solves everything

Text drives the design of parsers, which drives the design of languages. Data in text isn't clean, and no amount of computer magic is going to change that. Text is holding us back.

The Graph abstracts low-level communication and simplifies parsing. It gives language tools clean data to work with, and frees them up to solve real problems.

This is the future of programming.