Happy new year! I'm very happy to find myself writing a second "Earthstar January update". What's happened since November?

Earthstar v10 released

The features I worked on last year were finally released in Earthstar v10. This update added attachments so that syncing videos, music, or whatever else is now possible. It added read-only shares, so that Earthstar has a more granular permission model. And finally efficient sync, powered by range-based set reconciliation method.

The website also got a big update, with new tutorials, explainers, and a new splash illustration I drew to celebrate the new release.

I'm happy with the reception this release got. We've had a steady trickle of new people entering the community Discord, asking questions and joining in on workshops, and commenting on PRs and issues on the repos.

I've also released a patch release (v10.0.1) which fixed some issues to do with attachment syncing, as well as providing more informative error messages during syncing.

CinnamonOS

I've continued exploratory work on CinnamonOS, an in-browser launcher for Earthstar applets. Practically speaking, it's an extension for Earthstar servers which gives them ability to serve applets via the browser.

My work this month has been on applet authoring and uploading. The most important thing for me is that applet authors to be able to make ordinary web apps with the tools they prefer. If that means no framework, great! If that means using a framework, also great! If your process eventually ends up with a HTML page to land on, then you're covered.

Once you've made your web app, you make a file called applet.json like this:

{
  "authorId": "org.earthstar-project",
  "author": "The Earthstar Project",
  "appletName": "Bookshelf",
  "appletSlug": "bookshelf",
  "version": "0.1.0",
  "indexPath": "index.html",
  "iconPath": "icon.png",
  "contentSecurityPolicy": {
    "default-src": ["self", "blob:"],
    "connect-src": ["*"],
    "img-src": ["data:", "blob:"],
    "worker-src": ["blob:"]
  }
}

Put your app assets in a directory called assets next to it, zip it up, and it's ready! You can then pass the .zip to a friend, or use the new applet uploader I've made to add it to a share:

This applet is then synchronised to the peers interested in the +test3 share - presumably one of which is a server with CinnamonOS running on it. That server has been configured to use the +test3 share for applets, and to serve applets uploaded by my keypair. That combined with the Content Security Specified by the applet, and served in the applet's response headers is my first sketch of mitigating malware.

All the applets on a single CinnamonOS instance are served from the same origin, so data written in one applet is able to be read from another. What I hope this means is that applet authors get to to really zoom in on the problem they want to solve, and not the dozens of other things that make a UI feel complete. A good example of this is the CinnamonOS launcher itself handler user keypairs, share, and server management for you. I expect there would also be a bunch of good applets for editing your user profile, which other applets can then pick up on to to e.g. show avatars next to posts.

Because the applets are synced between all peers, these applets aren't limited to being accessed through online servers: imagine having some kind of small wall-wart which serves CinnamonOS on the local network, or a script which starts a server on localhost, so that all these applications are available even when offline.

Dana

New Design Congress has been working on Dana, "a set of tools for navigating and understanding research material in a time of information saturation and volatility".

Seeing Dana as a resilient, distributed archival system, providing different lenses on the archived data, it's clear it has a lot of overlap with Earthstar and CinnamonOS. Enough overlap that we really have to collaborate. Dana will most likely be the first big entry of Earthstar's ecosystem, using Earthstar to store and sync archival data.

To kick off this collaboration, yesterday we held our first online 'open building session', where we ported offln, a browser extension for capturing web data into Dana, to use Earthstar's primitives instead. By the end of the workshop, we could download YouTube videos to an Earthstar share and sync them to other peers with the click of a bookmarklet.

I'm very excited about working with NDC, what their research and hard work will bring to the Earthstar project, and what Earthstar can do in return.

P2P Basel

Last week I attended P2P Basel to give a presentation on the new range-based set reconciliation method used in Earthstar, and to meet with many friends and collaborators in person. I had a really great time of it, and was really impressed by the breadth and depth of everyone else's presentations.

As for my part, I think it went well! There was a recording, which I'll share here once I get access to it and if you want to subject yourself to 20 minutes of me talking about augmented red black trees.

My takeaway from the conference is that although it can seem kind of bonkers for dozens of projects to be building different tweaks of the same ideas over and over, and it is super tempting to find new efficiencies — maybe this is what progress should look like? We're in the Cambrian explosion phase of distributed systems, and there are going to be lots of strange one-offs and dominant genuses. It's all freaky invertebrate things now, but someday it'll add up to a brachiosaurus or a cute monkey thing.

Anyway. Shout outs to the tireless organisers, Erick Lavoie, Christian Tschudin, decentral1se, as well as my dear p2panda crew, and the most excellent Aljoscha Meyer. It was a blast.

Next steps

I am finishing some administrative steps with NLnet to begin this year's new project. I'll likely begin with peer discovery and sync over LAN, and have already started my research for this.

Until next time!