Deep dive into multiwriter hyperdb


#1

Today, I’m doing a deep dive into multi-writer hyperdb (the technology that will be at the heart of what we’re currently calling “indie site”) by dissecting Jim Pick’s dat-shopping-list demonstration (blog post, editable/forkable online demo; glitch).

I thought I’d document my process.

First things, first:

  1. I fork the git repo on GitHub and clone it and add the original as an upstream remote:

      git clone git@github.com:aral/dat-shopping-list.git
      git remote add upstream https://github.com/jimpick/dat-shopping-list.git
    

    Always fork a repo even when you’re just looking at it. Why? Because:

    a. It’s easy to do

    b. If you find a bug, etc., you can easily create a branch and submit a pull request (i.e., you will help strengthen the commons)

    c. If the original repository goes away, you still have a copy. If it‘s a repository that I am going to be using in a project, I’ve also started to get into the habit of forking it on our own GitLab instance (source.ind.ie) – see https://source.ind.ie/forks. I’m calling this “fork-first dependency management” because it sounds just pretentious enough to maybe get traction in the web community :wink:

    Seriously though, git is decentralised. It’s a great shame that we’ve re-centalised it with GitHub. Every time you fork elsewhere, you’re again strengthening the commons.

  2. Then, to get a feel for the culture of the project, I open up the project.json file and look through the dependencies:

    cat project.json | grep dependencies -C50
    

    I open each dependency …

[YAK SHAVING] OK, why am I doing this on Discourse, when I just demonstrated how you can live blog on the peer-to-peer Web? I’ll be right back*, I just need to set up my own live blogging system (start time: ~9AM, Friday, June 15th)…

Follow along live on this Mastodon thread: https://mastodon.ar.al/@aral/100207852262520843

And, when it’s ready, all my live blogs will go on https://live.ar.al


#2

So this looks great and the blog is good but am still a little lost

HyperDB will be integrated in Hyperdrive to allow for multi-user collaboration with Dat archives.

This line sounds like what I will require. But am a little lost.

Dat is the distributed network?
HyperDB is a database that works on Dat
What is Hyperdrive?

Off to listen to DATCast podcast see if I can glean anymore.

But basically I want to use a front end Vue.js with HyperDB for multi user collaboration … so I guess like all good things I better try and build that todo list app first.


#3

Hey Adam,

Check out the DAT shopping list example by Jim Pick.

Aral


#4

:+1:t3: will do, but I think i will need a tutorial or two to really get my head round what is doing what


#5

Good stuff, Aral!

Somewhat tangential: Have you considered asymmetrical sync, where everything is not stored everywhere? It is something I have been looking for, because I always seem to have more photos than hard disk space on my laptops… Maybe not applicable in your case.


#6

Hey Erik, yeah, definitely! (Imagine if we had to download every web site fully to browse the Web; it would be a nightmare!)

In DAT, the concept you’re looking for is “sparse archives” :slight_smile:


#7

Ah, should have done some homework :wink:

Even storing everything I ever posted on my phone could be a nightmare. Photos, videos…


#8

Also, on the topic of protocols you might want to have a look at, how about Zot? It’s the one powering Hubzilla and seems to have interesting properties: https://project.hubzilla.org/help/developer/zot_protocol