Pre-alpha: Conceptual design


#1

Ind.ie Alpha: conceptual design

Ind.ie has four main elements: Heartbeat, Waystone, Pulse, and the Pulse Discovery Server.

This is an introductory post. As such, instead of delving into the details of any single component, let’s take a look at how the whole system works.

In this example, Jo, who is on the Ind.ie alpha, wants to privately send a message (and maybe a kitten photo) to Laura. Here is a simplified overview of the main steps involved:

Friendship request

First, Jo has to find Laura. Jo knows that people who are on the Heartbeat alpha all have accounts on Ind.ie, which runs an instance of Waystone.

Jo looks through the public timeline in her Heartbeat. She finds the message announcing that Laura has joined Ind.ie. She clicks on her handle (“laura”) and sends her a friend request.

The friend request is a simple JSON file. Heartbeat writes it out to a folder that Jo’s local machine shares with the Waystone instance on Ind.ie. (Instead of saying “the Waystone instance on Ind.ie, I’ll just use Ind.ie from now on.) The message contains the Pulse Device ID of Jo’s device. Pulse notices that a file has changed in a shared folder and automatically synchronises it with Ind.ie. (1)

Waystone analyses the message and understands that it is a friend request for Laura, who is a registered person on the system. To alert Laura, Ind.ie copies the message into a folder that it shares with Laura’s device. Pulse notices that a file has changed in a shared folder and automatically synchronises it with Laura’s device. (2)

In this way, Waystone acts as a trusted intermediary between Jo and Laura. Its purpose is to introduce people to one another, nothing more.

Teleportation API

We call the method of writing out a message as a file on your local device and having it synchronised by Pulse the Teleportation API.

Friendship acceptance

Laura receives the friend request and chooses to accept it. When she does, Jo’s Pulse Device ID is added to Laura’s Pulse configuration. She notifies the Waystone instance at Ind.ie about her choice via the Teleportation API. The acceptance message contains Laura’s own Pulse Device ID. (3)

(Note: in the future we will support multiple devices per person but the pre-alpha is currently limited to one.)

Ind.ie relays Laura’s friendship acceptance message via the Teleportation API to Jo. Once Jo receives the message, her Heartbeat adds Laura’s Pulse Device ID to Jo’s Pulse configuration. This closes the loop: Laura and Jo’s devices have given each other permission to talk directly and know each other’s Pulse IDs. (4)

Device discovery

Before Laura and Jo can share messages, their devices must be able to find each other. To do this, they both send their current IP addresses to the Pulse Discovery Server. (5-8)

Private messaging (file sync) via Pulse

And that’s it! Now Jo can write Laura a message, attach the cute kitten picture she found to it, and send it to her privately. The message itself, in this case, is in HTML format. Heartbeat writes out the message into a folder along with any attachments (images, etc.) and Pulse handles the synchronisation between Laura and Jo’s devices. (9)

That concludes our brief conceptual overview of the alpha architecture. Needless to say, I’ve learned a lot while implementing this in the last six months. Continue by reading about the alpha limitations of Waystone and Pulse. This should give you an understanding of the limits of the current architecture as well how we should evolve it.


Heartbeat pre-alpha release
Pre-alpha: Waystone and Pulse limitations
Pre-alpha: Technical Design
Pre-alpha: Core sequence diagrams
#2

I’d like to understand a little more about the Pulse Discovery server. I assume it knows (everyones?) ip addresses that use the service? a bit like a DNS? Is your use of the service hidden in any way? How can you trust the person who says they are Laura? Does the Pulse Discovery service act as a certificate authority?

I don’t know if this is asking too much but am interested in knowing… or is there another appropriate place in the forum to ask these questions? Thanks


#3

… mmm ok, after reading about the article on limitations… I have another stupid question: If it is P2P then what is Pulse doing in the middle? (If Pulse is syncing files between devices surely it does not need a connetion always open?) … which leads to the question if its SSL who has exchanged keys with who?


#4

Hi Peter, Pulse is a fork of Syncthing. The Syncthing forums have a lot of technical details. We’re basically using it as a black box at the moment. Mark and Stefan are porting it to Swift (once we switch to that, we will abandon our Go fork). The discovery server does know everyone’s address.

Pulse itself is peer-to-peer. The files you share are synced directly between your devices. The only reason the discovery server exists is so that you can, well, as it says on the tin, discover each other.

The SSL connection exists between the devices that are syncing.

For more information, please see the Syncthing documentation.


#5

Thanks for your replies … does seem like ‘syncthing’ should dispel my ignorance, I’ll give it a closer look. Interesting the choice of Swift … its another Apple-centric choice. Do you explain somewhere the rationale for your Apple bias?


#6

It’s not so much a bias as a core element of our strategy :smiley:

I just wrote about it here: We’re building for OS X and iOS, not Linux. This is not a bug, it’s a feature. Here’s why…