Setting up a private Matrix server.

Jan 11 2020

A couple of years ago I spent some time trying to set up Matrix, a self-hosted instant messaging and chat system that works a little like Jabber, a little like IRC, a little like Discord and a little like Slack.  The idea is that anyone can set up their own server which can federate with other servers (in effect making a much larger network), and it can be used for group chat or one-on-one instant messaging.  Matrix also has voice and video conferencing capabilities so you could hold conference calls over the network if you wanted.  For example, one possible use case I have in mind is running games over the Matrix network.  You could even build more exotic forms of conferencing on top of Matrix if you wanted to.  Even more handy is that the Matrix protocol supports end-to-end encryption of message traffic between everyone in a channel as well as between private chats between pairs of people.  If you turn encryption on in a channel it can't be turned off; you'd have delete the channel entirely (which would then cause the chat history to be purged).

Chat history is something that was a stumbling block in my threat model the last time I ran a Matrix server, somewhen in 2016.  Things have changed quite a bit since then.  For usability Matrix servers store chat history in their database, in part as a synchronization mechanism (channels can exist across multiple servers at the same time) and in part to provide a history that users can search through to find stuff, especially if they've just joined a channel.  For some applications, like collaboration inside a company this can be a good thing (and in fact, may be legally required).  For other applications (like a bunch of sysadmins venting in a back channel), not so much.  This is why Matrix has three mechanisms for maintaining privacy: End to end encryption of message traffic (of entire channels as well as private chats), peer-to-peer voice and video using WebRTC (meaning that there is no server that can record the traffic, it merely facilitates the initial connection), and deleting the oldest chat logs from the back-end database.  While it is true that there is no guarantee that other servers are also rotating out their message databases, end-to-end encryption helps ensure that only someone who was in the channel would have the keys to decrypt any of it.  It also seems feasible to set up Matrix channels such that all of the users are on a single server (such as an internal chat) which means that the discussion will not be federated to other servers.  Channels can also be made invite-only to limit who can join them.  Additionally, who can see a channel's history and how much of it can be set on a by-channel basis.

For the record, on the server I built for writing this article the minimum lifetime of conversation history is one calendar day, and the maximum lifetime of conversation history is seven calendar days.  If I could I'd set it to Signal's default of "delete everything before the last 300 messages" but Synapse doesn't support that so I tried to split the difference between usability and privacy (maybe I should file a pull request?)  A maintenance mole crawls through the database once every 24 hours and deletes the oldest stuff.  I could probably make it run more frequently than that but I don't yet know what kind of performance impact that would have.

One of the things I'm going to do in this article is gloss over the common fiddly stuff.  I'm not going to explain how to create an account on a server because I'm going to assume that you know how to look up instructions for doing that.  Hell, I google it from time to time because I don't do it often.  I'm also going to break this process up into a couple of articles.  This one will give you a basic, working install of Synapse (a minimum viable server, if you like).  I also won't go over how to install Certbot (the Let's Encrypt client) to get SSL certificates even though it's a crucial part of the process.  I will explain how to migrate Synapse's database off of SQLite and over to Postgres for better performance in a subsequent article.  For what it's worth I have next to no experience with Postgres, so I'm figuring it out as I go along.  Seasoned Postgres admins will no doubt have words for me.  After that I'll talk about how to make Matrix's VoIP functionality work a little more reliably by installing a STUN server on the same machine.  Later, I'll go over a simple integration of Huginn with a Matrix server (because you just know it's not a technical article unless I bring Huginn into it).

A piece of advice: Don't try to go public with a Matrix server all at once.  The instructions are complex and problematic in places, so this article is written from my notes.  Take your time.  If you rush it you will screw it up, just like I did.  Get what you need working, then move on to the next bit in a day or so.  There's no rush.

Point in time documentation of the Keybase Chat API

Jun 19 2017

A couple of months back I did a brief writeup of Keybase and what it's good for.  I mentioned briefly that it implements a 1-to-n text chat feature, where n>=1.  Yes, this means that you can use Keybase Chat to talk to yourself, which is handy for prototyping and debugging code.  What does not seem to be very well known is that the Keybase command line utility has a JSON API, the documentation of which you can scan through by issuing the command `keybase chat help api` from a command window.  I'm considering incorporating Keybase into my exocortex so I spent some time one afternoon playing around with the API, seeing what I could make it do, and writing up what I had to do to make it work.  As far as I know there is no official API documentation anywhere; at least, Argus and I didn't find any.  So, under the cut are my notes in the hope that it helps other people work with the Keybase API.

The API may drift a bit, so here are the software versions I used during testing:

Client:  1.0.22-20170512224715+f5fba02ec
Service: 1.0.22-20170512224715+f5fba02ec

Setting up converse.js as a web-based chat client.

Apr 09 2017

As not bleeding edge, nifty-keen-like-wow the XMPP protocol is, Jabber (the colloquial name for XMPP I'll be using them interchangably in this article) has been my go-to means of person-to-person chat (as well as communication protocol with other parts of me) for a couple of years now.  There are a bunch of different servers out there on multiple platforms, they all support pretty much the same set of features (some have the experimental features, some don't), and the protocol is federated, which is to say that every server can talk to every other server out there (unless you turn that function off), kind of like e-mail.  You can also build some pretty crazy stuff on top of it and not have to worry about the low-level stuff, which isn't necessarily the case with newer protocols like Matrix.  There are also interface libraries for just about every programming language out there.  For example, in my Halo project I use SleekXMPP because it lets me configure only what I want to out of the box and handles all of the fiddly stuff for me (like responding to the different kinds of keepalive pings that Jabber clients send).  Hack to live, not live to hack, right?  There are also XMPP clients for just about every platform out there, from humble Android devices to Windows 10 monstrosities.  However, sometimes you find yourself in a situation in which your XMPP client can't reach the server for whatever reason (and there are some good reasons, let's be fair).

Neologism: Slackpathy

Mar 18 2017

Slackpathy - noun - The phenomenon where conversations in a Slack channel are carried out using roughly 50% emoji or reaction gifs and 50% written natural language.  The term derives from the hypothesized phenomenon of telepaths sending entire thought-complexes to each other rather than streams of speech.

What is Keybase good for, anyway?

Feb 23 2017

UPDATE - 20170228 - Added more stuff I've discovered about KBFS.

A couple of years ago you probably heard about this thing called Keybase launching with a private beta, and it purported itself to be a new form of public key encryption for the masses, blah blah blah, whatever.. but what's this thing good for, exactly?  I mean, it was pretty easy to request an invite from the service and either never get one, or eventually receive an e-mail and promptly forget about it.  I've been using it off and on for a while, and I recently sat down to really mess around with it and get a sense for how it's changed and what it can do.  Plus, there's a fair amount of outdated or bad information floating around out there, and I wanted to do my part to set the record straight.

I'm not going to spend time explaining public key crypto because I wrote a pretty decent introduction to it that I give at cryptoparties.  Take a look at the PDF of the presentation; I tried to make it as painless as I could.  I want to keep this post focused on Keybase.