The latest updates to my .plan file have been pushed.  The usual warnings apply.

quantum documentation inconsistency principle - An axiom of corporate life in which there will always be more than one piece of documentation for something, all of the documentation will be measurably contradictory, and none of the procedures will work anymore.  See also clandestine institutional knowledge.

A couple of years ago I spent some time trying to set up Matrix, a self-hosted instant messaging and chat system that works a little like Jabber, a little like IRC, a little like Discord and a little like Slack.  The idea is that anyone can set up their own server which can federate with other servers (in effect making a much larger network), and it can be used for group chat or one-on-one instant messaging.  Matrix also has voice and video conferencing capabilities so you could hold conference calls over the network if you wanted.  For example, one possible use case I have in mind is running games over the Matrix network.  You could even build more exotic forms of conferencing on top of Matrix if you wanted to.  Even more handy is that the Matrix protocol supports end-to-end encryption of message traffic between everyone in a channel as well as between private chats between pairs of people.  If you turn encryption on in a channel it can't be turned off; you'd have delete the channel entirely (which would then cause the chat history to be purged).

Chat history is something that was a stumbling block in my threat model the last time I ran a Matrix server, somewhen in 2016.  Things have changed quite a bit since then.  For usability Matrix servers store chat history in their database, in part as a synchronization mechanism (channels can exist across multiple servers at the same time) and in part to provide a history that users can search through to find stuff, especially if they've just joined a channel.  For some applications, like collaboration inside a company this can be a good thing (and in fact, may be legally required).  For other applications (like a bunch of sysadmins venting in a back channel), not so much.  This is why Matrix has three mechanisms for maintaining privacy: End to end encryption of message traffic (of entire channels as well as private chats), peer-to-peer voice and video using WebRTC (meaning that there is no server that can record the traffic, it merely facilitates the initial connection), and deleting the oldest chat logs from the back-end database.  While it is true that there is no guarantee that other servers are also rotating out their message databases, end-to-end encryption helps ensure that only someone who was in the channel would have the keys to decrypt any of it.  It also seems feasible to set up Matrix channels such that all of the users are on a single server (such as an internal chat) which means that the discussion will not be federated to other servers.  Channels can also be made invite-only to limit who can join them.  Additionally, who can see a channel's history and how much of it can be set on a by-channel basis.

For the record, on the server I built for writing this article the minimum lifetime of conversation history is one calendar day, and the maximum lifetime of conversation history is seven calendar days.  If I could I'd set it to Signal's default of "delete everything before the last 300 messages" but Synapse doesn't support that so I tried to split the difference between usability and privacy (maybe I should file a pull request?)  A maintenance mole crawls through the database once every 24 hours and deletes the oldest stuff.  I could probably make it run more frequently than that but I don't yet know what kind of performance impact that would have.

One of the things I'm going to do in this article is gloss over the common fiddly stuff.  I'm not going to explain how to create an account on a server because I'm going to assume that you know how to look up instructions for doing that.  Hell, I google it from time to time because I don't do it often.  I'm also going to break this process up into a couple of articles.  This one will give you a basic, working install of Synapse (a minimum viable server, if you like).  I also won't go over how to install Certbot (the Let's Encrypt client) to get SSL certificates even though it's a crucial part of the process.  I will explain how to migrate Synapse's database off of SQLite and over to Postgres for better performance in a subsequent article.  For what it's worth I have next to no experience with Postgres, so I'm figuring it out as I go along.  Seasoned Postgres admins will no doubt have words for me.  After that I'll talk about how to make Matrix's VoIP functionality work a little more reliably by installing a STUN server on the same machine.  Later, I'll go over a simple integration of Huginn with a Matrix server (because you just know it's not a technical article unless I bring Huginn into it).

A piece of advice: Don't try to go public with a Matrix server all at once.  The instructions are complex and problematic in places, so this article is written from my notes.  Take your time.  If you rush it you will screw it up, just like I did.  Get what you need working, then move on to the next bit in a day or so.  There's no rush.

clandestine institutional knowledge - The phenomenon in which everybody knows the documentation is wrong and people are so pissed off at said documentation that they don't ever bother to try to fix it.  Instead new hires have to play Indiana Jones to find the two people left in the organization who have any working knowledge of the thing and beg to be trained up so they can actually do their jobs.  Normally, the newly trained individual doesn't bother to update the documentation, either.

footnote: Most of the time, nobody has the access to update the documentation anymore, which is why nobody ever bothers to fix it.

It doesn't seem that long ago that I put together a Pi-Top and started tricking it out to use as a backup system.  It was problematic in some important ways (the keyboard's a bit wonky), but most of all the supported respin of Raspbian for use with the Pi-Top was really, really slow and a bit fragile.  While Windbringer was busy doing a full backup last week I took my Pi-Top for a spin while out and about, and to be blunt it was too bloody slow to use.  At first I figured that the microSD card I was using for the boot device was one of the lower-quality ones that bogs down once in a while, but that turned out not to be the case.  Out of desperation I started looking into possibly upgrading the RasPi in that particular shell to the latest and greatest version, which I happen to have received as a Yule gift last year.  Lo and behold, I was not the only person to think along these lines. (local mirror)  While the article in question talked at some length about the hardware challenges involved (mostly due to the different arrangement of connectors) the software part was the most valuable to me because it answered, concretely and concisely, how to get unmodified Raspbian working with a Pi-Top's unusual control hardware.  So that this information doesn't get lost in the ether I'm going to write up what I did.

tumbleweed mode - noun phrase - The phenomenon in which all official support forums for something are either abandoned (no activity for a protected period of time), or any posts that aren't lowball questions (such as "Where's the FAQ?" or replies to release announcements) are utterly ignored (meaning, actual technical support questions).

I'm writing this article well before the year 2020.ev starts, mostly due to the fact that Twitter's search function is possibly the worst I've ever seen and this is probably my last chance to find the post in question to refer back to.

Late in November of 2019.ev a meme was going around birbsite, "Please quote this tweet with a thing that everyone in your field knows and nobody in your industry talks about because it would lead to general chaos."  Due to the fact that I was really busy at work at the time I didn't have a chance to chime in, but then an old friend of mine (and, through strange circumstances, co-worker for a time) told an absolute, unvarnished truth of the telecom industry: "Telecommunications as a whole, which also encompasses The Internet, is in a constant state of failure and just in time fixes and functionally all modern communication would collapse if about 50 people, most of which are furries, decided to turn their pager off for a day."

I don't know of any words in the English language to adequately express how true this statement is.  He's serious as the proverbial heart attack.  For a brief period of time, one solar year almost to the minute in fact, I worked for a telecommunications company in Virginia that no longer exists for reasons that are equal parts fucked up and illegal.  The company was bought out and dismantled roughly a year after I escaped by Zander's employer at the time, and seeing as how this was about fifteen years ago as you read this, I guess I can talk in public about it.

tl;dr - If you value your physical and mental health, don't work in telecom.

Well, Happy New Year, everyone.  It's now 2020.ev, we're into the third decade of the twenty-first century.

I'm not sure what we're supposed to do now.  Hell, I'm not even sure of what to do with myself this afternoon.  I guess grab whatever downtime we can get before going back to work/school/whatever.

There have been quite a few people joking about bringing back the roaring 20's, with all sorts of memetic payloads (some silly, some not).  Personally, I wouldn't mind seeing the the Invisibles' take on the 1920's make something of a comeback, but what do I know.  Me being me, of course the first thing I thought of was embracing a little more of the cyberpunk in our world because, hey, why not, anything to stay afloat in a world where getting sick for a week can make the difference between having a roof over your head and destitution.

I know, I'm on a bit of a downer right now.  One part being at loose ends, one part feeling age in my hearts, one part... how in the hell did we make it to 2020?

I don't know.  I don't have one of those "best of 2019" or "best of the 201x's" playlists that folks have been passing around.  I don't have any sort of brilliant evocation to give, or inspirational words to say.  No major announcements to make.  I don't even have any public wishes of "please don't let this year suck" because those do about as much good as thoughts and prayers.  I'm just some schmuck trying to figure out what to do with my life, and maybe make the world a little better in the process.

Happy New Year, everyone.  Let's try to do things a bit better.

evolving situation - noun phrase - A situation where, if all hell hasn't broken loose yet it's well on its way.

UPDATE: 20191230 - Uploaded a copy to my Peertube account.

From time to time I carp about how generally lousy our bandwidth is out here.  Verizon (our CLEC in the Bay Area) has all but given up on maintaining their infrastructure out here, aside from the bare minimum to keep the copper from turning to verdigris.  They gave up on deploying fiber some years ago (local mirror) some years ago, and from the poking around I've done on their side of the fence, their general stance in the Bay Area appears to be "Get everyone on celllar so we can ignore the rest of the network."  Which sucks and does nobody but Verizon's shareholders any good in the long run.

Anyway, after yet another afternoon wasted on the phone with tech support because our speed fell to pre-dialup speeds for reasons unknown, I decided to take the bull by the horns and put some old skills to work.  Out came the fox and hound and my old lineman's test set, and I set about figuring out which lines in the fist-sized morass of ancient wiring outside, if any, were actually hooked up.  The way a fox and hound works is, you clip or plug a tone generator (the fox) into the line you want to trace, and you use a matching inductive probe (the hound) to listen for the sound.  Telephony cables are almost never insulated so you don't need to touch the copper directly, the faint EM field around the wire is sufficient.

I was able to trace the line successfully, but in so doing I found out why our bandwidth was so terrible.  Thankfully, after demonstrating the problem to the contractor that Verizon sent out, we were able to work together to not only rip out the dead cabling outside, but mostly resolve the interference.