I've been promising myself that I'd do a series of articles about tools that I've incorporated into my exocortex over the years, and now's as good a time as any to start.  Rather than jump right into the crunchy stuff I thought I'd start with something that's fairly simple to use, straightforward, and endlessly useful for many purposes - a wiki.

Usually, when somebody brings up the topic of wikis one either immediately thinks of Wikipedia or one of the godsawful corporate wikis that one might be forced to use on a daily basis.  And you're not that off the mark, because ultimately they're websites that let one or more people create, modify, and delete articles about just about anything one might be inclined to by using only a web browser.  Usually you need to set up or be given an account to log into them because wiki spam is to this day a horrendous problem to fight (I've had to do it as parts of previous jobs, and I wouldn't wish it on my worst enemy).  If you've been around a while, when you think of having a wiki you might think of setting up something like WikiWikiWeb or Mediawiki, which also means setting up a server, a database, web server software, the wiki software, configuring everything... and unless you have a big, important project that necessitates it, it's kind of overkill and you go right back to a text file on your desktop.  And I don't blame you.

There are other options out there that require much less in the way of overhead that are also nicer than the ubiquitous notes.txt file.  For the past couple of years (since 2012.ev at least) I've been using a personal wiki called Tiddlywiki for most of my projects which requires just a fairly modern web browser (if you're using Internet Explorer you need to be running IE 10 or later) and some room on your desktop for another file.

Quite a few years (and a couple of re-orgs) ago on the Zero State mailing list we were kicking around the idea of building an unhosted social network to keep in touch, which is to say, a socnet that was implemented only as a single file, with all of the JavaScript and CSS embedded at the end.  Some of the ideas included using a distributed hash table so each instance could find the others, as many crazy but feasible ways as possible to bootstrap a new member of the network into the DHT, and using using the browser's built-in local storage database to hold all of the information.  A lot of this stuff already exists, from the local storage functionality (which has been there, albeit silently, in every modern browser for years) to the DHT in JavaScript so I think that a fair amount of it would consist of tinker-toying it together.  However, and I must confess, the front-end stuff is well beyond me.  Not from lack of trying, mind you: The HTML5 and JavaScript classes I've taken over the years were largely toward the goal of making this happen.  However... I suck.  Web apps are not my thing, unfortunately.

Additionally, this was before I'd ever done any serious information architecture and communications stuff, so you will undoubtedly cringe upon reading some of my assumptions and JSON sketches.  Additionally, this was before I discovered PouchDB (which is basically CouchDB in the browser) so a few of my ideas really wouldn't wash today.  So, please consider these notes somewhat naive toward the goal of building the application.  Please don't facepalm too hard, you'll give yourself a concussion.  Maybe somebody will find them useful in their own work.

To keep the complexity of parts of my exocortex down I've opted to not separate everything into larger chunks using popular technologies these days, such as Linux containers (though I did Dockerize the XMPP bridge as an experiment) because there are already quite a few moving parts, and increasing complexity does not make for a more secure or stable system.  However, this brings up a valid and important question, which is "How do you restart everything if you have to reboot a server for some reason?"

A valid question indeed.  Servers need to be rebooted periodically to apply patches, upgrade kernels, and generally to blow the cruft out of the memory field.  Traditionally, there are all sorts of hoops and gymnastics one can go through with traditional initscripts but for home-grown and third party stuff it's difficult to run things from initscripts in such a way that they don't have elevated privileges for security reasons.  The hands-on way of doing it is to run a GNU Screen session when you log in and start everything up (or reconnect to one if it's already running).  This process, also, can be automated to run when a system reboots.  Here's how:

A month or two back (tired of me saying this over and over?) I had opportunity to attend the Aprilween edition of Turbo Drive at the DNA Lounge and dance the night away in costume to fine music and so much artificial fog that the Sisters of Mercy would have to admit their envy.

Well, I was sort of in costume.  I wasn't sure if I was going to be able to make it at the last minute, so I didn't actually put together a costume.  Danny Delorean, however did an awesome Driver cosplay from Drive that night, down to the varsity jacket.

Okay, enough of me going on about a night over a month ago.  Here are the pictures, few though they be.

Back in March of 2017 (I know, I'm still cleaning out my picture collection) I attended yet another Turbo Drive at the DNA Lounge to see yet another synthwave concert, that time Pixel Memory and Protector-101.  When I wasn't dancing I was snapping pictures of the performers as they blew our minds and melted n>0 faces in the crowd.

Aw, hell, I don't have anything witty to say right now.  Here are the pictures.

While digging around in my picture archive on Windbringer, I found a handful of photographs from the White House hackathon held by the Internet Archive back in January of this year.  I ran a couple of searches and didn't find anything I wrote about what I worked on there (weird...) so here are the pictures I took when I wasn't coding.

Some time ago, I found myself using a Kryoflux interface and a couple of old floppy drives that had been kicking around in my workshop for a while to rip disk images of a colleague's floppy disk collection.  It took me a day or two of screwing around to figure out how to use the Kryoflux's software to make it do what I wanted.  Of course, I took notes along the way so that I would have something to refer back to later.  Recently, I decided that it would probably be helpful to people if I put those notes online for everyone to use.  So, here they are.

A persistent risk of websites is the possibility of somebody finding a vulnerability in the CMS and backdooring the code so that commands and code can be executed remotely.  At the very least it means that somebody can poke around in the directory structure of the site without being noticed.  At worst it would seem that the sky's the limit.  In the past, I've seen cocktails of browser exploits injected remotely into the site's theme that try to pop everybody who visits the site, but that is by no means the nastiest thing that somebody could do.  This begs the question, how would you detect such a thing happening to your site?

I'll leave the question of logfile monitoring aside, because that is a hosting situation-dependent thing and everybody has their own opinions.  What I wanted to discuss was the possibility of monitoring the state of every file of a website to detect unauthorized tampering.  There are solutions out there, to be sure - the venerable Tripwire, the open source AIDE, and auditd (which I do not recommend - you'd need to write your own analysis software for its logs to determine what files, if any, have been edited.  Plus it'll kill a busy server faster than drilling holes in a car battery.)  If you're in a shared hosting situation like I am, your options are pretty limited because you're not going to have the access necessary to install new packages, and you might not be able to compile and install anything to your home directory.  However, you can still put something together that isn't perfect but is fairly functional and will get the job done, within certain limits.  Here's how I did it:

Most file monitoring systems store cryptographic hashes of the files they're set to watch over.  Periodically, the files in question are re-hashed and the outputs are compared.  If the resulting hashes of one or more files are different from the ones in the database, the files have changed somehow and should be manually inspected.  The process that runs the comparisons is scheduled to run automatically, while generation of the initial database is normally a manual process.  What I did was use command line utilities to walk through every file of my website, generate a SHA-1 hash (I know, SHA-1 is considered harmful these days; my threat model does not include someone spending large amounts of computing time to construct a boobytrapped index.php file with the same SHA-1 hash as the existing one; in addition, I want to be a good customer and not crush the server my site is hosted on several times a day when the checks run), and store the hashes in a file in my home directory.

I haven't spent much time with forge and Nicole since their wedding many, many years ago.  Forge was in mine back in '08, but weddings being what they are, I wasn't able to really hang out.  I think they lived in the Bay Area for a while, but now they're living in Maryland under what seems like less-than-optimal conditions..

Nicole recently announced that she's been suffering from polycistic kidney disease for much of her life; it is a disease in which cysts grow inside the kidney in the place of normal nephritic tissue.  If the cysts become too large or too numerous, the kidneys fail to function the way they're supposed to which causes a whole family of other health problems due to one's blood being filtered insufficiently.  If you have any doubts that this can be somewhat problematic, you might want to check out some medical photographs of the condition.  Unfortunately, while the condition can be treated to mitigate symptoms it cannot be cured entirely.  Nicole has lost 90% of her kidney function and she's going to need to go on dialysis within six months.

If you have it laying around, can you please spare a couple of dollars to help an old friend?  Also, if you can spread word of their Gofundme campaign around your respective social networks, can you please do so?  If you would like to sign up to see if you could be a possible kidney donor, please go here and fill out the forms: https://maryland.donorscreen.org/register/donate-kidney

Thank you.

UPDATE - 20170512 - More SQL surgery.

So, as you've no doubt noticed I've been running the Bolt CMS to power my website for a while now.  I've also mentioned once or twice that I've found it to be something of a finicky beast and doing anything major to it can be something of an adventure.  I tried to upgrade my site last week (tonight, by the datestamp on this post) and had to restore from backup yet again because something went sideways.  That something was the upgrade process going wrong and throwing an exception because of something in the cache directory, where Bolt temporarily stores HTML files rendered from templates used to make pages that your web browser displays.

As it turned out, the upgrade process was choking on the old cache directories created and used by v2.x of the Bolt CMS.  Here is the upgrade process that I used:

• BACK UP YOUR SITE.
• Log into your web hosting provider's server via SSH.
• Download the latest version of the flat file structure build of Bolt.
• If you didn't back up your website, BACK UP YOUR WEB SITE.
• cd ~/my.website.here
• If you didn't back up your website and things go pear-shaped, it's your fault.  Don't say I didn't warn you.
• Uncompress the new version of Bolt you just downloaded: tar xvfz ~/bolt-latest-flat-structure.tar.gz --strip-components=1
• Try running the upgrade: php app/nut setup:sync
• If it throws an exception on you, erase the entire on-disk cache.  Don't worry, it'll be rebuilt as people visit your site: rm -rf app/cache/*
• Try running the upgrade again: php app/nut setup:sync
• It should complete successfully.  If it doesn't you may need to do the following two things before re-running the upgrade command again:
  • mkdir -p app/cache/production/data/
  • chmod -R 0775 app/cache/
  • If you still have problems, jump into the Bolt CMS Slack chat and politely ask good questions: https://boltcms.slack.com/
• If the command finishes normally, try opening the frontpage of your website.  It should be up and running.
• If you can see the frontpage of your website, try logging in.  You should be able to.
• Try making a test post with a new entry.  Be sure to test saving the post partway through.  You do save your work every few minutes, don't you?
• Success.

Special thanks to Bob and thisiseduardo in the Bolt CMS Slack chat for their assistance and hand-holding while I stumbled around trying to make this hapen.