Parsing simple commands in Python.

Feb 04, 2017

A couple of weeks ago I ran into some of the functional limits of my web search bot, a bot that I wrote for my exocortex which accepts English-like commands ("Send me top 15 hits for HAL 9000 quotes.") and runs web searches in response using the Searx meta-search engine on the back end.  This is to say that I gave my bot a broken command ("Send hits for HAL 9000 quotes.") and the parser got into a state where it couldn't cope, threw an exception, and crashed.  To be fair, my command parser was very brittle and it was only a matter of time before I did something dumb and wrecked it.  At the time I patched it with a bunch of if..then checks for truncated and incorrect commands, but if you look at all of the conditionals and ad-hoc error handling I probably made the situation worse, as well as much more difficult to maintain in the long run.  Time for a rewrite.

Back to my long-term memory field.  What to do?

I knew from comp.sci classes long ago that compilers use things called parsers and grammars to interpret code so that it can be converted into an executable.  I also knew that the parser Infocom used in its interactive fiction was widely considered to be the best anyone had come up with in a long time, and it was efficient enough to run on humble microcomputers like the C-64 and the Apple II.  For quite a few years I also ran and hacked on a MOO, which for the purposes of this post you can think of as a massive interactive fiction environment that the players can modify as well as play in; a MOO's command parser does pretty much the same thing as Infocom's IF parser but is responsive to the changes the user's make to their environments.  I also recalled something called a parse tree, which I sort-of-kind-of remembered from comp.sci but because I'd never actually done anything with them, I only recalled a low-res sketch.  At least I had someplace to start from so I turned my rebooted web search bot loose with a couple of search terms and went through the results after work.  I also spent some time chatting with a colleague whose knowledge of the linguistics of programming languages is significantly greater than mine and bouncing some ideas off of him (thanks, TQ!)

But how do I do something with all this random stuff?

Click for the rest of the article...

Neologism: The Magick Poke

Feb 01, 2017

The Magick Poke - noun - When you touch a failing appliance, light bulb, or other gizmo in the just the right way as you're replacing it, and it spontaneously starts working again.  This usually saves it from the trashcan or dumpster.  Comes from the POKE command in Commodore BASIC which could let you do some pretty strange things by putting just the right value into just the right memory location, usually by fat-fingering a value.

Saving stuff before it vanishes down the memory hole.

Jan 26, 2017

UPDATE - 20170302 - Added Firefox plugin for the Internet Archive.

UPDATE - 20170205 - Added Chrome plugin for the Internet Archive.

Note: This article is aimed at people all across the spectrum of levels of experience with computers.  You might see a lot of stuff you already know; then again, you might learn one or two things that hadn't showed up on your radar yet.  Be patient.

In George Orwell's novel 1984, one of his plot points of the story was something called the Memory Hole. They were slots all over the building in which Winston Smith worked, into which documents which the Party considered seditious or merely inconvenient were deposited for incineration.  Anything that the Ministry of Truth decided had to go because it posed a threat to the party line was destroyed.  This meant that if anyone wanted to go back and double check to see what history might have been, the only thing they could get hold of were "officially sanctioned" documents written to reflect the revised Party policy.  Human memory's funny: If you don't have any static representation of something to refer back to periodically, eventually you come to think that whatever people have been telling you is the real deal, regardless of what you just lived through.  No mind tricks are necessary, just repetition.

The Net's a lot like that.  There are literally piles and piles of information everywhere you look, but most of it resides on systems that aren't yours.  This blog is running on somebody else's server, and it wouldn't take much to wipe it off the face of the Net.  All it would take is a DMCA takedown notice with no evidence (historically speaking, this is usually the case).  This has happened in the past a number of times, including to an archive maintained by Project Gutenberg and documents explicitly placed into the public domain so somebody could try to make a buck off of them.  This is a common enough thing that the IETF has made a standard HTTP error code to reflect it, Error 451 - Unavailable for legal reasons.

So, how would you make local copies of information that you think might be pulled down because somebody thought it was inconvenient?  For example, climatological data archives?

Click for the rest of the article...

Prediction: The United States of America will be at war again by 26 July 2017.

Jan 28, 2017

Hash: SHA512

I very much want to be wrong.

Within 180 days of 0000 hours UTC, Friday, 27 January 2017, the United States of
America will declare war once again.  That puts it at Wednesday, 26 July 2017
at 0000 hours UTC.  I do not know for sure, but countries in the Middle East
seem the most likely targets.

This seems due, in part, that the USA seems to be trying to start the Crusades
again (George W. Bush tried once).  The Trump administrations' public and
flagrant distrust, disapproval, and seeming pants-shitting-fear of Muslims
around the world would seem to point to this.

This seems due, in part, to the low approval ratings of President Donald J.
Trump.  As of 28 January 2017 @ 1454 hours PST8PDT, they are at 36% approval,
44% disapproval on  I've made a PDF
copy of this file here:

SHA-512 hash of trump_approval_ratings-20170128.pdf:

To jack up his approval ratings (and improve his chances of re-election, if
nothing else) this is a logical national action to take.  It certainly worked
for George W. Bush after 9/11 but wasn't enough to sustain his approval ratings
in the long run.

Evidence of the above claim:

Local copy of the above claim:

SHA-512 hash of george_w_bush-overall_approval_ratings.pdf:

I have received no orders, demands, or suggestions to write or post this.  I
have received no pay or other compensation to write or post this.

I have no insider knowledge.

I have only the patterns of history to suggest this, and I'm seeing everything
happen all over again.  And again.

I don't want to be right again.

- --The Doctor [412/724/301/703/415]



#datarefuge in the Bay Area - 11 February 2017.

Jan 28, 2017

UPDATE: 20170131 - The Eventbrite page for this event has gone live!  Sign up!

I haven't had time to write about #datarefuge yet, in part because people a lot closer to the matter have been doing so, and much better than I could at the moment.  An entire movement has arisen around scientific data being 451'd because it's politically inconvenient, and not many of us know if it's being erased or just shut down.  We also don't know for certain if it's being copied elsewhere for safekeeping so we're doing it ourselves.  To do my part, I've been communicating with some of the organizers and having Leandra suck down data as fast as my home link will permit to store it on her RAID array.  But, the important thing:

On 11 February 2017, the Datarescue SF Bay event will held at the Berkeley Institute for Data Science from 0900 PST until 1500 PST.  That day, everybody at the event will identify data sets at risk of vanishing, work out how to best mirror them, and download them as fast as possible so they can be archived elsewhere.  Bring your drives, bring your boxen, and get ready to burn up bandwidth.

Autostarting Kodi on an Arch Linux media box.

Jan 20, 2017

Not too long ago, when the USB key I'd built a set-top media machine died from overuse I decided to rebuild it using Arch Linux with Kodi as the media player.  The trick, I keep finding every time, lies in getting Kodi to start up whenever the machine starts up.  I think I've re-figured that out six or seven times by now, and each time after it works I forget all about it.  So, I guess I'd better write it down for once so that I've got a snapshot of what I did in case I need to do it again later.

The instructions in the Arch Linux wiki work, but you need to pick the right ones to follow.  The short-and-sweet ones with the automagickal AUR package don't work.  Forget it.

Install LightDM from the Arch package repository (sudo pacman -S lightdm).  Then install the instructions I linked to above to the letter.  That means carrying out the following tasks:

Create the file /etc/X11/Xwrapper.config.  The file should contain only the following text in bold (no double quotes): "needs_root_rights = yes"

Follow the LightDM "Enabling autologin" and "Enabling interactive passwordless login" instructions.  Create a user named "kodiuser" (you don't need to set a password" and give it access to system groups necessary to access resouces in the system.  I used the following command to do this: sudo useradd -c "Kodi Service Account" -G dbus,network,video,audio,optical,storage,users -m kodiuser

Create two additional groups which LightDM needs to enable autologin:

  • sudo groupadd -r autologin
  • sudo groupadd -r nopasswdlogin

Add kodiuser to those groups:

  • sudo gpasswd -a kodiuser autologin
  • sudo gpasswd -a kodiuser nopasswdlogin

Click for the rest of the article...