It should come as little surprise to anyone out there that I have a bit of a problem with hoarding data. Books, music, and of course files of all kinds that I download and read or use in a project for something. Legal briefs, research papers (arXiv is the bane of my existence), stuff people ask me to review, the odd Humble Bundle... So much so that a scant few years ago I rebuilt Leandra to better handle the volume of data in my library. However, it's taken me this long to both figure out and get around to making it easier to find anything in all that mess. If I can't find it, I can't do anything with it, or even figure out what I do or don't have. I also don't often have console access so it's not as if I can SSH in and grep for what I need. I use Nginx as a web server on Leandra so actually getting access to files when I need them is trivial.
It's been an interesting couple of weeks, to be sure. While lots of different things have been going on lately, none of them are related in any particularly clear or straightforward fashion, so fitting all of this stuff together is going to be a bit of a struggle. You may as well kick back with the beverage of your choice in a responsible fashion while I spin this yarn.
I suppose it all started with wardriving in northern Virginia many years ago. In a nutshell, I had loaded Windbringer up with a rather small for the time USB GPS unit, installed Kismet, put the wifi NIC into monitor mode so it would pick up frames from every access point within range, and went driving around for a couple of hours. The idea is that the software records the datestamp and GPS coordinates at which you picked up the strongest signal from a wireless access point. Rinse, repeat for as long as your power cells hold out, or as long as you care to drive, bike, walk, ski, or employ any other means of personal transportation to move around. At the time I was uploading my results to wigle.net to contribute to their crowdsourced global map of wireless coverage. Then I moved, and I seem to have accidentally tripped Wigle's bot detector (probably because I was going out for many hours at a time to cover very large areas). End result, I didn't go wardriving for a very long time.
A couple of months back I decided that I needed to get more exercise than I could get at home (which I'll probably ramble about in a later post) so I joined the local gym. Doing so gave me access to a much more broad selection of equipment to work with, and a lot more space than my office at home. There isn't much to say on that particular point other than it's been a great investment, and I spent a nontrivial amount of downtime there working out. While I haven't lost weight per se, I do seem to be trading some amount of body fat for muscle mass. I don't know how much adipose tissue I've actually lost but my clothes are getting tight against my body in different ways than before. I guess that's something.
GSCA - acronym, verb - Using grep, sed, cut, and awk on a Linux or UNIX box to chop up, mangle, or otherwise process data on the command line prior to doing anything serious with it. This is not to preclude the use of additional tools (such as sort).
A couple of days ago (a couple of minutes ago, as I happen to write this) I watched a documentary on Youtube about a modern urban legend, the video game called Polybius. I don't want to give away the entire story if you've not heard it before, but a capsule version is that in 1981.ev a strange video game called Polybius was installed in a number of video arcades in the Pacific Northwest. The game supposedly had a strange effect on some of the people playing it, ranging from long periods of hypnosis to night terrors, epileptic convulsions and, it is rumored, a small number of deaths due to sudden heart failure. It's a story circulated for years online in one form or another, and a number of people have built their own versions that fit the details of the story, with varying degrees of fidelity. I'll admit, one of my long-term plans is to build a MAME cabinet at home that looks like one as a conversation piece. It's a modern day tall tale, where chances are you know somebody who knows somebody whose brother dated the sister of a guy who wound up in the hospital in a coma back in 198x because he spent 50 hours entranced playing some weird game in an arcade while on a family trip, and mysteriously the cabinet was gone by the time he was released.
One thing that I don't think I've heard anybody say, though, is that the origins of the story might date back to the late 1990's. I first came across a story about a video game in the early 1980's that had strange effects on its players in the book GURPS Warehouse 23, published by Steve Jackson Games (first printing in 1997, second printing in 1999, available for purchase as a downloadable PDF from the Steve Jackson Online Store because the dead tree edition is out of print). The chapter Conspiracies, Cover-Ups, and Hoaxes of the game supplement opens with a story called The Astro Globs! Cover-Up, which talks about a video game called Astro Globs! (unsurprisingly) developed in 1983 by a computer programmer named Gina Moravec (after Hans Moravec?) which was uncannily adaptive to the person playing it. The video game described by the game book would figure out how the person playing it thought and tailored itself to be increasingly challenging and fascinating without ever getting frustrating, which also made it dangerously hypnotic. The son of the programmer of the game was hospitalized for dehydration after playing it for over 72 hours with neither sleep nor food nor water.
The first printing of Warehouse 23 was in 1997, which implies that the genesis of the Astro Globs! story was some time prior to that. From what little I know of the professional RPG authorship industry, factor in maybe a year's time for proofreading, layout, and the first print run to wind up in the warehouse for distribution (this was in the late 90's, after all - desktop publishing was nowhere near as advanced as it is now, and print-on-demand was certainly not a thing then) and two or three years for development, editing, playtesting, kicking around the group of people working on the text... so I would carefully guess that the idea came about some time in the early 1990's.
The documentary states that the page on coinop.org I linked to above was created on 3 August 1998 at 0000 hours (timezone unknown) (local mirror, 20171120), which puts it about a year after the first edition of Warehouse 23 hit the shelves. The researchers who made the documentary say that they traced the page as far back as 6 February 2000 using the Wayback Machine, which strongly implies that the date in the page footer is incorrect, possibly due to a default value entered in the back-end database during a site migration.
So... perhaps some GURPS conspiracy flavor can be found in the roots of this story? Maybe somebody trying to make their favorite part of the book come to life somehow?
UPDATE: 20170131 - The Eventbrite page for this event has gone live! Sign up!
I haven't had time to write about #datarefuge yet, in part because people a lot closer to the matter have been doing so, and much better than I could at the moment. An entire movement has arisen around scientific data being 451'd because it's politically inconvenient, and not many of us know if it's being erased or just shut down. We also don't know for certain if it's being copied elsewhere for safekeeping so we're doing it ourselves. To do my part, I've been communicating with some of the organizers and having Leandra suck down data as fast as my home link will permit to store it on her RAID array. But, the important thing:
On 11 February 2017, the Datarescue SF Bay event will held at the Berkeley Institute for Data Science from 0900 PST until 1500 PST. That day, everybody at the event will identify data sets at risk of vanishing, work out how to best mirror them, and download them as fast as possible so they can be archived elsewhere. Bring your drives, bring your boxen, and get ready to burn up bandwidth.