Building your own Google Alerts with Huginn and Searx.

Sep 30 2017

A Google feature that doesn't ordinarily get a lot of attention is Google Alerts, which is a service that sends you links to things that match certain search terms on a periodic basis.  Some people use it for  vanity searching because they have a personal brand to maintain, some people use it to keep on top of a rare thing they're interested in (anyone remember the show Probe?), some people use it for bargain hunting, some people use it for intel collection... however, this is all predicated on Google finding out what you're interested in, certainly interested enough to have it send you the latest search results on a periodic basis.  Not everybody's okay with that.

A while ago, I built my own version of Google Alerts using a couple of tools already integrated into my exocortex which I use to periodically run searches, gather information, and compile reports to read when I have a spare moment.  The advantage to this is that the only entities that know about what I'm interested in are other parts of me, and it's as flexible as I care to make it.  The disadvantage is that I have some infrastructure to maintain, but as I'll get to in a bit there are ways to mitigate the amount of effort required.  Here's how I did it...

What the loss of the Internet Privacy Bill means to you and I.

Mar 30 2017

It's probably popped up on your television screen that the Senate and then the House of Representatives voted earlier this week, 215 to 205, to repeal an Internet privacy bill passed last year.  In case you're curious, here's a full list of every Senator and Representative that voted to repeal the bill and how much they received specifically from the telecom lobby right before voting. (local mirror)  By the way, if you would like to contact those Senators (local mirror) or Representatives (local mirror) here's how you can do so... When the bill hits Trump's desk it's a foregone conclusion that he's going to sign it.  Some of the talking heads are expressing concern about this, while others are cheering that the removal of this regulation is an all-around win for the market, blah blah blah... but what does this actually mean for you?

First of all, if you're reading this, welcome to the Internet.  You're soaking in it.

Second of all, please read this blog post (local mirror) by the EFF.  Just a few years ago, a couple of very large ISPs (that you're probably a customer of) got caught doing things like monitoring your web searches and hijacking them with different results they were paid to insert and analyzing your net.traffic to figure out what advertisements to inject in realtime.  The bill that just got repealed put a stop to all of that.

I've spoken to a couple of people who expressed disbelief that such a thing was possible.  In point of fact, intercepting and meddling with communications traffic goes back a very long way.  In 1994 a bill called the Communications Assistance for Law Enforcement Act (CALEA) was passed and codified as 47 USC 1001-1010.  In a nutshell, what this law means is that manufacturers of just about every kind of network-side communications device, from the telephony switches that route your phone calls to the carrier class routers that make up the network core have surveillance capability built in.  In theory, only law enforcement agents with warrants are supposed to be able to use them.  In practice, they're used all the time by employees of the companies that own that equipment to silently troubleshoot problems before they get too out of hand, and yes, they get abused all the time for petty shit.  As you may have guessed already, the moment that CALEA-compliant equipment was deployed back in the day hackers immediately figured out how to use them more effectively than even the telecom companies and silently eavesdropping on people using that functionality was a common "This is how 1337 I am" stunt.  So, please keep in mind that this "monitor all the customers" infrastructure is going to be badly abused and constitutes one hell of a security risk.

CALEA is regularly updated as communications technology evolves, and now encompasses things like the backbone of the Net, Voice-over-IP telephony, cellular telephony and companies whose business it is happens to be running wireless hotspots.  As it so happens, much of this functionality is perfect for monitoring customers' traffic, analyzing it, and packaging it for sale as large bundles of anonymized information or as discrete dossiers, ala Cambridge Analytica.  Let me paint you a picture, based in part of how things worked before that bill was passed originally...