Interfacing Fess with Searx.

I promise I'll explain what Fess is in a later post. I want to get this information out there in preparation.

If you haven't used Searx before, it's a self-hosted meta-search engine which queries a wide array of search engines (some of which are also self-hosted), collates the search results, and returns them as a regular search result page, an RSS feed, or a JSON API.

One of the lesser known features is that you can add your own search engines. You can either write your own (using an existing one as a template) or you can leverage one of …

Read more...

Organizing a data hoard with YaCy.

 It should come as little surprise to anyone out there that I have a bit of a problem with hoarding data.  Books, music, and of course files of all kinds that I download and read or use in a project for something.  Legal briefs, research papers (arXiv is the bane of my existence), stuff people ask me to review, the odd Humble Bundle... So much so that a scant few years ago I rebuilt Leandra to better handle the volume of data in my library.  However, it's taken me this long to both figure out and get around to making …

Read more...

Sometimes the old ways may be best.

A couple of weeks back, I found myself in a discussion with a couple of friends about searching on the Internet and how easy it is to get caught up in a filter bubble and not realize it.  To put not too fine a point on it, because the big search engines (Google, Bing, and so forth) profile users individually and tailor search results to analyses of their search histories (and other personal data they have access to), it's very easy to forget that there are other things out there that you don't know about for the simple reason that …

Read more...

Simple things can be hard.

As the title of this post implies, I've been working on some stuff lately that's been taking up enough compute cycles that I haven't been around to post much.  Some of this is due to work, because we're getting into the really busy time of year and when I haven't been at work I've been relaxing.  Some of this is due to yet another run of dental work that, while it hasn't really been worth writing about has resulted in my going to bed and sleeping straight through until the next day.  And some of it's due to my hacking …

Read more...

Technomancer Tools: YaCy

UPDATED: Added an Nginx configuration block to proxy YaCy.

If you've been squirreling away information for any length of time, chances are you tried to keep it all organized for a certain period of time and then gave up the effort when the volume reached a certain point.  Everybody has therir limit to how hard they'll struggle to keep things organized, and past that point there are really only two options: Give up, or bring in help.  And by 'help' I mean a search engine of some kind that indexes all of your stuff and makes it searchable so you …

Read more...

Building your own Google Alerts with Huginn and Searx.

A Google feature that doesn't ordinarily get a lot of attention is Google Alerts, which is a service that sends you links to things that match certain search terms on a periodic basis.  Some people use it for  vanity searching because they have a personal brand to maintain, some people use it to keep on top of a rare thing they're interested in (anyone remember the show Probe?), some people use it for bargain hunting, some people use it for intel collection... however, this is all predicated on Google finding out what you're interested in, certainly interested enough to have …

Read more...

US Legal System puts the kibosh on warrantless seizure of laptops at the border.

For a couple of years now the US Department of Homeland Security has reserved the right to confiscate the laptop computers of US citizens for forensic analysis upon re-entry to the country after traveling abroad. It didn't matter if you were on one of their watchlists (and who isn't these days?), it didn't matter if you'd mouthed off to a security guard, it didn't matter whether or not they had probable cause, they could do it and possibly never return it to you depending on when the got around to going through it and how they felt that morning. It's …

Read more...

TSA at PHL raising eyebrows; travelers consider taking Amtrak.

I've only been to Philadelphia a couple of times, all of them by driving to and from there. After reading about some of the stuff going on there not only do I not particularly want to visit that city, but I'm not entirely certain that I really want to fly again.

This particular news story leaped out at me for its sheer WTF factor even though the incident seems to have taken place in March of 2009. Bob Thomas, a 53 year old Camden police officer, his wife Leona, and their four year old son Ryan were flying down to …

Read more...

Legal battles over unwarranted search and seizure at the borders are spinning up.

For a couple of years now the Customs and Border Patrol of the United States has had the legal authority to confiscate the laptops of people entering the country to perform forensic analysis on an indefinite basis. If you don't give them your laptop (or you refuse to give them the passphrases to decrypt your data) they can and will send you back or incarcerate you, even if you're an American citizen. They also have standing orders to seize any and all data storage media you're transporting (including USB keys, cameras, cellular phones, MP3 players, and disks) for duplication and …

Read more...

Portable power for search and seizure.

A major problem faced by data forensics professionals and law enforcement was how to confiscate computer systems without running the risk of damaging or losing access to information. It's all well and good if you seize a machine running full-disk encryption while it's online because, by definition, the disk is being transparently decrypted so that the machine can operate. Once you power it down, however, all bets are off because the machine won't boot back up without someone supplying a passphrase to the disk encryption system, and no one with anything shady in mind is going to give up their …

Read more...