Installing Searx by hand.

In monitoring the Searx Github repository because I'm a pretty heavy user of this software, I've noticed a common trend. Folks seem to have a hard time getting the automatic installation script to work right. I realize that it would probably make sense to figure out what's going on in there and file a pull request, but given how work's been riding me like a wet pony lately I can't reliably budget time to debug the script under a couple of different distros of Linux and figure out what's wrong. That means that I can't actually help any of the …

Read more...

Distributing Huginn workers across servers.

For quite a few years I've written about strange and sundry things you can do with Huginn, but not a lot about what to do when you run into systemic limitations. The nice thing about Huginn is that you can spin up as many workers (subprocesses that execute agents from the database) as you want, subject to the limitations of what you happen to be running it on. The downside, however, is that it's easy to accidentally upgrade your VPS to the point where it's just really expensive. I just ran into this purely by accident and spent a day …

Read more...

Cross-compiling go-sendxmpp.

I used to joke that the day setting up a cross-compilation environment was easy we'd be one short step away from having true artificial general intelligence. For the most part neither has happened yet. However, I must admit that Go has come pretty close to making it easy, but it's also kind of opaque unless you go all-in on Go to the exclusion of all other languages. It's not really a language that you can just toy around with, kind of like FORTH.

Long-time readers know that I'm all about XMPP as a command and control channel for my exocortex …

Read more...

Optimizing Searx with UWSGI.

Long time readers have probably read about some of the stuff I do with Searx and I hope that some of you have given some of them a try on your own. If you have you're probably wondering how I get the performance I do because there are some limitations of Searx that have to be worked around. Most of those limitations have to do with the global interpreter lock that is part of the Python programming language which haven't been completely solved yet. What this basically adds up to is that multithreading in Python doesn't actually make great use …

Read more...

Fixing YaCy?

A couple of weeks back I decided to upgrade the YaCy installs running on Leandra to the latest supported versions, because they'd been lagging behind for a while. Due to the fact that they're enterprisey Java web applications and I can't readily get hold of any live chickens to sacrifice, I'd been putting it off as much as possible.

As it turned out, the lack of sacrificial barnyard fowl wound up being a crucial factor in how things transpired.

The first install that I upgraded was an install from source code and was indexing my personal library. It got re-indexed …

Read more...

Interfacing Fess with Searx.

I promise I'll explain what Fess is in a later post. I want to get this information out there in preparation.

If you haven't used Searx before, it's a self-hosted meta-search engine which queries a wide array of search engines (some of which are also self-hosted), collates the search results, and returns them as a regular search result page, an RSS feed, or a JSON API.

One of the lesser known features is that you can add your own search engines. You can either write your own (using an existing one as a template) or you can leverage one of …

Read more...

Embedded environment monitoring.

Disclaimer: This post has lots of links to the Adafruit website.  There are no referral links, I received no consideration, I just buy parts from there and do cool things with them.

A couple of weeks months ago I did a writeup of a prototype environment monitoring device for my office built out of a Raspberry Pi Zero W and some off the shelf components.  In the time since I've found time here and there to work on the embedded version, which doesn't use a full computer system but a microcontroller with just enough functionality to drive a couple of …

Read more...

Simple environment monitoring with spare parts.

It's going on summer in the Bay Area, which means that it's warming up a bit both outside and inside (because air conditioning is Not A Thing out here).  That, coupled with the not inconsiderable research infrastructure I have at home has left me wondering and worrying about just how hot my office gets during the day while I'm working.  Now, I could just put a simple little thermometer on my shelf (and I did) but my concerns are a bit bigger than that.  What happens if my office temperature reaches a critical point and servers start melting down on …

Read more...

Tunneling across networks with Nebula.

Longtime readers have no doubt observed that I plug a lot weird shit into my exocortex - from bookmark managers to card catalogues to just about anything that has an API.  Sometimes this is fairly straightforward; if it's on the public Net I can get to it (processing that data is a separate issue, of course).  But what about the stuff I have around the lab?  I'm always messing with new toys that are network connected and occasionally useful.  The question is, how do I get it out of the lab and out to my exocortex?  Sometimes I write bots to …

Read more...

Nifty things to do with Searx.

Not too long ago I was noodling over a problem: I wanted to break up the scheduling queues in Huginn to make my fleets of agents a little more efficient when the execute.  The best way I could think of was to make some of the schedules stochastic - periodically have an agent roll some dice and depending on what comes up decide whether or not to trigger the agents downstream.  So, of course I started looking for a random number generator that would basically roll 1d10.  However, the Liquid templating language that Huginn uses internally doesn't have any function to …

Read more...