Optimizing Searx with UWSGI.

Long time readers have probably read about some of the stuff I do with Searx and I hope that some of you have given some of them a try on your own. If you have you're probably wondering how I get the performance I do because there are some limitations of Searx that have to be worked around. Most of those limitations have to do with the global interpreter lock that is part of the Python programming language which haven't been completely solved yet. What this basically adds up to is that multithreading in Python doesn't actually make great use …

Read more...

Fixing YaCy?

A couple of weeks back I decided to upgrade the YaCy installs running on Leandra to the latest supported versions, because they'd been lagging behind for a while. Due to the fact that they're enterprisey Java web applications and I can't readily get hold of any live chickens to sacrifice, I'd been putting it off as much as possible.

As it turned out, the lack of sacrificial barnyard fowl wound up being a crucial factor in how things transpired.

The first install that I upgraded was an install from source code and was indexing my personal library. It got re-indexed …

Read more...

Interfacing Fess with Searx.

I promise I'll explain what Fess is in a later post. I want to get this information out there in preparation.

If you haven't used Searx before, it's a self-hosted meta-search engine which queries a wide array of search engines (some of which are also self-hosted), collates the search results, and returns them as a regular search result page, an RSS feed, or a JSON API.

One of the lesser known features is that you can add your own search engines. You can either write your own (using an existing one as a template) or you can leverage one of …

Read more...

Embedded environment monitoring.

Disclaimer: This post has lots of links to the Adafruit website.  There are no referral links, I received no consideration, I just buy parts from there and do cool things with them.

A couple of weeks months ago I did a writeup of a prototype environment monitoring device for my office built out of a Raspberry Pi Zero W and some off the shelf components.  In the time since I've found time here and there to work on the embedded version, which doesn't use a full computer system but a microcontroller with just enough functionality to drive a couple of …

Read more...

Simple environment monitoring with spare parts.

It's going on summer in the Bay Area, which means that it's warming up a bit both outside and inside (because air conditioning is Not A Thing out here).  That, coupled with the not inconsiderable research infrastructure I have at home has left me wondering and worrying about just how hot my office gets during the day while I'm working.  Now, I could just put a simple little thermometer on my shelf (and I did) but my concerns are a bit bigger than that.  What happens if my office temperature reaches a critical point and servers start melting down on …

Read more...

Tunneling across networks with Nebula.

Longtime readers have no doubt observed that I plug a lot weird shit into my exocortex - from bookmark managers to card catalogues to just about anything that has an API.  Sometimes this is fairly straightforward; if it's on the public Net I can get to it (processing that data is a separate issue, of course).  But what about the stuff I have around the lab?  I'm always messing with new toys that are network connected and occasionally useful.  The question is, how do I get it out of the lab and out to my exocortex?  Sometimes I write bots to …

Read more...

Nifty things to do with Searx.

Not too long ago I was noodling over a problem: I wanted to break up the scheduling queues in Huginn to make my fleets of agents a little more efficient when the execute.  The best way I could think of was to make some of the schedules stochastic - periodically have an agent roll some dice and depending on what comes up decide whether or not to trigger the agents downstream.  So, of course I started looking for a random number generator that would basically roll 1d10.  However, the Liquid templating language that Huginn uses internally doesn't have any function to …

Read more...

Integrating Huginn with a Matrix server.

Throughout this series I've shown you how to set up a Matrix server and client using Synapse and Riot, and make it much more robust as a service by integrating a database server and a mechanism for making VoIP more reliable.  Now we'll wrap it up by doing something neat, building a simple agent network in Huginn to post what I'm listening to into a Matrix Room.  I have an account on libre.fm that my media players log to which we'll be using as our data source.  Of course, this is only a demonstration of the basic technique, you …

Read more...

Using Ansible to restart a bunch of services running under systemd in --user mode.

Let's say that you have a bunch of servers that you admin en masse using Ansible.  You have all of them listed and organized in your /etc/ansible/hosts file.  Let's say that each server is running a system service (like my Systembot) running under systemd in --user mode.  (Yes, I'm going to use my exocortex-halo/ repository for this, because I just worked out a good way to keep everything up to date and want to share the technique for everyone new to Ansible.  Pay it forward, you know?)  You want to use Ansible to update your copy of Systembot …

Read more...