Optimizing Searx with UWSGI.

Long time readers have probably read about some of the stuff I do with Searx and I hope that some of you have given some of them a try on your own. If you have you're probably wondering how I get the performance I do because there are some limitations of Searx that have to be worked around. Most of those limitations have to do with the global interpreter lock that is part of the Python programming language which haven't been completely solved yet. What this basically adds up to is that multithreading in Python doesn't actually make great use …

Read more...

Interfacing Fess with Searx.

I promise I'll explain what Fess is in a later post. I want to get this information out there in preparation.

If you haven't used Searx before, it's a self-hosted meta-search engine which queries a wide array of search engines (some of which are also self-hosted), collates the search results, and returns them as a regular search result page, an RSS feed, or a JSON API.

One of the lesser known features is that you can add your own search engines. You can either write your own (using an existing one as a template) or you can leverage one of …

Read more...

Nifty things to do with Searx.

Not too long ago I was noodling over a problem: I wanted to break up the scheduling queues in Huginn to make my fleets of agents a little more efficient when the execute.  The best way I could think of was to make some of the schedules stochastic - periodically have an agent roll some dice and depending on what comes up decide whether or not to trigger the agents downstream.  So, of course I started looking for a random number generator that would basically roll 1d10.  However, the Liquid templating language that Huginn uses internally doesn't have any function to …

Read more...

Building your own Google Alerts with Huginn and Searx.

A Google feature that doesn't ordinarily get a lot of attention is Google Alerts, which is a service that sends you links to things that match certain search terms on a periodic basis.  Some people use it for  vanity searching because they have a personal brand to maintain, some people use it to keep on top of a rare thing they're interested in (anyone remember the show Probe?), some people use it for bargain hunting, some people use it for intel collection... however, this is all predicated on Google finding out what you're interested in, certainly interested enough to have …

Read more...