Constructing Exocortices with Huginn and Halo

Presenter Notes

Disclaimer

I speak only for myself, not my employers

  • Past
  • Present
  • Future

Presenter Notes

Overview

  • What's an exocortex?
  • Possible components
  • What can an exocortex do?
  • Example: Personal web archive
  • How can you build one?
  • Software
  • Interacting with Huginn
  • Simple agent network design
  • Complex agent network design
  • Events
  • Why did I build an exocortex?
  • What do I do with my exocortex?
  • Demo time!
  • What do I get out of it?
  • What can you get out of it?

Presenter Notes

What's an exocortex?

  • From Greek prefix exo-: Outside or external to.
  • A system first postulated by JCR Licklider in March, 1960 in the paper Man-Computer Symbiosis.
  • A hypothetical (or nascent) additional cortex of the human brain situated outside of the organism, in addition to the various lobes of the brain (frontal, parietal, occipital, temporal, et al).
  • External information processing systems which provide additional cognitive capacity or abilities to a human mind, not yet directly connected to the human brain (but we're getting there: See TDCS, TMS, BrainGate...)
  • Benjamin Houston: "...an organ that resides outside of the brain that aids in high level thinking."
  • Ideally, a symbiotic relationship between the user and the information processing infrastructure where each complements and compensates for the shortcomings in the other.

Ultimately, a hardware/software construct which works in concert with the brain to extend its capabilities.

Presenter Notes

Possible components

  • Networked hand-held devices
    • Smart watches and glasses
  • Smartphones
    • Cellular data service, Wifi, and, Bluetooth
  • Flash media
    • External non-volatile storage
  • Information processing and storage capabilities
    • Notebook, netbook and laptop computers
    • Virtual machines and bare metal
  • Security infrastructure
    • Multifactor authentication, firewalls, ID/PS, enforced crypto, realtime response, you know the drill
  • Externally addressable mass storage
    • Blogs, link collections, wikis, personal archives, file dumps, home directories...
    • In short, your stuff.
  • And, of course, software. Lots of software.

Presenter Notes

What can an exocortex do?

Whatever you design it to do.

  • Monitor data feeds in real time (and respond to them)
  • Manage some aspects of your finances
  • Personal search engine
  • Manage and archive data
  • Monitor your infrastructure
  • Find things that are useful to you

Presenter Notes

Example: Personal web archive

  • UI frontend: Instant messenger client running on smartphone and desktop.
  • UI backend: Bot connected to the same IM server that stores requests in a message queue.
  • Software agent: Bot that polls its message queue looking for URLs, downloads the page, renders it into text, and uploads it into the archive application.
  • Archive: Application that stores text efficiently and presents it in a form that is easy to read later.
    • Editing pages
    • Renaming pages
    • Deleting pages
  • Search engine: Search the archive for desired content later.
  • Eventing system: Alerts the user when archival is complete or a problem arises.

Presenter Notes

How can you build one?

  • Figure out what you want to accomplish.
  • Stand up a server or two.
    • Alternatively, set up an account with a service like Heroku.
  • Install software.
  • Build some agent networks.
  • Install some bots if you want.
  • Set up your user interface.
  • Enjoy not having to hit refresh all the time.

Presenter Notes

Software

  • Huginn - https://github.com/cantino/huginn
    • Written by Andrew Cantino.
    • Framework for building networks of autonomous agents that carry out tasks.
    • Written using Rails.
    • Event-based architecture.
    • Dozens of classes of agents that are optimized for carrying out specific tasks.
  • Halo - https://github.com/virtadpt/exocortex-halo
    • Collection of bots that carry out more complex tasks that Huginn's event based architecture isn't well suited for.
    • Long-running tasks aren't a good fit.
      • Speech synthesis
      • Voice Over IP
      • Integration with other external applications
        • Etherpad-Lite
        • GPS receivers and mapping
        • Interaction with search engines

Presenter Notes

Ways to interact with Huginn

Input

  • Webhooks (WebhookAgent)
  • MQTT (MQTTAgent)
  • Other websites and APIs (WebsiteAgent) (agents for specific services)
  • Files located elsewhere (ReadFileAgent)
  • Local shell commands

Output

  • MQTT (MQTTAgent)
  • Other websites and APIs (PostAgent)
  • Implementing its own RSS feeds and REST APIs (DataOutputAgent)

Presenter Notes

Simple agent network design

 1 RSSAgent:
 2 
 3     {
 4       "expected_update_period_in_days": "5",
 5       "clean": "false",
 6       "url": "https://www.2600.com/rss.xml"
 7     }
 8 
 9 E-mailAgent:
10 
11     {
12       "subject": "{{ title }}",
13       "headline": "Published on {{ date_published }}",
14       "body": "{{ description }}{% line_break %}{% line_break %}
15         {{ content }}",
16       "expected_receive_period_in_days": "365"
17     }

...and the article is e-mailed automatically.

Presenter Notes

More complex agent network design

  • RSS Agent: Pull the RSS Feed for archive.org's DNA Lounge collection
  • Trigger Agents looking for specific keywords in the {{ title }} field
  • RSS Agent: Pull the RSS Feed for archive.org's Hacker Con collection
  • RSS Agent: Pull the RSS Feed for archive.org's Infocom Cabinet collection
  • RSS Agent: Pull the RSS Feed for archive.org's National Security Archive
  • Trigger Agents looking for specific keywords in the {{ title }} field
    • COINTELPRO
    • Hack
    • Phreak
  • Deduplication Agent: Deduplicate events sent from the above agents based on the outbound message
    • Could just as easily be the RSS event's {{ id }} field
  • E-Mail Agent: E-mail the default user when something new hits the Archive

Presenter Notes

Events

(Edited for size)

 1 {
 2   "date_published": "2016-04-10 20:50:00 +0000",
 3   "last_updated": "Sun, 10 Apr 2016 20:50:00 +0000",
 4   "url": "http://www.voanews.com/content/fierce-campaigning-ahead-...",
 5   "description": "The next U.S. presidential primary contest is...",
 6   "content": "The next U.S. presidential primary contest is more...",
 7   "title": "Fierce Campaigning Ahead of New York Presidential Primary",
 8   "authors": [
 9     "webdesk@voanews.com (Michael Bowman)"
10   ],
11   "categories": [
12     "USA",
13     "2016 USA Votes"
14   ]
15 }

Presenter Notes

Why did I build an exocortex?

It didn't start out as a unified project or even a personal tool.

It was my "I want to learn about $foo" project.

It was also a manifestation of "Hey, I'm not on dialup anymore!" syndrome.

This meant that I had constant access to all of the information sources available at the time... and never enough time to keep up with all of it, let alone figure out what was useful and what wasn't.

So I started writing bots that would poll those information sources for updates, figure out what was in them, sift out the useful stuff and send me digests of what they found.

(Remember: RSS wasn't invented until 1999!)

Years went by and maintaining my code wasn't actually possible anymore.

I started writing a framework to re-implement them with when one of my co-workers sent me a link to Huginn, which already did way more than my existing software.

I ported my existing bots to Huginn in an afternoon.

Later I started writing my own bots because I started running into Huginn's functional limits.

Presenter Notes

What do I do with my exocortex?

  • Antigone: Submits URLs to search engines, including my own.
  • Jackhammer: Personal web archive.
  • Argus and Montauk: Personal search engines.
  • Butterfly in China: Daily weather and air quality index reports.
  • Cherrybomb: Monitors police, fire, and EMS radio dispatch networks for statistically unusual spikes in listenership.
  • Edison: Follows and analyzes feeds of several dozen news agencies, the WHO, defense-related news magazines, and online archives.
  • Firefly: Monitors (crypto-)currency markets, accounts, and addresses for statistically unusual activity.
  • Glitch: Monitors manufacturer websites, Linux distributions, software homepages, mailing list archives and RSS feeds for vulnerabilities and patches.
  • Ironmonger: Follows stock prices of defense contractors, petroleum markets, investment funds and multinational corporations as geopolitical indicators.
  • Pathfinder: Connected to the public APIs of public transit services, calculates best routes, alerts on delays and police activities.
  • Switchboard: Secretarial duties, such as monitoring e-mail, watching for new files, autoposting and syndicating blog posts, informing people of new developments, reminding me of calendar events and tasks, watching subscription account credits.

Presenter Notes

Demo time!

  • Butterfly In China
  • Cherrybomb
  • Edison
  • Ironmonger

(Please, bandwidth gods, be with me!)

Presenter Notes

What do I get out of it?

I spend less time at work reading security briefs, websites, and Twitter feeds to keep on top of new vulnerabilities, attacks, and data breaches. I get more work done by applying that information.

When doing research I farm the tasks of executing searches against multiple search engines out to bots so I can spend more time digesting the information and writing. My personal search engines let me search my archives and notes more efficiently.

I find out about new and interesting stuff without having to spend hours every day browsing dozens of sites. I get e-mail digests with links that I scan whenver I have a free moment. I can also filter out stuff that's not interesting.

I have a framework for experimenting with new stuff in a practical way, like machine learning and neural networks.

I get to spend more time

  • with my family
  • hacking
  • enjoying myself

Presenter Notes

What can you get out of it?

  • News and social network monitoring, whatever your interests and use cases happen to be.
    • Data breaches
    • The next mass death or terrorist strike
    • Leaked 0-days
  • Connecting devices and software so they can interact with each other
    • Speech recognition (Mycroft)
    • Media playback (Mopidy, Kodi)
    • RF hardware and software (RTL-based SDRs and GQRX)
  • Connecting RADIUS to the Metasploit RPC API
    • New system joins your network
    • Gets scanned
    • Gets the kitchen sink thrown at it
    • Vulnerability report is sent to Operations
  • Interacting with (crypto-)currency networks
  • Responding to sudden changes in your online social activity
  • Interacting with third-party APIs and making them do things they're not supposed to

Presenter Notes

Comments or questions?

Presenter Notes

How to contact me.

The Doctor [412/724/301/703/415]

E-mail: drwho at virtadpt dot net

PGP: 0x807B17C1 / 7960 1CDC 85C9 0B63 8D9F DD89 3BD8 FF2B 807B 17C1

Web: https://drwho.virtadpt.net

Keybase: https://keybase.io/drwho

Twitter: @virtadpt

Github: virtadpt

Public profile: about.me

Presenter Notes

Thank you very much for attending! Travel home safely!

Presenter Notes