A new way to write web applications.

14 March 2013

It's almost taken for granted these days that your data lives Out There Somewhere on the Internet. If you set up a webmail account at a service like Gmail or Hushmail, your e-mail will ultimately be stored on a bunch of servers racked in a data center someplace you will probably never see. Users of social networks implicitly accept that whatever they post - updates, notes, images, videos, comments, what have you - will probably never touch any piece of hardware they own ever again. Everything stays in someone else's server farm whether or not you want it to, and while there are sometimes options for extracting it rarely has anyone written any software which can actually do anything with it (like re-importing it someplace else) because the formats are never identical. It'd be a lot of work. Additionally, if you lose access to your account somehow - for example, if someone manages to successfully guess your password, social engineer their way in, or force a password reset it's exceedingly difficult to get your access back (Why would someone want to do that? Since when have people ever needed a reason to be malicious?)

But what if you didn't have to trust someone else to hold your data? What if you didn't have to worry about logging into a service somewhere and managing yet another password? Granted, we're not completely there yet, but there are definitely options waiting to be assembled into something more...

In the past couple of years a new paradigm of web application has started to see a fair amount of development. Called unhosted apps, they use the state of the art in HTML, CSS, and JavaScript to build completely self-contained applications that load and run in practically any modern browser (yes, even IE, much to my surprise). The web apps that we usually think of are found on web servers Somewhere Out There. A lot of personal sites and web apps these days are written using PHP, which is executed by a subprocess on the web server, usually accesses a database or two (MySQL remains the de facto standard even though there are better ones out there), and generates the HTML which you see in your web browser. The barrier to entry for PHP apps is so low (upload files, set permissions, create database, profit) that practically every web hosting service worth the name supports it as part of the basic package. More recently frameworks to build web applications which are closer in architecture to desktop apps under the hood have become popular (like Django and Ruby On Rails) and consequently are hot fields of employment. The thing about most of them is that their dependencies and runtime requirements are much greater than those of PHP. Just about every web hosting service out there imposes limits on how long a process owned by a particular username can execute, and most of those take the form of a service which runs in the background continually, with the web server sitting in front as a proxy. PHP apps are executed only when someone browses the page, so the system admin and accounting overhead is far less. Still, these frameworks are incredibly useful and see a lot of use, which is probably one of the reasons that personal VPSes have gotten so cheap.

Unhosted applications are unique in that they're not written in a language like PHP or Ruby, they're implemented using just HTML, CSS, and JavaScript. They might be broken out across a couple of files but there is no reason that all of the code can't be stacked neatly into a single file (for example, an unhosted wiki called Tiddlywiki that I use frequently in software development projects (because I can check it into a version control repository along with the source code); it's an entirely self contained self-modifying HTML file). Unhosted apps can be dropped anywhere you can upload an HTML file, from a web hosting provider like Dreamhost (disclaimer: I've been a Dreamhost customer for several years), to a service like Dropbox to a random anonymous text hosting service like Pastebin. Of course, you can also keep the HTML file on your desktop along with other documents.

Or you could save the entire app as a bookmark so you'll never be without it. There is an example unhosted text editor called Codemirror which is linked from this article which is implemented as a single serialized block of code that your web browser will execute if you click on the link. It can also be saved to your bookmarks like any other link. You can load and save local files and edit them as you like. You can also toggle syntax highlighting for several languages (JavaScript, HTML, and Markdown). I've copied the code into a gist at Github if you want to poke around in it, but be warned that it's a single very long line; if you're so inclined I'd recommend pasting it into a text editor first. While it looks like a mess I think it proves the point rather nicely that it's possible to write a completely self-contained webapp that you can save for later with nothing more than a web browser.

There are other unhosted apps floating around out there in various states of completion. I'll leave you to do some digging around on your own because I need to finish this post before I get distracted again. Among some of the more interesting projects out there are a peer-to-peer transaction system which appears to be a good start for asset tracking and a peer-to-peer application framework that is being used for everything from video chat to file sharing. If you're interested in experimenting with coding your own unhosted apps there are a few frameworks out there that implement basic functionality. remotestorage.js is a JavaScript library which allows your code to load from and store to arbitrary datastores on the Net. To put it another way, the RemoteStorage protocol acts as a translation layer that sits between unhosted apps and whatever database you want on the back end (from MySQL to CouchDB and grants JavaScript code access in a standardized fashion. There are free or low-cost RemoteStorage services out there (I use 5apps for my experiments) that cater to unhosted apps, or if you are so inclined you can set up one of your own using a number of different implementations of the protocol.

However, that is something that merits closer examination. When you're not using it, where does your data live? For relatively simple applications like text editors, the files are stored on the same system they were edited on. But what about more complex applications, like social networks or data mining software? Rather a lot of data will accumulate in a short time and it has to be stored in such a fashion that it can be accessed, searched, and updated rapidly. That means a database, but not many people are willing to set up a database server on their laptops (or their smartphones) just to manage their communications. That can be a fairly hefty undertaking that relatively few are sufficiently motivated to initiate. The alternative that most people would likely choose is to use a database hosted on a server somewhere, like at an ISP or web hosting company. The problem then becomes somebody gaining access to the database server and either denying the user access or seizing copies of sensitive information, which pretty much puts us back where we started. Just as you can buy a virtual machine running in the network of a hosting provider, you could set up your own remotestorage.js instance pretty easily, but then that's one more thing to worry about maintaining and backing up, plus not all good VPS providers are inexpensive enough to be an option.

An option that I'm particularly curious about right now is called PouchDB. PouchDB is a re-implementation of the noSQL database CouchDB (mentioned earlier) in pure JavaScript so it can be included in and accessed by unhosted applications. The APIs of the two databases are the same (in fact, they're compatible and can synchronize with one another), but PouchDB runs entirely in the context of your web browser as long as it's running, wherever you take it. This works because modern web browsers have a lesser-known feature called localStorage which allows your web browser to store arbitrary data for arbitrary periods of time (even surviving the termination of the browser and the off button) with a couple of simple JavaScript commands. PouchDB uses localStorage to hold data and provides an easy-to-use interface to do things with it. So, the data your unhosted apps use can be carried around with the applications on your device of choice. The problem then becomes hooking unhosted applications together to exchange information (such as social networking or transaction software), but I think that the WebP2P protocol is going to go a long way toward fixing that.

That's about all I have to say about unhosted web applications. If this technology is already on your radar, then great. Happy hacking. If not, I hope that I've made you curious about applications that basically live in text files that anybody with a web browser can access and use. If you've never programmed before but you're comfortable developing web pages, perhaps you'll do a little experimenting with the Unhosted library to see what you can make it do. If not, there isn't any harm in looking up a tutorial or two, grabbing a copy of Bootstrap, and trying your hand at it.