Keybase and Git.

29 November 2017

A couple of weeks ago a new release of the Keybase software package came out, and this one included as one of its new features support for natively hosting Git repositories.  This doesn't seem like it's very useful for most people, and it might really only be useful to coders, but it's a handy enough service that I think it's worth a quick tutorial.  Prior to that feature release something in the structure of the Keybase filesystem made it unsuitable for storing anything but static copies of Git repositories (I don't know exactly waht), but they've now made Git a first class citizen.

I'm going to assume that you use the Git distributed version control system already, and you have at least one Git repository that you want to host on Keybase; for the purposes of this example I'm going to use my personal copy of the Exocortex Halo code repository on Github.  I'm further going to assume that you know the basics of using Git (cloning repositories, committing changes, pulling and pushing changes).  I'm also going to assume that you already have a Keybase account and a fairly up-to-date copy of the software installed.  I am, however, going to talk a little bit about the idea of remotes in Git.  My discussion will necessarily have some technical inaccuracies for the sake of usability if you're not an expert on the internals of Git.

If you've been using Git for a while, you should have some understanding of what it means for Git to be a distributed version control system.  When you clone a Git repo, you get a full copy of the entire history of the repository dating back to the very first commit (change to the state of the files in the repository). By convention, there is one "core" or "server" copy of the repository on a server somewhere that everyone copies and pushes edits to.  This does not have to be the case, however.  It is common for people to use Git repositories that exist nowhere other than their laptop.  For example, I write most of my presentations' slides using a package called Landslide, and I keep the source documents in a Git repository on Windbringer.  The Exocortex Halo repository, however, is hosted on Github, so every copy of the source code has a pointer to that "core" repository.  This pointer is called a remote, and it can be viewed like this:

[drwho@windbringer exocortex-halo]$ git remote -v
origin    git@github.com:virtadpt/exocortex-halo.git (fetch)
origin    git@github.com:virtadpt/exocortex-halo.git (push)

Again, by convention, that "core" repository at Github is called the "origin", and is what gets fetched from (when updates are pulled down) and pushed to (when changes are committed back to the origin for archival and dissemination).  Because the "origin" repository isn't on my laptop's hard drive, it's also referred to as a "remote".  In one sense, it's the default repo used when I do a git push to sync Github with my local repository:

[drwho@windbringer exocortex-halo]$ git push --verbose
Pushing to git@github.com:virtadpt/exocortex-halo.git
X11 forwarding request failed on channel 1
To github.com:virtadpt/exocortex-halo.git
 = [up to date]      master -> master
updating local tracking ref 'refs/remotes/origin/master'
Everything up-to-date

This doesn't mean that you can only have one set of remotes, though.  I have another Git repository that I back up my Huginn scenarios into that is kept in three places: On Windbringer, on Leandra (at home), and on Keybase:

[drwho@windbringer agents]$ git remote -v
keybase    keybase://private/drwho/agents (fetch)
keybase    keybase://private/drwho/agents (push)
origin    ssh://leandra/home/drwho/agents (fetch)
origin    ssh://leandra/home/drwho/agents (push)

Note that there are two pairs of remotes: "origin" (which syncs to my server Leandra) and "keybase" (which is stored inside of Keybase).  The copy of Git installed on Windbringer knows about a remote-helper utility (think of it like a plugin system for supporting new kinds of remotes) called git-remote-keybase that just happens to know how to interact with the Keybase filesystem.  Git can find it because its name starts with git-remote and it's in the user's path.  As far as Git knows or cares, it calls out to that remote-helper when the user interacts with the remote called "keybase" and waits to hear about a success or a failure.  No fancy jiggery pokery is required to make it work.  Here's how you set it up:

Open the Keybase desktop app and click on "New encrypted git repository..."  Select "New personal git repository" and give it a name.  For the sake of our example, let's call it "exocortex-halo".  Click the "Create" button.  It'll take a minute or two to complete so don't worry if it's not immediate.  When it's done, click on the repo in the Keybase app and see what the Git remote is.  In this case, the remote's URL is keybase://private/drwho/exocortex-halo.

Now to add the Keybase remote to the existing repo.  Here's how I did it:

[drwho@windbringer ~]$ cd exocortex-halo/
[drwho@windbringer exocortex-halo]$ git remote add keybase
    keybase://private/drwho/exocortex-halo
[drwho@windbringer exocortex-halo]$ git push keybase
Initializing Keybase... done.
Syncing with Keybase... done.
Syncing encrypted data to Keybase: (100.00%) 5.97/5.97 KB... done.
Counting and decrypting: 1.90K objects... done.
Preparing and encrypting: (100.00%) 1.90/1.90K objects... done.
Indexing hashes: (100.00%) 1.90/1.90K objects... done.
Indexing CRCs: (100.00%) 1.90/1.90K objects... done.
Indexing offsets: (100.00%) 1.90/1.90K objects... done.
Syncing encrypted data to Keybase: (100.00%) 142.98/142.98 KB... done.
Syncing encrypted data to Keybase: (100.00%) 4.06/4.06 KB... done.
To keybase://private/drwho/exocortex-halo
 * [new branch]      master -> master

Now there is a copy of the exocortex-halo repository stored in my account at Keybase.  If you look at the list of configured remotes for that repo, you should see the Keybase remotes for fetch and push alongside the "origin" ones at Github:

[drwho@windbringer exocortex-halo]$ git remote -v
keybase    keybase://private/drwho/exocortex-halo (fetch)
keybase    keybase://private/drwho/exocortex-halo (push)
origin    git@github.com:virtadpt/exocortex-halo.git (fetch)
origin    git@github.com:virtadpt/exocortex-halo.git (push)

Now that this is done, the repo at Keybase can be accessed just like any other remote:

  • git push keybase (pushing changes into Keybase)
  • git fetch keybase (pulling updates from Keybase)

I primarily use this for backing up source code and documents but there's nothing that says that you can't host a project entirely inside of Keybase.