Using Nginx to spoof HTTP Host headers.

Feb 02 2020

EDIT: s/alice.bob.com/alice.example.com/ to fix part of the backstory.

Let's say that you have a server (like Prosody) that has one or more subsystems (like BOSH and Websockets).  You want to stick them behind a web server like Nginx so that they can be accessed via HTTP - let's say that you want a browser to be able to communicate with those subsystems for some reason.  Or more likely you have a web application that needs to communicate with them in the same way (because Javascript).  Assuming that the above features are already enabled in Prosody, you would put something like this in one of your Nginx config files for, let's say for the sake of argument alice.example.com:

...
    location /http-bind {
        proxy_pass http://localhost:5280/http-bind;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_buffering off;
        tcp_nodelay on;
    }
    location /xmpp-websocket {
        proxy_pass http://localhost:5280/xmpp-websocket;
        proxy_http_version 1.1;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_read_timeout 900s;
    }
...

location is the part of the URL Nginx knows it has resources for.  proxy_pass tells Nginx that, whenever something tries to access that part of the URL (https://alice.example.com/http-bind or https://alice.example.com/xmpp-websocket) it should transparently proxy the connection to the given URL (http://localhost:5280/http-bind or /xmpp-websocket, depending) and forward responses back to the client).

But what if you did something a bit less sensible, like put the client on a different host?

I did this on my XMPP server not too long ago.  For reasons I don't fully understand converse.js stopped working for me and no amount of upgrading, rebuilding, or configuration tweaking got it going again.  But the remarkably plainly named xmpp-web application did, modulo putting it on a different host.  Here's what went wrong, and here's how I fixed it:

When you make an HTTP request a bunch of headers are part of it, like User-Agent (which describes your browser or other HTTP client), Accept-Language (which states what language you'd like the server to respond in, if capable), and Host, which tells the server what website you're asking about, because web servers handling multiple websites has been a thing for years.  The thing is, if you ask a webserver to send you content for a website it's not set up to handle, it won't find it and will return an error.  This also happens if a service the webserver is proxying doesn't know about that website... or in this case, value of the Host header.

Here's what happened, in a nutshell:

I have a Prosody XMPP server at alice.example.com.  It's configured to respond to alice.example.com.  So is Nginx on that server.  This means that BOSH and Websockets in Prosody are configured to respond to alice.example.com, via the Host header I just talked about.  However, for various annoying reasons I wasn't able to put a copy of xmpp-web at alice.example.com, I had to create a separate host called bob.example.com and put xmpp-web there instead.  As one might reasonably expect I configured xmpp-web to talk to alice.example.com:

// xmpp-web/local.js - config file
// Jabber web client found on bob.example.com
// Configured to talk to alice.example.com
var config = {
  name: 'XMPP Web Client',
  transports: {
    websocket: 'wss://alice.example.com/xmpp-websocket',
    bosh: 'https://alice.example.com/http-bind',
  },
  isTransportsUserAllowed: true,
  hasHttpAutoDiscovery: false,
  resource: 'XMPP Web Client',
  defaultDomain: 'alice.example.com',
}

I tried to log in to test it, of course.  And it didn't work.  The login attempt timed out over and over again.  So to troubleshoot I tried to hit the BOSH URL in the above config file with curl and see what I'd get:

[drwho @ windbringer ~] () $ curl https://bob.example.com/http-bind
<html><body>
<p>It works! Now point your BOSH client to this URL to connect to Prosody.</p>
<p>For more information see <a href="http://prosody.im/doc/setting_up_bosh">Prosody: Setting up BOSH</a>.</p>
</body></html>

This is expected.  So what's in the web server logs?

...
1.2.3.4 - - [02/Feb/2020:23:43:02 +0000] "POST /http-bind HTTP/1.1" 404 413 "https://bob.example.com/login?redirect=%2F" "Mozilla/5.0 (X11; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0"
...

What the hell?  That error message looks like it came from part of xmpp-web and not curl.

The answer to this part of the problem was relatively easy: Take that snippet of Nginx code above the fold out of the config file for alice.example.com and splice it into the one for bob.example.com, then restart nginx (sudo /usr/sbin/nginx -s reload) and give it another try with curl:

[drwho @ windbringer ~] () $ curl https://bob.example.com/http-bind
<!DOCTYPE html>
<html>
<head><meta charset="utf-8"><style>body{margin-top:14%;text-align:center;background-color:#F8F8F8;font-family:sans-serif;}h1{font-size:xx-large;}p{font-size:x-large;}p+p { font-size: large; font-family: courier }</style>
</head>
<body><h1>404 Not Found</h1><p>Whatever you were looking for is not here. Where did you put it?</p><p>Unknown host: bob.example.com</p>
</body>
</html>

Excuse me?

I scratched my head over this for quite a while.  After another cup or two of coffee I realized something: That wasn't an Nginx error, this looked different.  On a hunch I decided to grep Prosody's code for part of the error message:

drwho@box:~$ cd /usr/lib/prosody/modules/
drwho@box:/usr/lib/prosody/modules$ grep 'Whatever you were looking for is not here.' *
grep: adhoc: Is a directory
mod_http_errors.lua:        [404] = { "Whatever you were looking for is not here. %";
grep: mod_mam: Is a directory
grep: mod_pubsub: Is a directory
grep: mod_s2s: Is a directory
grep: muc: Is a directory
drwho@box:~$ 

As it turn out Prosody has a miniature web server built into it to support BOSH and Websockets.  If you configured xmpp-web to talk directly to Prosody instead of through Nginx (by using https://alice.example.com:5280/http-bind or /xmpp-websocket) you'd be talking directly to that little embedded webserver (spoiler alert, it still wouldn't work).  So, that meant that the problem was that Prosody was getting a bad hostname to talk to - an incorrect Host: header, in other words.  Now, I probably could have pulled some jiggery pokery to make Prosody equate alice.example.com and bob.example.com but I had no idea what it might break, and I didn't want to accidentally mess up anything that server's users had going on.

Remember that snippet of Nginx config file at the very beginning of this post?  The line that says "proxy_set_header Host $host;"?  What that basically means is that when an HTTP request is proxied and handed off to an underlying server, if the Host: header does not exist, add it to the request and set its value to the server name (bob.example.com), or rewrite it to bob.example.com if it is.  And that was the problem: A web server that only knew how to answer for alice.example.com was getting requests for bob.example.com and doing the right thing, returning an error.

Fun fact: proxy_set_header Host can be used to spoof the incoming hostname.  You don't have to just give it $host as an argument if you don't need to.  So, here's what I did in the Nginx config file for bob.example.com:

...
    location /http-bind {
        proxy_pass http://localhost:5280/http-bind;

        # --------------------------------------
        proxy_set_header Host alice.example.com;
        # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_buffering off;
        tcp_nodelay on;
    }
    location /xmpp-websocket {
        proxy_pass http://localhost:5280/xmpp-websocket;
        proxy_http_version 1.1;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Upgrade $http_upgrade;

        # --------------------------------------
        proxy_set_header Host alice.example.com;
        # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_read_timeout 900s;
    }
...

What this means is, when a request to bob.example.com gets proxied to Prosody, the Host: header is rewritten to alice.example.com.  The embedded web server knows how to answer for alice.example.com, so it responds the way we want it to.  Those responses get proxied back by Nginx (which doesn't mess with the headers on the way out, only on the way in).  The copy of xmpp-web I used to log into the server just cares that it's getting responses back, so I'm able to happily chat away with it.