For the past couple of days I’ve been trying to figure out the best way to run an HTTP accelerator on the front-end of ScribbleLive. To handle the really spiky loads (20,000 GET/s) we get around large events, we do a lot of memory caching using .NET static variables and memcached for the bigger items that need to be distributed. But we’re still limited by IIS’s ability to serve the requests, which adds on a lot of WindowsAuth, OutputCaching, etc. overhead that really isn’t needed if we’re just serving static files. And it would be great to use something running on Linux (cheaper licensing and servers) on Amazon EC2 (easier to add new servers when we need them).
I’ve known about high performance servers like lighttpd that have been around for a while, and the new up-and-comer Nginx. I tried them out and they are amazing at serving-up content (PHP, Perl, mono, Rails) but their proxy abilities are still in their infancy. For example, Nginx when acting as a proxy has a Memcached Module which can check Memcached for an object, before going back to the origin/upstream server. However, you have to load your cached items into Memcached yourself, so it’s very useful yet. (Sidenote: the Nginx project is very impressive and I’m definitely going to watch them over the next year). Squid is the obvious choice when it comes to “proxy” and “caching”, but the other projects seem to perform much better under stress; not to mention, Squid configuration for a reverse-proxy is horrible.
After a lot of messing around in /etc config files, I stumbled on Varnish. Varnish isn’t a proxy product or a server like the others. It was built from-the-ground-up to be an HTTP accelerator, and it shows. Configuration is very straight-forward if you’ve used a CDN before, and it can do a lot of URL rewriting, managing cookies, edge-server includes (ESI), backend health polling, and support for inline C. The docs leave a lot to be desired, but you can’t have everything 
I got Varnish running on a Fedora virtual machine in Amazon EC2 with ./configure && make && make install. Then you just have to set-up a configuration via /usr/local/etc/varnish/default.vcl and launch the daemon.
On ScribbleLive, we use an ASP.NET session (stored in Memcached, with SQL 2005 failover) to keep track of your login. But until you login, we don’t use the session for anything. So my caching strategy is this: cache everything (images, stylesheets, etc) including ASPX files without any Set-Cookie headers until you login, then don’t cache ASPX files. Logging in is done via an HTTP POST, so I just told Varnish to not cache POSTs, and strip all cookies from cached responses unless an “ASP.NET_SessionId” cookie was sent in the request.
Here’s the configuration file I’m using which seems to work great. It remains to be seen if I actually roll this out into Production sometime soon, but using it myself in our Staging environment, it gives a massive improvement in page loads and origin offload, so I wouldn’t be surprised if someday soon when you visit ScribbleLive, you’ll be going through Varnish