Comments on: Lighttpd as reverse proxy

By: Brian

Brian — Tue, 24 Jun 2008 20:50:21 +0000

peter, it’s not hard to get Apache to be on a diet, just don’t load stuff you don’t need. Plus, apache 2.x, with full threading, has been out for several years. Also, the whole “a server must be event/async to perform well” mantra doesn’t hold up in the real world. I like alot about lightttpd and nginx, but the performance FUD is just that. It just seems they come with a more “reasonable” default config.

By: peter

peter — Tue, 24 Jun 2008 20:46:07 +0000

Brian,

I think when we speak about Apache we mainly mean apache in prefork mode w process per connection which obviously does not scale. Recent apache versions have event driven handling which should place them close to lighty and nginx. There is still more “fat” in apache though for most users it does not matter.

By: peter

peter — Tue, 24 Jun 2008 20:43:12 +0000

Lukas,

We tried using Varnish for one of the projects but had so many crashes we had to pull back to the squid for a while.
I think Varnish is a great project but it needs to mature still, at least for some use cases.

By: Brian

Brian — Tue, 24 Jun 2008 18:42:33 +0000

JohnSep, of course we don’t do 50k all day long… But 50k connections on a “normal” server is not that “hard” for apache. You have to tune your OS and Apache - not as much as you would think - but you have to do that anyway. The GigE interface is almost always the bottleneck.

An old presentation with 2004 data I did : http://www.akins.org/files/apachecon-cnn-good.ppt

Most of the cool stuff is in normal Apache now (or some implementation of it)

By: Simetrical

Simetrical — Tue, 24 Jun 2008 17:45:02 +0000

FWIW, an old Wikimedia report[1] says that they maintain only 14,000-32,000 connections per *Squid*, never mind the Apaches (which are presumably much lower). Of course, they use a mod_php setup for the Apaches, apparently because nobody’s ever gotten around to changing it.

[1] http://www.scribd.com/doc/43872/Wikimedia-architecture (not the original source, but the first Google hit)

By: JohnShep

JohnShep — Tue, 24 Jun 2008 17:37:38 +0000

Brian, I am dumbfounded that one server can handle 50k connections. Say each request is processed in a second that would equate to over 4 Billion hits a day !

By: Brian

Brian — Tue, 24 Jun 2008 17:30:09 +0000

We just use apache. We can handle around 50k connections fairly per server easily. We run all the dynamic stuff via fastcgi or mod_jk to Tomcat for Java. The whole “apache is slow” thing doesn’t make sense if you actually try to configure apache beyond the configs most vendors supply.

By: Lukas

Lukas — Thu, 19 Jun 2008 12:16:37 +0000

@6: Yes, it is. Using lighty in front of Apache instead of in parallel is a somewhat strange construct but depending on your setup/overall goal it might make sense.

Though what most ppl want to achieve is a faster and less CPU&IO intensive delivery of their static contents.

Now squid hasn’t been the best choice to do this for about 10 years now, since it’s internal architecture is somewhat 80s style (find a good description of the details at http://varnish.projects.linpro.no/wiki/ArchitectNotes). There’s been commercial alternatives like cachebox and netcache around for a while now.

A very nice, open source, alternative is varnish (http://varnish.projects.linpro.no/). It provides a pretty flexible scripting language that gets compiled on the fly and offers fine-grained control over “what to do with this http request”. We measured a throughput of about 10.000 requests/sec. on a single Core2Duo Desktop Box using just one of the two cores. On a typical modern Xeon 2x Quadcore Server you should expect to get something around 40-80k requests/sec - that is, if your servers network connection offers sufficient bandwith.

A quick apache bench from my macbook pro to the desktop box mentioned shows the following result on a GigE network:
=========================
Server Software: Apache/2.2.3
Server Hostname: cache2.lcms.c4.dev.local
Server Port: 80

Document Path: /login.html
Document Length: 2241 bytes

Concurrency Level: 200
Time taken for tests: 10.946997 seconds
Complete requests: 100000
Failed requests: 0
Write errors: 0
Total transferred: 254900000 bytes
HTML transferred: 224100000 bytes
Requests per second: 9134.93 [#/sec] (mean)
Time per request: 21.894 [ms] (mean)
Time per request: 0.109 [ms] (mean, across all concurrent requests)
Transfer rate: 22739.11 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 3 2.4 3 15
Processing: 4 18 1.8 18 27
Waiting: 0 7 1.9 8 17
Total: 6 21 3.3 21 32

Percentage of the requests served within a certain time (ms)
50% 21
66% 23
75% 24
80% 24
90% 26
95% 27
98% 27
99% 28
100% 32 (longest request)
=========================

So, if all you want to do is accelerate access to your static content, varnish might be a pretty good option. Of course lighty offers alot more than just content caching so I guess depending on what one’s goals are the one or the other might be better suited.

Cheers,
– Lukas

By: peter

peter — Thu, 19 Jun 2008 06:19:13 +0000

Emil,

Why do you do that ? for mod_security ?

By: Vadim

Vadim — Thu, 19 Jun 2008 05:08:45 +0000

To all.

Regarding lighttpd vs ngnix.

Lighttpd is used by youtube.com and wikipedia, so it is proven to be able to work in high load environment.

When we started use lighttpd it seemed more advanced than ngnix, but currently ngnix is still begin active developed, while author of lighttpd is more interested in mysql-proxy development and lighttpd looks frozen, so I even agree that ngnix looks preferable.