Comments on: Pitfall of proxying HTTP requests through Lighttpd http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/ Percona's MySQL & InnoDB performance and scalability blog Sat, 11 Feb 2012 16:45:54 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: Dieter@be http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/comment-page-1/#comment-692396 Dieter@be Tue, 08 Dec 2009 10:57:46 +0000 http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/#comment-692396 > The store-and-forward nature of lighttpd is good for getting data away from the backends as quickly as possible regardless of client > speed, thus releasing them to handle other requests. Are you sure of this? A while back I also assumed such logic and implemented nginx proxying in front of an apache webfarm (+-25 webservers). Too my surprise however, it didn't help at all: the apache servers were as loaded as before. Later, a more experienced colleague explained me that the Linux kernel will automatically change the size of the outgoing tcp buffers so that they are big enough to hold most of the outgoing reponses entirely, yet small enough to not waste too much ram. So the result would be that when apache has processed an http request and generated the response, it can put it in the tcp buffer and start handling a new request. I.e. if the outgoing response all fits in the buffer, the kernel buffers the data as long as the client is in the middle of getting his data. Dieter > The store-and-forward nature of lighttpd is good for getting data away from the backends as quickly as possible regardless of client > speed, thus releasing them to handle other requests.

Are you sure of this? A while back I also assumed such logic and implemented nginx proxying in front of an apache webfarm (+-25 webservers). Too my surprise however, it didn’t help at all: the apache servers were as loaded as before. Later, a more experienced colleague explained me that the Linux kernel will automatically change the size of the outgoing tcp buffers so that they are big enough to hold most of the outgoing reponses entirely, yet small enough to not waste too much ram.
So the result would be that when apache has processed an http request and generated the response, it can put it in the tcp buffer and start handling a new request. I.e. if the outgoing response all fits in the buffer, the kernel buffers the data as long as the client is in the middle of getting his data.

Dieter

]]> By: Dieselstation http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/comment-page-1/#comment-204991 Dieselstation Mon, 26 Nov 2007 14:04:49 +0000 http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/#comment-204991 Julian & Maciej, Considering your input, I will change my server setup and see how it behaves over a period of week. Dieselstation serves high res wallpapers, so probably it makes sense to serve images directly through lighttpd instead of proxying the request from apache to lighttpd. Julian & Maciej,

Considering your input, I will change my server setup and see how it behaves over a period of week. Dieselstation serves high res wallpapers, so probably it makes sense to serve images directly through lighttpd instead of proxying the request from apache to lighttpd.

]]> By: peter http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/comment-page-1/#comment-204361 peter Sun, 25 Nov 2007 18:46:58 +0000 http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/#comment-204361 I spoke to Jan about this case a while back and he was going to add on disk buffering for large sizes though I'm not sure if this was done in lighty 1.5 or waits for next version. I spoke to Jan about this case a while back and he was going to add on disk buffering for large sizes though I’m not sure if this was done in lighty 1.5 or waits for next version.

]]> By: Michael Monashev http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/comment-page-1/#comment-204246 Michael Monashev Sun, 25 Nov 2007 14:58:09 +0000 http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/#comment-204246 nginx forever! :-) nginx forever! :-)

]]> By: Maciej Dobrzanski http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/comment-page-1/#comment-203656 Maciej Dobrzanski Sat, 24 Nov 2007 21:37:31 +0000 http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/#comment-203656 Dieselstation, As Julian pointed out configuring it other way around makes no sense as Apache is the heavy one. The correct setup is to avoid any larger objects sent through Lighttpd proxy, but to serve them with Lighty directly. But this, obviously, may not be possible to do however in case of bugs as mentioned in my post :) Dieselstation,

As Julian pointed out configuring it other way around makes no sense as Apache is the heavy one. The correct setup is to avoid any larger objects sent through Lighttpd proxy, but to serve them with Lighty directly. But this, obviously, may not be possible to do however in case of bugs as mentioned in my post :)

]]> By: Julian http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/comment-page-1/#comment-203475 Julian Sat, 24 Nov 2007 15:23:09 +0000 http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/#comment-203475 @Dieseelstation: Such a setup would be nonsense, because you would have to start a 'big' apache process just to proxy your static content which is served by lighttpd. Just run both on different ports and do no proxying. @Dieseelstation:
Such a setup would be nonsense, because you would have to start a ‘big’ apache process just to proxy your static content which is served by lighttpd. Just run both on different ports and do no proxying.

]]> By: Dieselstation http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/comment-page-1/#comment-203005 Dieselstation Sat, 24 Nov 2007 02:27:21 +0000 http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/#comment-203005 Do you think this problem would exists if the setup was reversed? Meaning, let Apache be your prime server and do the proxy forward for the static content to the lighttpd! In that case, lighttpd will not consume as much memory and will do its job efficiently. Do you think this problem would exists if the setup was reversed? Meaning, let Apache be your prime server and do the proxy forward for the static content to the lighttpd! In that case, lighttpd will not consume as much memory and will do its job efficiently.

]]> By: davies http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/comment-page-1/#comment-202577 davies Fri, 23 Nov 2007 13:55:45 +0000 http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/#comment-202577 o, I will check it again, thank you! o, I will check it again, thank you!

]]> By: citrin http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/comment-page-1/#comment-202366 citrin Fri, 23 Nov 2007 08:51:51 +0000 http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/#comment-202366 nginx with default config in such situation buffer upstream response on disk. nginx with default config in such situation buffer upstream response on disk.

]]> By: davies http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/comment-page-1/#comment-202289 davies Fri, 23 Nov 2007 07:05:02 +0000 http://www.mysqlperformanceblog.com/2007/11/22/pitfall-of-proxying-requests-with-lighttpd/#comment-202289 I had the problem with Nginx too, when i downloaded a big file (750M), and using Nginx as a reversed proxy before lighttpd. I had the problem with Nginx too, when i downloaded a big file (750M), and using Nginx as a reversed proxy before lighttpd.

]]>