Ruby Forum NGINX > nginx-0.7.9

Changes with nginx 0.7.9                                         12 Aug 
2008

    *) Change: now ngx_http_charset_module works by default with 
following
       MIME types: text/html, text/css, text/xml, text/plain,
       text/vnd.wap.wml, application/x-javascript, and 
application/rss+xml.

    *) Feature: the "charset_types" and "addition_types" directives.

    *) Feature: now the "gzip_types", "ssi_types", and 
"sub_filter_types"
       directives use hash.

    *) Feature: the ngx_cpp_test_module.

    *) Feature: the "expires" directive supports daily time.

    *) Feature: the ngx_http_xslt_module improvements and bug fixing.
       Thanks to Denis F. Latypoff and Maxim Dounin.

    *) Bugfix: the "log_not_found" directive did not work for index 
files
       tests.

    *) Bugfix: HTTPS connections might hang, if kqueue, epoll, rtsig, or
       eventport methods were used; the bug had appeared in 0.7.7.

    *) Bugfix: if the "server_name", "valid_referers", and "map" 
directives
       used an "*.domain.tld" wildcard and exact name "domain.tld" was 
not
       set, then the exact name was matched by the wildcard; the bugs 
had
       appeared in 0.3.18.

Did Maxim's index patch get in there? to cycle through all the index
files before going to a directory scan?

On Tue, Aug 12, 2008 at 09:47:15AM -0700, mike wrote:

> Did Maxim's index patch get in there? to cycle through all the index
> files before going to a directory scan?

No, I need think over.

On 8/12/08, Igor Sysoev <is@rambler-co.ru> wrote:
> On Tue, Aug 12, 2008 at 09:47:15AM -0700, mike wrote:
>
> > Did Maxim's index patch get in there? to cycle through all the index
> > files before going to a directory scan?
>
> No, I need think over.

i think it makes perfect sense. if people don't want a bunch of files
to be tested, don't list them explicitly in the "index" directive. it
gives them control over that now :)

and the log_not_found patch did get in right?

Hello!

On Tue, Aug 12, 2008 at 09:30:28PM +0400, Igor Sysoev wrote:

>On Tue, Aug 12, 2008 at 09:47:15AM -0700, mike wrote:
>
>> Did Maxim's index patch get in there? to cycle through all the index
>> files before going to a directory scan?
>
>No, I need think over.

Some random thoughts about:

- Many index files is rare setup.  Setups I've seen usually have at
most 3, more often 1 or 2.  In case of 2 index'es as far as I
understand it doesn't make sence to test directory at all - for
performance reasons.

- Situation where something under root closed by unix permissions
from nginx is rare.  And this may be optimized by adding appropriate
location{} to nginx config.

- Situation where directory listing is forbidden (i.e. 711
permissions) may make sence in shared environments.

- Optimization is good, but only if doesn't change expected
behaviour.

 From the above I think that fallback to sequential index tests in
case of EACCES returned for directory test is good or at least
acceptable solution.

Am I missed something?

Maxim Dounin

On 8/12/08, Maxim Dounin <mdounin@mdounin.ru> wrote:
> >
> > No, I need think over.
> >
>
> Some random thoughts about:
>
> - Many index files is rare setup.  Setups I've seen usually have at most 3,
> more often 1 or 2.  In case of 2 index'es as far as I understand it doesn't
> make sence to test directory at all - for performance reasons.

I run into at least 2-3 index files quite often.

> - Situation where directory listing is forbidden (i.e. 711 permissions) may
> make sence in shared environments.

Correct. Never had this issue in the past with other webservers.

> - Optimization is good, but only if doesn't change expected behaviour.

How is this an optimization? If the first test fails, then check
directory listing? Isn't it faster to take an enumerated list of
parameters and check those first? You can test for a specific file, as
opposed to opening a directory handle to check the listing or
whatever.

> From the above I think that fallback to sequential index tests in case of
> EACCES returned for directory test is good or at least acceptable solution.

I think your patch works perfectly. If performance is in people's
minds, then they can easily do this themselves by defining a single
index file - it will behave exactly like it does today. However, for
those of us who need a couple index file options (or those with
/index.html /index.$lang.html or whatever the examples are on the wiki
even) Maxim's patch seems to make nginx behave properly in my opinion.

The user has control over how many index files they define, so they
can control their own performance, or order the index files in the
manner in which is quickest for them. It's just simple file testing,
as opposed to opening up the directory before going forward with the
rest of the index file checking. That is actually a negative
performance hit, not a positive one.

Today:

my config might be "index index.html index.php index.htm"

1) check index.html - not found
2) check directory
3) check index.php - not found
4) check index.htm - found
5) if it wasn't found, then do directory listing, or throw access
denied 403 or whatever if not

Maxim's patch:

1) check index.html - not found
2) check index.php - not found
3) check index.htm - found
4) if it wasn't found, then do directory listing, or throw access
denied 403 or whatever if not

Why bother with a directory check? It's an extra step in the middle of
cycling through -predefined- index filenames.

Hello!

On Tue, Aug 12, 2008 at 03:35:40PM -0700, mike wrote:

>On 8/12/08, Maxim Dounin <mdounin@mdounin.ru> wrote:

[...]

>> - Optimization is good, but only if doesn't change expected behaviour.
>
>How is this an optimization? If the first test fails, then check
>directory listing? Isn't it faster to take an enumerated list of
>parameters and check those first? You can test for a specific file, as
>opposed to opening a directory handle to check the listing or
>whatever.

It doesn't "check the listing".  It only checks existence of
directory.  This saves some syscalls if directory doesn't exists
and more than two index files configured.

Maxim Dounin

On 8/12/08, Maxim Dounin <mdounin@mdounin.ru> wrote:

> It doesn't "check the listing".  It only checks existence of directory.
> This saves some syscalls if directory doesn't exists and more than two index
> files configured.

Hmm.

In which case, I guess I don't care, it -adds- a syscall for me where
I don't need one, but as long as it does not error out on a 0711 web
directory (the dir exists, it just is not executable/listable for
non-owner!)

Personally I like allowing the user to customize how many items they
check for. That gives ultimate control of performance to the end user.
If they don't like extra syscalls, force them to rename their files or
something. :p

On 8/12/08, mike <mike503@gmail.com> wrote:

> directory (the dir exists, it just is not executable/listable for
> non-owner!)
>
> Personally I like allowing the user to customize how many items they
> check for. That gives ultimate control of performance to the end user.
> If they don't like extra syscalls, force them to rename their files or
> something. :p
>

was any decision ever made on this?

was hoping to see it in .10... it adds a syscall if there is more than
one index file enabled (from my logic) and it doesn't hurt anything if
there's only one defined since it falls back to normal directory
behavior anyway right?

the only people this affects negatively are those with multiple index
files defined who have become used to how nginx handles it and are
used to that many syscalls/order of operations. and that is as simple
as changing their config or renaming their files so it matches the
first index file defined...

thanks

On Tue, Aug 12, 2008 at 03:35:40PM -0700, mike wrote:

> 
> 2) check index.php - not found
> 3) check index.htm - found
> 4) if it wasn't found, then do directory listing, or throw access
> denied 403 or whatever if not
> 
> Why bother with a directory check? It's an extra step in the middle of
> cycling through -predefined- index filenames.

If no one index file is found than directory test must be done anyway:
*) if directory exists, but no index files exist, then nginx should
   pass control to next module (autoindex) and may eventually return 
403.
*) if directory does not exist, then nginx should return 404.

There no way in Unix to know in single syscall if intermidiate 
directories
exist when file not found. Win32 returns ERROR_PATH_NOT_FOUND in this 
case
instead of ERROR_FILE_NOT_FOUND.

On 8/16/08, Igor Sysoev <is@rambler-co.ru> wrote:

> If no one index file is found than directory test must be done anyway:
> *) if directory exists, but no index files exist, then nginx should
>   pass control to next module (autoindex) and may eventually return 403.
> *) if directory does not exist, then nginx should return 404.

but doesn't it make sense to go through all index file options before
checking the directory and handing it off to autoindex? maybe i just
don't know filesystem architecture good enough, but it seems like if
the user states "check A, B, C" it should check A, B, C and only if it
fails all 3 then fall back to sanity checks / directory checks /
autoindex checks / etc.

On Sat, Aug 16, 2008 at 12:32:30PM -0700, mike wrote:

> the user states "check A, B, C" it should check A, B, C and only if it
> fails all 3 then fall back to sanity checks / directory checks /
> autoindex checks / etc.

There 3 reasons:

1) you assume that user will ask only existent directories.

2) directory test should be cheap syscall, because kernel has already 
read,
   tested, and cached file path after first open().

3) the last index may be default absolute file, like /index.html.

On 8/16/08, Igor Sysoev <is@rambler-co.ru> wrote:

> 1) you assume that user will ask only existent directories.
>
> 2) directory test should be cheap syscall, because kernel has already read,
>   tested, and cached file path after first open().
>
> 3) the last index may be default absolute file, like /index.html.

okay, since you've created nginx and it's amazingly fast you're the
resident expert on this - as long as it doesn't error on a 0711
directory i will be happy enough. if this can be put into .11 i will
be happy :)