New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rds_json not triggering on RDS output from Postgres #2

Closed
rjp opened this Issue Dec 1, 2011 · 17 comments

Comments

Projects
None yet
2 participants
@rjp

rjp commented Dec 1, 2011

Freshly compiled and installed ngx_openresty-1.0.10.21 running against Postgres 8.4 on Ubuntu Hardy.

location ~ /url/(?<num>\d+) {
    rds_json          on;
    rds_json_root     url;

    postgres_query    GET  "SELECT * FROM urls WHERE id=$num";
    postgres_pass     database;
    postgres_output   rds;
}

Result is always Content-Type: application/x-resty-dbd-stream. Where can I start looking to debug why this isn't triggering?

@rjp

This comment has been minimized.

rjp commented Dec 1, 2011

I should add that the correct data is returned, just in RDS format, not JSON.

@agentzh

This comment has been minimized.

Member

agentzh commented Dec 2, 2011

anything interesting in your nginx error.log?

@agentzh

This comment has been minimized.

Member

agentzh commented Dec 2, 2011

On Thu, Dec 1, 2011 at 9:32 PM, rjp
reply@reply.github.com
wrote:

Freshly compiled and installed ngx_openresty-1.0.10.21 running against Postgres 8.4 on Ubuntu Hardy.

   location ~ /url/(?\d+) {
       rds_json          on;
       rds_json_root     url;

       postgres_query    GET  "SELECT * FROM urls WHERE id=$num";
       postgres_pass     database;
       postgres_output   rds;
   }

I've tested your sample in both linux x86_64 and i386 and it works for
me. Here's the test case that I use:

https://github.com/agentzh/rds-json-nginx-module/blob/master/t/pg.t#L106

Result is always Content-Type: application/x-resty-dbd-stream. Where can I start looking to debug why this isn't triggering?

Is it possible that you forgot to reload the config for your nginx?

I can add more debugging outputs for you to ngx_rds_json if you're
still having this issue on your side.

Thanks!
-agentzh

@rjp

This comment has been minimized.

rjp commented Dec 2, 2011

Did a "killall nginx", nothing in the error log on startup or URL request. I've tried shuffling various bits of the config around, moving the postgres to its own server block with its own port, etc., with no success. I'm sure it's just something simple I've misunderstood in how all this fits together, sorry.

This is my current configuration: https://gist.github.com/1422845

And this is what I get:

rjp> curl http://new.frottage.org:8080/url/84764
?:plop?url?comment??wh?by   id  idnum   private?title?cache?    cache_url?source?   submitter?Z thcreated?Z
                                                                                                                       urlmodified????,http://www.award.szpilman.de/best11.hor.html????2011-12-01 16:50:25.614377+00moo@conference.jabber.pi.st/rjp84764????0,Szpilman Award 2011: Best Six: David Horvitz????????????????????????
@agentzh

This comment has been minimized.

Member

agentzh commented Dec 2, 2011

On Fri, Dec 2, 2011 at 7:21 PM, rjp
reply@reply.github.com
wrote:

Did a "killall nginx", nothing in the error log on startup or URL request. I've tried shuffling various bits of the config around, moving the postgres to its own server block with its own port, etc., with no success. I'm sure it's just something simple I've misunderstood in how all this fits together, sorry.

Thanks for providing the config file. But I'd like to learn more:

  1. output of the "/path/to/your/nginx -V" command
  2. output of the "uname -a" command

I'll also add more debugging output to ngx_rds_json soon. I'll let you
know when I'm done :)

Thanks for reporting this issue!
-agentzh

@rjp

This comment has been minimized.

rjp commented Dec 2, 2011

nginx: nginx version: ngx_openresty/1.0.10.21
nginx: built by gcc 4.2.4 (Ubuntu 4.2.4-1ubuntu4)
nginx: TLS SNI support enabled
nginx: configure arguments: --prefix=/usr/local/openresty/nginx --add-module=../ngx_devel_kit-0.2.17 --add-module=../echo-nginx-module-0.37rc7 --add-module=../xss-nginx-module-0.03rc6 --add-module=../set-misc-nginx-module-0.22rc3 --add-module=../form-input-nginx-module-0.07rc5 --add-module=../encrypted-session-nginx-module-0.01 --add-module=../ngx_postgres-0.9rc2 --add-module=../ngx_lua-0.3.1rc37 --add-module=../headers-more-nginx-module-0.16rc4 --add-module=../srcache-nginx-module-0.13rc2 --add-module=../array-var-nginx-module-0.03rc1 --add-module=../memc-nginx-module-0.13rc2 --add-module=../redis2-nginx-module-0.08rc2 --add-module=../upstream-keepalive-nginx-module-0.3 --add-module=../auth-request-nginx-module-0.2 --add-module=../rds-json-nginx-module-0.12rc6 --add-module=../rds-csv-nginx-module-0.04 --with-rtsig_module --with-select_module --with-poll_module --with-ipv6 --with-file-aio --with-http_addition_module --with-http_dav_module --with-http_gzip_static_module --with-http_stub_status_module --with-http_perl_module --with-mail --with-mail_ssl_module --with-pcre --with-libatomic --with-http_ssl_module

Linux backup 2.6.24-29-xen #1 SMP Wed Aug 10 19:09:00 UTC 2011 i686 GNU/Linux

Thanks for investigating.

@rjp

This comment has been minimized.

rjp commented Dec 2, 2011

I have done some experimenting and finally managed to get JSON output by removing everything but --with-http_postgres_module from the configure. Since the previous attempt had --with-http_postgres_module --with-poll_module --with-http_perl_module --with-mail --with-mail_ssl_module it seems likely that Perl or the email stuff is interfering.

@rjp

This comment has been minimized.

rjp commented Dec 2, 2011

Strange - if I configure a local copy with --with-http_postgres_module --with-poll_module --with-http_perl_module --with-mail --with-mail_ssl_module and run it by hand as my user, I get JSON output. But if I install it and run it as root under daemontools, no JSON output.

nginx: nginx version: ngx_openresty/1.0.10.21
nginx: built by gcc 4.2.4 (Ubuntu 4.2.4-1ubuntu4)
nginx: TLS SNI support enabled
nginx: configure arguments: --prefix=/usr/local/openresty/nginx --add-module=../ngx_devel_kit-0.2.17 --add-module=../echo-nginx-module-0.37rc7 --add-module=../xss-nginx-module-0.03rc6 --add-module=../set-misc-nginx-module-0.22rc3 --add-module=../form-input-nginx-module-0.07rc5 --add-module=../encrypted-session-nginx-module-0.01 --add-module=../ngx_postgres-0.9rc2 --add-module=../ngx_lua-0.3.1rc37 --add-module=../headers-more-nginx-module-0.16rc4 --add-module=../srcache-nginx-module-0.13rc2 --add-module=../array-var-nginx-module-0.03rc1 --add-module=../memc-nginx-module-0.13rc2 --add-module=../redis2-nginx-module-0.08rc2 --add-module=../upstream-keepalive-nginx-module-0.3 --add-module=../auth-request-nginx-module-0.2 --add-module=../rds-json-nginx-module-0.12rc6 --add-module=../rds-csv-nginx-module-0.04 --with-poll_module --with-http_perl_module --with-mail --with-mail_ssl_module --with-http_ssl_module

@rjp

This comment has been minimized.

rjp commented Dec 2, 2011

Even stranger: I can use the exact same nginx binary with the same configuration (except for log file and port changes) running it by hand from the command line (no daemontools or anything involved) and it still works fine for my user but not for root.

agentzh added a commit that referenced this issue Dec 5, 2011

@agentzh

This comment has been minimized.

Member

agentzh commented Dec 5, 2011

Hello!

Sorry for the delay on my side over the weekend :P

I've already added detailed debugging outputs to ngx_rds_json in
ngx_openresty devel version 1.0.10.23:

http://agentzh.org/misc/nginx/ngx_openresty-1.0.10.23.tar.gz

You can enable the debugging output like this:

  1. while building ngx_openresty, add the "--with-debug" option to the
    ./configure script.
  2. use the "debug" level in your error_log directive in your nginx.conf

You can see the debugging messages with the tag "rds json" at the
"[debug]" level in your nginx.conf.

Looking forward to your findings!

Thanks!
-agentzh

@rjp

This comment has been minimized.

rjp commented Dec 5, 2011

Downloaded, built with --with-debug, configured error.log to be debug and whilst I get debugging, there's no mention of rds.

https://gist.github.com/1434542 is the debugging output for my simple request with the same configuration as previously.

@agentzh

This comment has been minimized.

Member

agentzh commented Dec 6, 2011

On Tue, Dec 6, 2011 at 1:52 AM, rjp
reply@reply.github.com
wrote:

Downloaded, built with --with-debug, configured error.log to be debug and whilst I get debugging, there's no mention of rds.

Hmm, it seems that you did not enable rds_json in your nginx.conf (I
cannot imagine other possibilities). If you think you have, maybe
you're running nginx with another wrong file? Could you try using the
-c option to explicitly specify the absolute path of your nginx.conf
while starting your nginx? Like this:

/usr/local/openresty/nginx/sbin/nginx \
       -c /usr/local/openresty/nginx/conf/nginx.conf

https://gist.github.com/1434542 is the debugging output for my simple request with the same configuration as previously.

Yeah, "rds json" messages are indeed missing there. Try the above tip
to ensure that you're using the right nginx.conf file :)

Thanks!
-agentzh

@rjp

This comment has been minimized.

rjp commented Dec 6, 2011

Honestly, it's enabled. I changed my run script to specifically specify the specific file and ... still no output in the debugging. I'm beginning to suspect that something is terribly wrong with either Ubuntu Hardy, this GCC, or running openresty under daemontools.

If I run the same binary as my user, I get the JSON and "rds json" debugging output.

I've done some more digging with adding extra debugging into the C code and it seems that the filter is turned on but then turned off again.

2011-12-06 11:05:52.524464500 OpenResty starting up
2011-12-06 11:05:52.532966500 rds-json *** ngx_http_rds_json_pre_config: rds json filter=0 at ../rds-json-nginx-module-0.12rc7/src/ngx_http_rds_json_filter_module.c line 690.
2011-12-06 11:05:52.567461500 rds-json *** ngx_http_rds_json_filter_init: setting next filter at ../rds-json-nginx-module-0.12rc7/src/ngx_http_rds_json_filter_module.c line 345.
2011-12-06 11:05:52.567696500 rds-json *** ngx_http_rds_json_pre_config: rds json filter=0 at ../rds-json-nginx-module-0.12rc7/src/ngx_http_rds_json_filter_module.c line 690.
2011-12-06 11:05:52.567798500 rds-json *** ngx_http_rds_json: rds json filter=1 at ../rds-json-nginx-module-0.12rc7/src/ngx_http_rds_json_filter_module.c line 681.
2011-12-06 11:05:52.567942500 rds-json *** ngx_http_rds_json: rds json filter=1 at ../rds-json-nginx-module-0.12rc7/src/ngx_http_rds_json_filter_module.c line 681.
2011-12-06 11:05:52.575221500 rds-json *** ngx_http_rds_json_filter_init: setting next filter at ../rds-json-nginx-module-0.12rc7/src/ngx_http_rds_json_filter_module.c line 345.
2011-12-06 11:05:52.575270500 rds-json *** ngx_http_rds_json_filter_init: rds json filter set at ../rds-json-nginx-module-0.12rc7/src/ngx_http_rds_json_filter_module.c line 354.
2011-12-06 11:05:52.575479500 rds-json *** ngx_http_rds_json_pre_config: rds json filter=0 at ../rds-json-nginx-module-0.12rc7/src/ngx_http_rds_json_filter_module.c line 690.
2011-12-06 11:05:52.604791500 rds-json *** ngx_http_rds_json_filter_init: setting next filter at ../rds-json-nginx-module-0.12rc7/src/ngx_http_rds_json_filter_module.c line 345.

AND WITH A BLINDING FLASH OF INSPIRATION...I figured it out. I have another server configured after my postgres-fetching one and that didn't have "rds_json on" which turned it off globally. Is it supposed to be a global setting?

@agentzh

This comment has been minimized.

Member

agentzh commented Dec 6, 2011

On Tue, Dec 6, 2011 at 6:47 PM, rjp
reply@reply.github.com
wrote:

Honestly, it's enabled. I changed my run script to specifically specify the specific file and ... still no output in the debugging. I'm beginning to suspect that something is terribly wrong with either Ubuntu Hardy, this GCC, or running openresty under daemontools.

Well, I don't think it's a good idea to run nginx under daemontools
because nginx's master process does already have the functionality to
monitor and maintain all of its worker processes. And to use with
daemontools, you have to disable the "daemon" mode of your nginx
server.

If I run the same binary as my user, I get the JSON and "rds json"

Try manually using the "root" account to start your nginx server? Just
like this:

sudo /path/to/your/nginx ...

It's fine to configure a unprivileged user account (like "nobody") for
your nginx worker processes.

Regards,
-agentzh

@rjp

This comment has been minimized.

rjp commented Dec 6, 2011

It all works now - see the bottom bit of my previous comment. "rds_json on" seems to be a global setting - all the http{} blocks have to set it (or just the last one, I guess.)

@agentzh

This comment has been minimized.

Member

agentzh commented Dec 6, 2011

On Tue, Dec 6, 2011 at 10:03 PM, rjp
reply@reply.github.com
wrote:

It all works now - see the bottom bit of my previous comment. "rds_json on" seems to be a global setting - all the http{} blocks have to set it (or just the last one, I guess.)

That's interesting. I've never tested multiple http {} blocks in a
single nginx.conf file. I'll investigate it later. Seems like an
optimization in ngx_rds_json is causing troubles in this case :)

Thanks!
-agentzh

@agentzh

This comment has been minimized.

Member

agentzh commented May 31, 2012

Using multiple http {} blocks could have various issues and I think it should really be disabled in the nginx config loader.

Also, I've seen the following conversation on #nginx:

<krogant> hi, is it possible to have multiple http blocks in the nginx conf? ...
<mdounin> krogant: no (well, it may accidently work, but it's not expected to); ...

It seems that even Maxim Dounin denies this :)

So I'm closing this :)

@agentzh agentzh closed this May 31, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment