Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Once setup, where do you set Persona or Google Keys so you can login to admin? #27

Closed
refindly opened this issue Jun 2, 2014 · 13 comments

Comments

@refindly
Copy link

refindly commented Jun 2, 2014

Hello, I have everything setup and running, however I do not know how to login as admin. My two options is google or persona. Where do I put the keys in the code to enable myself to login to admin?

@GUI
Copy link
Member

GUI commented Jun 3, 2014

Are you running this in the development environment or in staging/production/something else? In development mode, you should also have the "Login with a dummy login (development only)" option, which will allow you to login to the admin with any made up username.

However, if you're deploying this to a non-development environment, then Persona or Google will indeed be your default login options. In that case, there's a few manual steps you'll need to go through the first time:

  • You'll need to pre-create your very first admin login manually (from then on, you can use the web admin to create further admins). To manually create your first user directly in the database, you could use the Rails console to do this something like:

    $ cd /vagrant/workspace/web # Or wherever your api-umbrella-web app root is
    $ rails c production # Or whatever environment, if not production
    irb(main):001:0> Admin.create!(:username => "your.email@here.com")
  • That should be all that's necessary if you want to login with Persona.

  • If you want to login with Google, then you'll also need to go through the process of setting up OAuth2 credentials with Google. Google's instructions provide the bulk of the details, but the main thing to note is that after you get your Client ID and Client Secret from Google, you'll need to add environment variables for the app to use. An easy way to set these environment variables is to use the .env file inside the root of the api-umbrella-web application and fill it out like:

    GOOGLE_OAUTH_CLIENT_ID: YOUR_CLIENT_ID_HERE
    GOOGLE_OAUTH_CLIENT_SECRET: YOUR_CLIENT_SECRET_HERE

Sorry this process isn't a bit smoother and better documented--we're in the process of trying to improve the installation and setup process, but in the meantime, give us a shout if you have any questions.

@refindly
Copy link
Author

refindly commented Jun 3, 2014

Thank you for the information. I got the admin user inserted, and logged in with persona. It showed verified, but after it tried to login I got:

We're sorry, but something went wrong.
We've been notified about this issue and we'll take a look at it shortly.

Unsure what to try now. Let me know if you have any ideas. Thanks again for the help.

@refindly
Copy link
Author

refindly commented Jun 3, 2014

Hello, Not sure if this helps, below are some of the logs I get:

$
2014/06/03 12:50:45 [error] 1198#0: *52 connect() failed (111: Connection refused) while connecting to upstream, client: 200.29.238.180, server: _, request: "HEAD / HTTP/1.0", upstream: "http://127.0.0.1:50090/"
2014/06/03 15:39:13 [error] 1198#0: *98 connect() failed (111: Connection refused) while connecting to upstream, client: 184.0.166.133, server: developer.refindly.com, request: "GET /api/api-users/benchmark_delayed_cached HTTP/1.1", ups$
2014/06/03 15:40:17 [error] 4401#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 184.0.166.133, server: developer.refindly.com, request: "GET /api/api-users/benchmark_delayed_cached HTTP/1.1", upst$
2014/06/03 17:13:41 [error] 1215#0: *9 connect() failed (111: Connection refused) while connecting to upstream, client: 222.186.56.7, server: _, request: "GET /manager/html HTTP/1.1", upstream: "http://127.0.0.1:50090/manager/html", hos$
2014/06/03 17:47:00 [error] 1215#0: *11 connect() failed (111: Connection refused) while connecting to upstream, client: 117.21.191.206, server: _, request: "GET /manager/html HTTP/1.1", upstream: "http://127.0.0.1:50090/manager/html", $
2014/06/03 18:58:44 [error] 1215#0: *14 connect() failed (111: Connection refused) while connecting to upstream, client: 115.239.248.56, server: _, request: "GET /manager/html HTTP/1.1", upstream: "http://127.0.0.1:50090/manager/html", $
2014/06/03 20:46:46 [error] 1215#0: *34 connect() failed (111: Connection refused) while connecting to upstream, client: 89.248.160.155, server: _, request: "GET //cgi-bin/php HTTP/1.1", upstream: "http://127.0.0.1:50090//cgi-bin/php", $
2014/06/03 20:46:46 [error] 1215#0: *36 connect() failed (111: Connection refused) while connecting to upstream, client: 89.248.160.155, server: _, request: "GET //cgi-bin/php5 HTTP/1.1", upstream: "http://127.0.0.1:50090//cgi-bin/php5"$
2014/06/03 20:46:46 [error] 1215#0: *38 connect() failed (111: Connection refused) while connecting to upstream, client: 89.248.160.155, server: _, request: "GET //cgi-bin/php-cgi HTTP/1.1", upstream: "http://127.0.0.1:50090//cgi-bin/ph$
2014/06/03 20:46:46 [error] 1215#0: *40 connect() failed (111: Connection refused) while connecting to upstream, client: 89.248.160.155, server: _, request: "GET //cgi-bin/php.cgi HTTP/1.1", upstream: "http://127.0.0.1:50090//cgi-bin/ph$
2014/06/03 20:46:46 [error] 1215#0: *42 connect() failed (111: Connection refused) while connecting to upstream, client: 89.248.160.155, server: _, request: "GET //cgi-bin/php4 HTTP/1.1", upstream: "http://127.0.0.1:50090//cgi-bin/php4"$

@GUI
Copy link
Member

GUI commented Jun 6, 2014

Very sorry for the delay in following up! I've was meaning to run through a fresh setup of my own to work out any of these kinks this week, but haven't quite gotten to that yet (but hopefully sometime in the next few days). In the meantime, if the problem is what I think it might be, you might give this a spin:

$ cd /vagrant/workspace/web # Or wherever your api-umbrella-web app root is
$ RAILS_ENV=production rake db:seed # Or whatever environment, if not production

I think that might be what's triggering this error on initial setup and what needs to be integrated into the setup steps.

If that doesn't do the trick, can you tail /vagrant/workspace/web/log/development.log (again substituting for wherever your api-ubmrella-web app root is and what environment you're running the app in)? The error details should show up in that Rails log file whenever you load the page and get the error message.

@refindly
Copy link
Author

refindly commented Jun 6, 2014

Thanks for your help. That got me into admin. Now I have another problem :)

When I try to add a Backend, I am using the Getting started, to test, so setting up the Google Maps example.

When I enter in the information, and click save, I am getting a popup: 502 Bad Gateway - From Nginx.

The tail of my development.log in Web is:
Started GET "/admin/api_doc_collections" for 184.0.166.133 at 2014-06-06 10:18:09 +0000
Processing by Admin::ApiDocCollectionsController#index as HTML
Rendered admin/api_doc_collections/_tree_node.html.erb (0.0ms)
Rendered admin/api_doc_collections/index.html.erb within layouts/admin (2.5ms)
Completed 200 OK in 49.4ms (Views: 43.8ms)
Started GET "/admin/" for 184.0.166.133 at 2014-06-06 10:18:16 +0000
Processing by Admin::BaseController#empty as HTML
Rendered text template within layouts/admin (0.0ms)
Completed 200 OK in 7.7ms (Views: 5.8ms)
Started GET "/admin/apis.json?sEcho=1&iColumns=5&sColumns=&iDisplayStart=0&iDisplayLength=50&mDataProp_0=name&mDataProp_1=frontend_host&mDataProp_2=frontend_prefixes&mDataProp_3=sort_order&mDataProp_4=&sSearch=&bRegex=false&sSearch_0=&b$
Processing by Admin::ApisController#index as JSON
Parameters: {"sEcho"=>"1", "iColumns"=>"5", "sColumns"=>"", "iDisplayStart"=>"0", "iDisplayLength"=>"50", "mDataProp_0"=>"name", "mDataProp_1"=>"frontend_host", "mDataProp_2"=>"frontend_prefixes", "mDataProp_3"=>"sort_order", "mDataPr$
Rendered admin/apis/index.rabl (6.7ms)
Completed 200 OK in 13.3ms (Views: 11.3ms)
Started GET "/admin/apis.json?sEcho=1&iColumns=5&sColumns=&iDisplayStart=0&iDisplayLength=50&mDataProp_0=name&mDataProp_1=frontend_host&mDataProp_2=frontend_prefixes&mDataProp_3=sort_order&mDataProp_4=&sSearch=&bRegex=false&sSearch_0=&b$
Processing by Admin::ApisController#index as JSON
Parameters: {"sEcho"=>"1", "iColumns"=>"5", "sColumns"=>"", "iDisplayStart"=>"0", "iDisplayLength"=>"50", "mDataProp_0"=>"name", "mDataProp_1"=>"frontend_host", "mDataProp_2"=>"frontend_prefixes", "mDataProp_3"=>"sort_order", "mDataPr$
Rendered admin/apis/index.rabl (2.6ms)
Completed 200 OK in 4.9ms (Views: 3.3ms)
Started GET "/admin/apis.json?sEcho=1&iColumns=5&sColumns=&iDisplayStart=0&iDisplayLength=50&mDataProp_0=name&mDataProp_1=frontend_host&mDataProp_2=frontend_prefixes&mDataProp_3=sort_order&mDataProp_4=&sSearch=&bRegex=false&sSearch_0=&b$
Processing by Admin::ApisController#index as JSON
Parameters: {"sEcho"=>"1", "iColumns"=>"5", "sColumns"=>"", "iDisplayStart"=>"0", "iDisplayLength"=>"50", "mDataProp_0"=>"name", "mDataProp_1"=>"frontend_host", "mDataProp_2"=>"frontend_prefixes", "mDataProp_3"=>"sort_order", "mDataPr$
Rendered admin/apis/index.rabl (2.2ms)
Completed 200 OK in 4.2ms (Views: 2.8ms)
Started GET "/admin/" for 184.0.166.133 at 2014-06-06 10:21:59 +0000
Processing by Admin::BaseController#empty as HTML
Rendered text template within layouts/admin (0.0ms)
Completed 200 OK in 6.2ms (Views: 4.9ms)
Started GET "/admin/apis.json?sEcho=1&iColumns=5&sColumns=&iDisplayStart=0&iDisplayLength=50&mDataProp_0=name&mDataProp_1=frontend_host&mDataProp_2=frontend_prefixes&mDataProp_3=sort_order&mDataProp_4=&sSearch=&bRegex=false&sSearch_0=&b$
Processing by Admin::ApisController#index as JSON
Parameters: {"sEcho"=>"1", "iColumns"=>"5", "sColumns"=>"", "iDisplayStart"=>"0", "iDisplayLength"=>"50", "mDataProp_0"=>"name", "mDataProp_1"=>"frontend_host", "mDataProp_2"=>"frontend_prefixes", "mDataProp_3"=>"sort_order", "mDataPr$
Rendered admin/apis/index.rabl (2.3ms)
Completed 200 OK in 6.5ms (Views: 2.9ms)

The tail of my error.log for nginx is:
2014/06/06 09:52:38 [error] 4242#0: *142 connect() failed (111: Connection refused) while connecting to upstream, client: 184.0.166.133, server: developer.refindly.com, request: "GET /api-umbrella/v1/users/06151c96-4bd1-48cd-948d-7745ec$
2014/06/06 10:04:17 [error] 1202#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 184.0.166.133, server: developer.refindly.com, request: "GET /api-umbrella/v1/admins/5ab5a830-0f91-4b04-b219-fafb8e5$
2014/06/06 10:06:50 [error] 1202#0: *18 connect() failed (111: Connection refused) while connecting to upstream, client: 184.0.166.133, server: developer.refindly.com, request: "GET /api-umbrella/v1/apis/73d73937-49b9-4aa9-a763-23fdd1f9$
2014/06/06 10:10:43 [error] 1202#0: *24 connect() failed (111: Connection refused) while connecting to upstream, client: 184.0.166.133, server: developer.refindly.com, request: "POST /api-umbrella/v1/apis.json HTTP/1.1", upstream: "http$
2014/06/06 10:10:55 [error] 1202#0: *24 connect() failed (111: Connection refused) while connecting to upstream, client: 184.0.166.133, server: developer.refindly.com, request: "POST /api-umbrella/v1/apis.json HTTP/1.1", upstream: "http$
2014/06/06 10:12:34 [error] 2124#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 184.0.166.133, server: developer.refindly.com, request: "POST /api-umbrella/v1/apis.json HTTP/1.1", upstream: "http:$
2014/06/06 10:13:53 [error] 2124#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 184.0.166.133, server: developer.refindly.com, request: "POST /api-umbrella/v1/apis.json HTTP/1.1", upstream: "http:$
2014/06/06 10:18:27 [error] 2389#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 184.0.166.133, server: developer.refindly.com, request: "GET /api-umbrella/v1/apis/73d73937-49b9-4aa9-a763-23fdd1f94$
2014/06/06 10:19:57 [error] 2389#0: *9 connect() failed (111: Connection refused) while connecting to upstream, client: 184.0.166.133, server: developer.refindly.com, request: "POST /api-umbrella/v1/apis.json HTTP/1.1", upstream: "http:$
$ "http://127.0.0.1:50090/api-umbrella/v1/apis.json", host: "developer.refindly.com", referrer: "http://developer.refindly.com/admin/"

Please let me know if you have any ideas. Getting close! Thanks.

@refindly
Copy link
Author

refindly commented Jun 6, 2014

Also, here is my log for Router/log/gatekeeper/production.log

This is after a restart. Pretty sure mongo is fine because I can login with out a problem now to the Admin panel. So seems like something is going on with the connection to the backend servers:

[Fri Jun 06 2014 11:41:16 GMT+0000 (UTC)] ERROR Mongo disconnected {}
[Fri Jun 06 2014 11:41:16 GMT+0000 (UTC)] ERROR Mongo close {}
[Fri Jun 06 2014 11:41:16 GMT+0000 (UTC)] ERROR Mongo disconnected {}
[Fri Jun 06 2014 11:41:16 GMT+0000 (UTC)] ERROR Mongo disconnected {}
[Fri Jun 06 2014 11:41:16 GMT+0000 (UTC)] ERROR Mongo close {}
[Fri Jun 06 2014 11:41:16 GMT+0000 (UTC)] ERROR Mongo disconnected {}
[Fri Jun 06 2014 11:41:16 GMT+0000 (UTC)] ERROR Mongo disconnected {}
[Fri Jun 06 2014 11:41:16 GMT+0000 (UTC)] ERROR Mongo close {}
[Fri Jun 06 2014 11:41:16 GMT+0000 (UTC)] ERROR Mongo disconnected {}
[Fri Jun 06 2014 11:41:16 GMT+0000 (UTC)] ERROR Mongo disconnected {}
[Fri Jun 06 2014 11:41:16 GMT+0000 (UTC)] ERROR Mongo close {}
[Fri Jun 06 2014 11:41:16 GMT+0000 (UTC)] ERROR Mongo close {}
[Fri Jun 06 2014 11:41:16 GMT+0000 (UTC)] ERROR Mongo close {}
[Fri Jun 06 2014 11:41:16 GMT+0000 (UTC)] ERROR Mongo close {}
[Fri Jun 06 2014 11:41:17 GMT+0000 (UTC)] ERROR Mongo error { '0': [Error: failed to connect to [127.0.0.1:27017]] }
[Fri Jun 06 2014 11:41:17 GMT+0000 (UTC)] ERROR Mongo error { '0': [Error: failed to connect to [127.0.0.1:27017]] }
[Fri Jun 06 2014 11:41:17 GMT+0000 (UTC)] ERROR Mongo error { '0': [Error: failed to connect to [127.0.0.1:27017]] }
[Fri Jun 06 2014 11:41:17 GMT+0000 (UTC)] ERROR Mongo error { '0': [Error: failed to connect to [127.0.0.1:27017]] }
[Fri Jun 06 2014 11:41:17 GMT+0000 (UTC)] ERROR Mongo error { '0': [Error: failed to connect to [127.0.0.1:27017]] }
[Fri Jun 06 2014 11:41:17 GMT+0000 (UTC)] ERROR Mongo error { '0': [Error: failed to connect to [127.0.0.1:27017]] }
[Fri Jun 06 2014 11:41:17 GMT+0000 (UTC)] ERROR Mongo error { '0': [Error: failed to connect to [127.0.0.1:27017]] }
[Fri Jun 06 2014 11:42:06 GMT+0000 (UTC)] INFO Reading new config (PID 1520)...
[Fri Jun 06 2014 11:42:06 GMT+0000 (UTC)] INFO Reading new config (PID 1465)...
[Fri Jun 06 2014 11:42:06 GMT+0000 (UTC)] INFO Reading new config (PID 1462)...
[Fri Jun 06 2014 11:42:06 GMT+0000 (UTC)] INFO Reading new config (PID 1460)...
[Fri Jun 06 2014 11:42:06 GMT+0000 (UTC)] INFO Reading new config (PID 1463)...
[Fri Jun 06 2014 11:42:06 GMT+0000 (UTC)] INFO Reading new config (PID 1461)...
[Fri Jun 06 2014 11:42:06 GMT+0000 (UTC)] INFO Reading new config (PID 1466)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO System DNS discovered: [ { address: '172.31.0.2', port: 53 } ]
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Writing new config...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Resolving all hosts..
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Handle resolving all hosts..
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Writing nginx config...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Nginx config already up-to-date - skipping...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Writing gatekeeper config...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO No gatekeeper config changes found - reseting...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Writing new config (PID 1466)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Config file change detected (PID 1461)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Config file change detected (PID 1520)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Config file change detected (PID 1465)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Config file change detected (PID 1463)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Config file change detected (PID 1460)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Config file change detected (PID 1462)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Reading new config (PID 1520)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Reading new config (PID 1465)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Reading new config (PID 1462)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Reading new config (PID 1463)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Reading new config (PID 1460)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Reading new config (PID 1461)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Config file change detected (PID 1466)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Reading new config (PID 1466)...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Gatekeeper config written...
[Fri Jun 06 2014 11:42:11 GMT+0000 (UTC)] INFO Config files written...
[Fri Jun 06 2014 11:42:06 GMT+0000 (UTC)] INFO Reading new config (PID 1521)...
[Fri Jun 06 2014 11:42:12 GMT+0000 (UTC)] INFO Config file change detected (PID 1521)...
[Fri Jun 06 2014 11:42:12 GMT+0000 (UTC)] INFO Reading new config (PID 1521)...
[Fri Jun 06 2014 11:42:12 GMT+0000 (UTC)] ERROR Template error: { [Error: No Living connections] message: 'No Living connections' }
[Fri Jun 06 2014 11:42:12 GMT+0000 (UTC)] ERROR Template error: { [Error: No Living connections] message: 'No Living connections' }

@refindly
Copy link
Author

refindly commented Jun 6, 2014

Hello, sorry for the many comments. Do you know of anyone that would offer paid support to fix the few things we're having trouble with? The environment is close, there must just be a small thing causing our final issues. If there is that would be great as we really need to get this up for our project. Thanks again for your help.

@GUI
Copy link
Member

GUI commented Jun 6, 2014

No worries, and sorry for the troubles! There were some semi-recent changes that I think have changed this setup process, and unfortunately the setup documentation hasn't caught up. As I mentioned, we're working to really improve this setup and installation process, but apologies for the current state of flux.

I'm in the process of standing up a new environment of my own to test a fresh setup out again, but my one other thought that might get things working for you: After logging into the admin, first navigate to Configuration > Publish Changes. Is there a big Publish button available on this page? If so, try hitting that. Then after things reload, try going back to the Configuration > Backends area and completing the example from the Getting Started docs.

Unfortunately, there's no paid support at this time. I'd be happy to try and sort out any issues you may encounter with this (but obviously that's not quite the same as paid support).

@refindly
Copy link
Author

refindly commented Jun 6, 2014

Thanks for sticking with me to try to figure this out, very much appreciated.

I went to the publish changes, and hit the green button, however I got the same result when trying to add the sample backend.

Ive been looking through logs, and wanted to post a couple things that may give you an idea of the issue, and show you some of the configs I have:

  1. In /router/current/gatekeeper/config/default.yml
    -- The mongo block shows as below. Should it have the /api_umbrella_development ?

apiUmbrella:
mongodb: "mongodb://127.0.0.1:27017/api_umbrella_development"
mongodb_options:
server:
socketOptions:
keepAlive: 500
replset:
socketOptions:
keepAlive: 500

  1. In my /router/current/log/supervisor/supervisor.log Im seeing this:

fs.js:427
return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);

fs.js:427
^
Error: EACCES, permission denied '/srv/sites/router/shared/config/gatekeeper/runtime.yml'
at Object.fs.openSync (fs.js:427:18)
at _.extend.initialize (/vagrant/workspace/router/gatekeeper/lib/config.js:42:17)
at new Config (/vagrant/workspace/router/gatekeeper/lib/config.js:17:19)
at Object. (/vagrant/workspace/router/gatekeeper/lib/config.js:143:16)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:364:17)
at require (module.js:380:17)
return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
^
Error: EACCES, permission denied '/srv/sites/router/shared/config/gatekeeper/runtime.yml'
at Object.fs.openSync (fs.js:427:18)
at _.extend.initialize (/vagrant/workspace/router/gatekeeper/lib/config.js:42:17)
at new Config (/vagrant/workspace/router/gatekeeper/lib/config.js:17:19)
at Object. (/vagrant/workspace/router/gatekeeper/lib/config.js:143:16)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:364:17)
at require (module.js:380:17)

and

Elasticsearch WARNING: 2014-06-06T11:42:12Z
Unable to revive connection: http://127.0.0.1:9200/

Elasticsearch WARNING: 2014-06-06T11:42:12Z
No living connections

Elasticsearch WARNING: 2014-06-06T11:42:12Z
Unable to revive connection: http://127.0.0.1:9200/

Elasticsearch WARNING: 2014-06-06T11:42:12Z
No living connections

Elasticsearch WARNING: 2014-06-06T11:42:12Z
Unable to revive connection: http://127.0.0.1:9200/

Elasticsearch WARNING: 2014-06-06T11:42:12Z
No living connections

Elasticsearch WARNING: 2014-06-06T11:42:12Z
Unable to revive connection: http://127.0.0.1:9200/

Elasticsearch WARNING: 2014-06-06T11:42:12Z
No living connections

Elasticsearch WARNING: 2014-06-06T11:42:12Z
Unable to revive connection: http://127.0.0.1:9200/

Elasticsearch WARNING: 2014-06-06T11:42:12Z
No living connections

Elasticsearch WARNING: 2014-06-06T11:42:12Z
Unable to revive connection: http://127.0.0.1:9200/

Elasticsearch WARNING: 2014-06-06T11:42:12Z
No living connections

  1. For my main nginx.conf, when it asks for server_name, unsure if I should use the domain we have mapped or not, so this is what I have there:

The main, front-facing public host server. This serves up both the APIs and

the web content.

server {
listen 9080 ssl;
listen 9443;
server_name developer.refindly.com localhost;

include ./public_defaults.conf;
include ./ssl.conf;

# Route the docs and other static pages to the Rails web app. This may
# change, so that's why we're treating it separately.
location ~* ^/(about|community|doc|docs|images|static|)(/|$) {
  proxy_pass http://api_umbrella_web_backend;
}

# Route the dynamic portions of the site to the Rails web app.
location ~* ^/(account|admin|admins|assets|contact|favicon\.ico|javascripts|robots.txt|signup)(/|$) {
  proxy_pass http://api_umbrella_web_backend;
}

# Route all other requests as APIs going through the API Umbrella Gatekeeper.
location / {
  # Remove the legacy /api/ prefix from all API urls. Now we assume API URLs
  # are all top-level.
  rewrite ^/api(/.*) $1 break;

  proxy_pass http://api_umbrella_gatekeeper_frontend;
}

}

Secondary hosts. These are other hosts we're handling only API traffic for,

but don't want to show web page content for.

server {
listen 9080 default_server;
listen 9443 ssl default_server;
server_name _;

include ./public_defaults.conf;

I apologize for the long response, just trying to provide as much info as I can to try to sort this out. Thanks again for looking at this for me.

@GUI
Copy link
Member

GUI commented Jun 7, 2014

I ran through things and was able to get things working after the addition of the rake db:seed and publishing the backend configuration changes. So those are some missing steps I'll get integrated into the setup instructions, but that was in the development environment, and it looks like you're encountering some other issues related to this being in a new production environment.

Thanks for providing your logs. One issue I see in the supervisor logs is that I don't think the gatekeeper is starting due to some file permission issues. With the default setup, your /srv/sites/router/shared/config/gatekeeper and /srv/sites/router/shared/log/gatekeeper directories must be writable by the api-umbrella-gatekeeper user. The default deployment scripts attempt to do this using filesystem ACLs, but if your disk isn't mounted with ACL permissions, this is silently failing. Getting away from this ACL requirement is also part of the setup improvements we're in the process of making, but in the meantime, if you can enable filesystem ACLs for this /srv area, then your next deployment should setup the proper permissions. Alternatively, if you want to avoid filesystem ACLs, just make make sure those two directories are owned by the api-umbrella-gatekeeper user.

In general, there's a few config things you'll need to setup for a new production environment. It seems like you've maybe figured some of this out, but I think you might be missing a few of these, so here's my attempt at actually putting together a list of the configuration things you'll need to tweak (and my apologies this wasn't already better documented):

  • In the web project:
    • Update the config/mongoid.yml.deploy file. In the production area update the section under sessions.default.hosts to point to the host for your Mongo database (multiple hosts can be listed if it's part of a replicaset).

    • Update the config/elasticsearch.yml.deploy file. Add production block similar to the development block, but pointing to the an elasticsearch host (if you have multiple hosts as part of an elasticsearch cluster, only list one in this case).

    • Add a config/deploy/production.rb file:

      # Set the servers for this stage.
      role :app, "10.0.0.1", "10.0.0.2" # Substitute your real web server hosts/IPs here
      role :web, "10.0.0.1", "10.0.0.2" # Substitute your real web server hosts/IPs here
      
      # Set the base path for deployment.
      set :deploy_to_base, "/srv"
      
      # Set the accessible web domain for this site.
      set :base_domain, "developer.refindly.com"
      
      # Set the Rails environment.
      set :rails_env, "production"
  • In the router project:
    • Add a router/current/gatekeeper/production.yml looking something like (but substituting for your real mongo or elasticsearch hosts):

      apiUmbrella:
        mongodb: "mongodb://10.0.0.1:27017,10.0.0.2,10.0.0.3:27017/api_umbrella_production"
        elasticsearch:
          hosts:
            - host: "10.0.0.1"
              port: 9200
            - host: "10.0.0.2"
              port: 9200
            - host: "10.0.0.3"
              port: 9200
        proxy:
          workers: 4
    • Add a config/deploy/production.rb file:

      # Set the servers for this stage.
      role :app, "10.0.0.1", "10.0.0.2" # Substitute your real web server hosts/IPs here
      role :web, "10.0.0.1", "10.0.0.2" # Substitute your real web server hosts/IPs here
      
      # Set the base path for deployment.
      set :deploy_to_base, "/srv"
      
      # Set the accessible web domain for this site.
      set :base_domain, "developer.refindly.com"
      
      # Set the Rails environment.
      set :rails_env, "production"

And to your third point about the domain, if you're using the default capistrano scripts for deployment, then the nginx hostname should get automatically setup based on your config/deploy/production.rb files (the base_domain variable). And you would set those to your public-facing domain name, like developer.refindly.com.

I think you're close. Hopefully this helps, and if so, we can use these as better documentation for getting a production environment setup until we get the broader setup improvements made (basically, we want to get this down to a single RPM or DEB package installer and one config file you tweak--but I don't have a firm timeframe on when that will be ready).

Let me know if you have any further questions or you're still running into any errors. Sorry again for the trouble!

@refindly
Copy link
Author

refindly commented Jun 9, 2014

Thanks again for all the help.

Went through all your input, and got further.

Before your comments, if I accessed any of the analytics in the admin I got an error, now the analtics are loading and showing api calls. Each calls status is a 404, which may help in debugging.

I still cannot add a backend. When I try now, the modal does not show me the nginx error, it just shows Error, with no actual error output.

  1. Here is a screen of the analytics logs, showing the calls and 404's: http://www.screencast-o-matic.com/screenshots/u/VW8k/1402302892377-73065.png

  2. I did notice the /api-umbrella/api - Not sure if the /api-umbrella/ should be there? I believe all those are me trying to save a backend in the admin panel.

  3. Not knowing what logs to show you, I will show you the ones I see errors in. Most have stopped showing error after the previous changes you gave me.

This is /srv/sites/router/current/log/error.log

2014/06/08 10:32:17 [error] 1129#0: *285 connect() failed (111: Connection refused) while connecting to upstream, client: 23.22.216.162, server: _, request: "HEAD / HTTP/1.1", upstream: "http://127.0.0.1:50090/", host: "54.209.87.206"
2014/06/08 10:32:17 [error] 1129#0: *287 connect() failed (111: Connection refused) while connecting to upstream, client: 23.22.216.162, server: _, request: "GET /clientaccesspolicy.xml HTTP/1.1", upstream: "http://127.0.0.1:50090/clientaccesspolicy.xml", host: "54.209.87.206"
2014/06/08 14:46:10 [error] 1129#0: *291 connect() failed (111: Connection refused) while connecting to upstream, client: 198.133.224.185, server: _, request: "GET /robots.txt HTTP/1.1", upstream: "http://127.0.0.1:50090/robots.txt", host: "54.209.87.206", referrer: "http://www.google.com"
2014/06/08 14:46:10 [error] 1129#0: *293 connect() failed (111: Connection refused) while connecting to upstream, client: 198.133.224.185, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:50090/", host: "54.209.87.206", referrer: "http://www.google.com"
2014/06/09 00:06:05 [error] 1129#0: *295 connect() failed (111: Connection refused) while connecting to upstream, client: 66.85.140.110, server: _, request: "GET / HTTP/1.0", upstream: "http://127.0.0.1:50090/"
2014/06/09 03:07:32 [error] 1129#0: *297 connect() failed (111: Connection refused) while connecting to upstream, client: 66.85.144.237, server: _, request: "GET / HTTP/1.0", upstream: "http://127.0.0.1:50090/"
2014/06/09 04:37:38 [error] 3811#0: *299 connect() failed (111: Connection refused) while connecting to upstream, client: 54.198.149.180, server: _, request: "HEAD / HTTP/1.1", upstream: "http://127.0.0.1:50090/", host: "54.209.87.206"
2014/06/09 04:37:38 [error] 3811#0: *301 connect() failed (111: Connection refused) while connecting to upstream, client: 54.198.149.180, server: _, request: "GET /clientaccesspolicy.xml HTTP/1.1", upstream: "http://127.0.0.1:50090/clientaccesspolicy.xml", host: "54.209.87.206"
2014/06/09 05:43:44 [error] 1212#0: *132 connect() failed (111: Connection refused) while connecting to upstream, client: 184.0.166.133, server: developer.refindly.com, request: "POST /api-umbrella/v1/apis.json HTTP/1.0", upstream: "http://127.0.0.1:50090/api-umbrella/v1/apis.json", host: "developer.refindly.com", referrer: "https://developer.refindly.com/admin/"

Actually, those are the only errors I can find at this point in the logs. All the others are looking good, no error and showing that the other services are running properly. Mongo is showing the connection, no errors, elasticsearch looks good now. So these are all I'm seeing at this point.

Your changes seems to got me closer, just need to figure out what the errors are above. Potentially permissions? Not sure.

Thanks again for all your help.

@GUI
Copy link
Member

GUI commented Jun 9, 2014

Ah, I think I missed one more piece that might explain this:

  • In the web project:
    • Add a config/settings/production.yml like so:

      default_host: developer.refindly.com
    • Re-seed the database and re-publish the backend configuration changes:

      $ RAILS_ENV=production rake db:seed
      $ RAILS_ENV=production bundle exec rails runner 'ConfigVersion.publish!'

@GUI
Copy link
Member

GUI commented Oct 27, 2014

We've rolled out a greatly simplified setup process that should hopefully alleviate these type of setup issues: http://nrel.github.io/api-umbrella/download/ (see #26 for more background). So I'm going to close this issue, but let me know if you're still interested in API Umbrella and run into any issues with the new setup.

@GUI GUI closed this as completed Oct 27, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants