New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ability to automatically use ENV variables the you input into Opsworks #28

Closed
pjammer opened this Issue Aug 4, 2016 · 7 comments

Comments

Projects
None yet
3 participants
@pjammer

pjammer commented Aug 4, 2016

would be great to create an application.yml file with any environment variables you create in the Opsworks console. Then it'd be there if you use Figaro, or if you use dot.env maybe we put that format in there too. Right now, i'm struggling to figure out how to get those ENVs into this recipe so i can create a Factory for you. i tried even just cowboy coding it but ran up against time and talent.

@ajgon ajgon added the enhancement label Aug 5, 2016

@ajgon ajgon added this to the 0.6.0 milestone Aug 5, 2016

@ajgon ajgon self-assigned this Aug 5, 2016

@pjammer

This comment has been minimized.

Show comment
Hide comment
@pjammer

pjammer Aug 8, 2016

if you look on my fork, which is way too messy to bring over right now, I had to add the code here, and add the symlink to attributes/default.rb reason why is reference in i #25 but it works!

pjammer commented Aug 8, 2016

if you look on my fork, which is way too messy to bring over right now, I had to add the code here, and add the symlink to attributes/default.rb reason why is reference in i #25 but it works!

@ajgon

This comment has been minimized.

Show comment
Hide comment
@ajgon

ajgon Aug 8, 2016

Owner

I'm afraid, I don't get it. The environmental variables are passed to unicorn (currently the only supported app-server), and they should be available out-of-the box in the ENV object.

Why add another layer of abstraction?

Owner

ajgon commented Aug 8, 2016

I'm afraid, I don't get it. The environmental variables are passed to unicorn (currently the only supported app-server), and they should be available out-of-the box in the ENV object.

Why add another layer of abstraction?

@ajgon ajgon removed this from the 0.6.0 milestone Aug 8, 2016

@ajgon ajgon added question and removed enhancement labels Aug 8, 2016

@pjammer

This comment has been minimized.

Show comment
Hide comment
@pjammer

pjammer Aug 9, 2016

When you have a bunch of ENV variables, like AWS_SECRET, or STRIPE... those are cool to keep in Opsworks in their console; but to get them into the app during deploy, you have to use (if you don't role your own) a gem like Figaro which takes an application.yml file and allows you to use them in the app.

As an aside: I did notice whilst doing this that all ENV variables that you enter into the Opsworks console are also passed, along with RAILS_ENV=production and it mucks up a stage for migration when you try to symlink_before_migrate in a custom json stanza in the console.
Removing that custom json and not using a symlink_before_migrate worked.

So I'm all ears if I did this thing wrong, and you have a better way man. I was in a pinch and this is what i had to do to get past it.

pjammer commented Aug 9, 2016

When you have a bunch of ENV variables, like AWS_SECRET, or STRIPE... those are cool to keep in Opsworks in their console; but to get them into the app during deploy, you have to use (if you don't role your own) a gem like Figaro which takes an application.yml file and allows you to use them in the app.

As an aside: I did notice whilst doing this that all ENV variables that you enter into the Opsworks console are also passed, along with RAILS_ENV=production and it mucks up a stage for migration when you try to symlink_before_migrate in a custom json stanza in the console.
Removing that custom json and not using a symlink_before_migrate worked.

So I'm all ears if I did this thing wrong, and you have a better way man. I was in a pinch and this is what i had to do to get past it.

@ajgon

This comment has been minimized.

Show comment
Hide comment
@ajgon

ajgon Aug 16, 2016

Owner

Can you explain me in layman's terms, with examples, what exactly do you try to achieve? Sorry, I still don't get it...

Owner

ajgon commented Aug 16, 2016

Can you explain me in layman's terms, with examples, what exactly do you try to achieve? Sorry, I still don't get it...

@phongsi

This comment has been minimized.

Show comment
Hide comment
@phongsi

phongsi Aug 23, 2016

Contributor

hey @ajgon and @pjammer,

I get what pjammer wants. Basically if you're booting up a process that's not unicorn, you'll want all the environment variables.

Right now you only get that directly booting unicorn. https://github.com/ajgon/opsworks_ruby/blob/master/templates/default/unicorn.conf.erb#L23
This will get run on deploy.

I think the advantages to having a yml are obvious. we dont need to insert the database configs into unicorn every time because its already available when the rails environment loads. If we want to spin up a non unicorn process like sidekiq it knows about the database too. Someone might want to ssh into a machine and boot up the rails console, they wouldn't need to enter in every variable into the command line.

@pjammer what you want is available already in open source projects. for example i've used this before https://github.com/joeyAghion/opsworks_custom_env
it does exactly what you are describing. I haven't tested this for chef12 though.

making the case to have it included in this project, i don't know. I think its up to @ajgon
but doing so would remove the need to inject environment vars into https://github.com/ajgon/opsworks_ruby/blob/master/templates/default/sidekiq.monitrc.erb#L10
and
https://github.com/ajgon/opsworks_ruby/blob/master/templates/default/unicorn.conf.erb#L23

Contributor

phongsi commented Aug 23, 2016

hey @ajgon and @pjammer,

I get what pjammer wants. Basically if you're booting up a process that's not unicorn, you'll want all the environment variables.

Right now you only get that directly booting unicorn. https://github.com/ajgon/opsworks_ruby/blob/master/templates/default/unicorn.conf.erb#L23
This will get run on deploy.

I think the advantages to having a yml are obvious. we dont need to insert the database configs into unicorn every time because its already available when the rails environment loads. If we want to spin up a non unicorn process like sidekiq it knows about the database too. Someone might want to ssh into a machine and boot up the rails console, they wouldn't need to enter in every variable into the command line.

@pjammer what you want is available already in open source projects. for example i've used this before https://github.com/joeyAghion/opsworks_custom_env
it does exactly what you are describing. I haven't tested this for chef12 though.

making the case to have it included in this project, i don't know. I think its up to @ajgon
but doing so would remove the need to inject environment vars into https://github.com/ajgon/opsworks_ruby/blob/master/templates/default/sidekiq.monitrc.erb#L10
and
https://github.com/ajgon/opsworks_ruby/blob/master/templates/default/unicorn.conf.erb#L23

@ajgon ajgon closed this in c989494 Aug 29, 2016

@ajgon

This comment has been minimized.

Show comment
Hide comment
@ajgon

ajgon Aug 29, 2016

Owner

Ah, now I see it. I like the idea - added support for both figaro and dotenv - they are configurable via app['appserver']['application_yml'] and app['appserver']['dot_env'] stack options (both are turned off by default).

However I disagree, with a concept to remove those ENV for appserver/worker raw configuration files. The reason behind this is, that some people might not want/know/need to use them, and I want to make those scripts as less dependant on other gems as possible.

Owner

ajgon commented Aug 29, 2016

Ah, now I see it. I like the idea - added support for both figaro and dotenv - they are configurable via app['appserver']['application_yml'] and app['appserver']['dot_env'] stack options (both are turned off by default).

However I disagree, with a concept to remove those ENV for appserver/worker raw configuration files. The reason behind this is, that some people might not want/know/need to use them, and I want to make those scripts as less dependant on other gems as possible.

@pjammer

This comment has been minimized.

Show comment
Hide comment
@pjammer

pjammer Oct 14, 2016

I can hardly wait to update my opsworks stack to this latest cookbook. I hate dependencies too, but for some reason what figaro specially gives me I couldn't just make happen. i.e., the magic.

Did you happen to fix the fact that unicorn gets ALL the ENV variables and i am getting this

> grep -c "WARNING" log/unicorn.stderr.log 
493

example line is:

I, [2016-10-14T15:34:17.280463 #7590]  INFO -- : reaped #<Process::Status: pid 7599 exit 0> worker=2
I, [2016-10-14T15:34:17.280587 #7590]  INFO -- : reaped #<Process::Status: pid 7602 exit 0> worker=3
I, [2016-10-14T15:34:17.487023 #7590]  INFO -- : reaped #<Process::Status: pid 7593 exit 0> worker=0
I, [2016-10-14T15:34:17.487132 #7590]  INFO -- : reaped #<Process::Status: pid 7596 exit 0> worker=1
I, [2016-10-14T15:34:17.487218 #7590]  INFO -- : master complete
I, [2016-10-14T15:34:18.344518 #6993]  INFO -- : Refreshing Gem list
WARNING: Skipping key "DEVISE_SECRET_KEY". Already set in ENV..
I, [2016-10-14T15:34:24.194831 #6993]  INFO -- : unlinking existing socket=/srv/www/wickedapp/shared/sockets/unicorn.sock
I, [2016-10-14T15:34:24.195029 #6993]  INFO -- : listening on addr=/srv/www/wickedapp/shared/sockets/unicorn.sock fd=11
I, [2016-10-14T15:34:24.409746 #6998]  INFO -- : worker=0 ready
I, [2016-10-14T15:34:24.549711 #7001]  INFO -- : worker=1 ready
I, [2016-10-14T15:34:24.609199 #7004]  INFO -- : worker=2 ready
I, [2016-10-14T15:34:24.627257 #6993]  INFO -- : master process ready
I, [2016-10-14T15:34:24.707866 #7007]  INFO -- : worker=3 ready

Basically 11 lines each time the reaper comes. or during a deploy... unclear as to why yet, but instead of just passing the UNICORN based envs, the whole kit'n caboodle comes over.

pjammer commented Oct 14, 2016

I can hardly wait to update my opsworks stack to this latest cookbook. I hate dependencies too, but for some reason what figaro specially gives me I couldn't just make happen. i.e., the magic.

Did you happen to fix the fact that unicorn gets ALL the ENV variables and i am getting this

> grep -c "WARNING" log/unicorn.stderr.log 
493

example line is:

I, [2016-10-14T15:34:17.280463 #7590]  INFO -- : reaped #<Process::Status: pid 7599 exit 0> worker=2
I, [2016-10-14T15:34:17.280587 #7590]  INFO -- : reaped #<Process::Status: pid 7602 exit 0> worker=3
I, [2016-10-14T15:34:17.487023 #7590]  INFO -- : reaped #<Process::Status: pid 7593 exit 0> worker=0
I, [2016-10-14T15:34:17.487132 #7590]  INFO -- : reaped #<Process::Status: pid 7596 exit 0> worker=1
I, [2016-10-14T15:34:17.487218 #7590]  INFO -- : master complete
I, [2016-10-14T15:34:18.344518 #6993]  INFO -- : Refreshing Gem list
WARNING: Skipping key "DEVISE_SECRET_KEY". Already set in ENV..
I, [2016-10-14T15:34:24.194831 #6993]  INFO -- : unlinking existing socket=/srv/www/wickedapp/shared/sockets/unicorn.sock
I, [2016-10-14T15:34:24.195029 #6993]  INFO -- : listening on addr=/srv/www/wickedapp/shared/sockets/unicorn.sock fd=11
I, [2016-10-14T15:34:24.409746 #6998]  INFO -- : worker=0 ready
I, [2016-10-14T15:34:24.549711 #7001]  INFO -- : worker=1 ready
I, [2016-10-14T15:34:24.609199 #7004]  INFO -- : worker=2 ready
I, [2016-10-14T15:34:24.627257 #6993]  INFO -- : master process ready
I, [2016-10-14T15:34:24.707866 #7007]  INFO -- : worker=3 ready

Basically 11 lines each time the reaper comes. or during a deploy... unclear as to why yet, but instead of just passing the UNICORN based envs, the whole kit'n caboodle comes over.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment