New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use deployment tools to install a test server on localhost? #403

Open
jimallman opened this Issue Aug 6, 2014 · 7 comments

Comments

Projects
None yet
3 participants
@jimallman
Member

jimallman commented Aug 6, 2014

This would be a nice way to handle the quirks and complexities of our system, but so far it's pretty tricky on systems other than the expected Debian GNU/Linux.

In our scripts, we've made lots of assumptions about things like home directories. So installing on another system like OS X will necessarily involve some kind of privileged "preflight" script to enable services, create users and directories, etc. There are probably other obstacles ahead.

Perhaps the real answer is to use a local VM (VirtualBox, Parallels, VMWare, etc) with a minimal Debian. Or a prepared VM (opentree and admin users already created, keys in place) that's ready to accept push commands.

jimallman added a commit to OpenTreeOfLife/deployed-systems that referenced this issue Aug 6, 2014

@jimallman

This comment has been minimized.

Show comment
Hide comment
@jimallman

jimallman Aug 6, 2014

Member

To explore this further, I've added a branch to deployed-systems that adds a new server-config file localhost.config that tries to deploy the devtree server to localhost.

Member

jimallman commented Aug 6, 2014

To explore this further, I've added a branch to deployed-systems that adds a new server-config file localhost.config that tries to deploy the devtree server to localhost.

@jar398

This comment has been minimized.

Show comment
Hide comment
@jar398

jar398 Aug 6, 2014

Member

Or maybe docker.

-- apologies for brevity / using handheld gizmo --

On Aug 5, 2014, at 22:18, Jim Allman notifications@github.com wrote:

This would be a nice way to handle the quirks and complexities of our
system, but so far it's pretty tricky on systems other than the
expected Debian
GNU/Linux
.

In our scripts, we've made lots of assumptions about things like home
directories
http://en.wikipedia.org/wiki/Home_directory#Default_home_directory_per_operating_system.
So installing on another system like OS X will necessarily involve some
kind of privileged "preflight" script to enable services, create users and
directories, etc. There are probably other obstacles ahead.

Perhaps the real answer is to use a local VM (VirtualBox, Parallels,
VMWare, etc) with a minimal Debian. Or a prepared VM (opentree and admin
users already created, keys in place) that's ready to accept push commands.


Reply to this email directly or view it on GitHub
#403.

Member

jar398 commented Aug 6, 2014

Or maybe docker.

-- apologies for brevity / using handheld gizmo --

On Aug 5, 2014, at 22:18, Jim Allman notifications@github.com wrote:

This would be a nice way to handle the quirks and complexities of our
system, but so far it's pretty tricky on systems other than the
expected Debian
GNU/Linux
.

In our scripts, we've made lots of assumptions about things like home
directories
http://en.wikipedia.org/wiki/Home_directory#Default_home_directory_per_operating_system.
So installing on another system like OS X will necessarily involve some
kind of privileged "preflight" script to enable services, create users and
directories, etc. There are probably other obstacles ahead.

Perhaps the real answer is to use a local VM (VirtualBox, Parallels,
VMWare, etc) with a minimal Debian. Or a prepared VM (opentree and admin
users already created, keys in place) that's ready to accept push commands.


Reply to this email directly or view it on GitHub
#403.

@jar398

This comment has been minimized.

Show comment
Hide comment
@jar398

jar398 Aug 11, 2014

Member

I don't think that incrementally enhancing the deployment system is
necessarily a good investment. It really needs an overhaul, or we need to
switch to something like ansible.

Why do you want to use a local VM? It is very easy to set up new EC2
servers and I'll make one for anyone who wants one. The asterales system is
ready to be used if what you need is the whole system and not just the
front end.

On Tue, Aug 5, 2014 at 10:18 PM, Jim Allman notifications@github.com
wrote:

This would be a nice way to handle the quirks and complexities of our
system, but so far it's pretty tricky on systems other than the expected Debian
GNU/Linux
.

In our scripts, we've made lots of assumptions about things like home
directories
http://en.wikipedia.org/wiki/Home_directory#Default_home_directory_per_operating_system.
So installing on another system like OS X will necessarily involve some
kind of privileged "preflight" script to enable services, create users and
directories, etc. There are probably other obstacles ahead.

Perhaps the real answer is to use a local VM (VirtualBox, Parallels,
VMWare, etc) with a minimal Debian. Or a prepared VM (opentree and admin
users already created, keys in place) that's ready to accept push commands.


Reply to this email directly or view it on GitHub
#403.

Member

jar398 commented Aug 11, 2014

I don't think that incrementally enhancing the deployment system is
necessarily a good investment. It really needs an overhaul, or we need to
switch to something like ansible.

Why do you want to use a local VM? It is very easy to set up new EC2
servers and I'll make one for anyone who wants one. The asterales system is
ready to be used if what you need is the whole system and not just the
front end.

On Tue, Aug 5, 2014 at 10:18 PM, Jim Allman notifications@github.com
wrote:

This would be a nice way to handle the quirks and complexities of our
system, but so far it's pretty tricky on systems other than the expected Debian
GNU/Linux
.

In our scripts, we've made lots of assumptions about things like home
directories
http://en.wikipedia.org/wiki/Home_directory#Default_home_directory_per_operating_system.
So installing on another system like OS X will necessarily involve some
kind of privileged "preflight" script to enable services, create users and
directories, etc. There are probably other obstacles ahead.

Perhaps the real answer is to use a local VM (VirtualBox, Parallels,
VMWare, etc) with a minimal Debian. Or a prepared VM (opentree and admin
users already created, keys in place) that's ready to accept push commands.


Reply to this email directly or view it on GitHub
#403.

@jimallman

This comment has been minimized.

Show comment
Hide comment
@jimallman

jimallman Aug 11, 2014

Member

Why do you want to use a local VM?

To quote @kcranston from a recent discussion on IRC:

two use cases: 1. me, today: I want to make a simple UI change and test locally before creating a pull request. 2. A new developer wants to test the application for his / her use.

So it would be great to have a more "lightweight" option for getting up and running. Agreed that docker or a pre-built appliance could work well for these use cases, if it's easy to push and test changes.

I'll hold off on any work here for now, since (as you point out) the deployment system is still kind of in flux.

Member

jimallman commented Aug 11, 2014

Why do you want to use a local VM?

To quote @kcranston from a recent discussion on IRC:

two use cases: 1. me, today: I want to make a simple UI change and test locally before creating a pull request. 2. A new developer wants to test the application for his / her use.

So it would be great to have a more "lightweight" option for getting up and running. Agreed that docker or a pre-built appliance could work well for these use cases, if it's easy to push and test changes.

I'll hold off on any work here for now, since (as you point out) the deployment system is still kind of in flux.

@jimallman

This comment has been minimized.

Show comment
Hide comment
@jimallman

jimallman Sep 29, 2014

Member

@jar398, @kcranston, @temi : I spent most of this weekend tinkering with a fork of @temi 's Ansible deployment repo. I cleaned up some problems with the Ansible playbook (i.e., task and role definitions) and learned enough about Ansible to confirm that it can handle all the complications (virtualenv, etc) we would throw at it.

I'm setting this aside for now, but we might want to revisit this and come up with a series of playbooks to handle use cases like routine deployment, code review, and isolated systems for scientific exploration.

Member

jimallman commented Sep 29, 2014

@jar398, @kcranston, @temi : I spent most of this weekend tinkering with a fork of @temi 's Ansible deployment repo. I cleaned up some problems with the Ansible playbook (i.e., task and role definitions) and learned enough about Ansible to confirm that it can handle all the complications (virtualenv, etc) we would throw at it.

I'm setting this aside for now, but we might want to revisit this and come up with a series of playbooks to handle use cases like routine deployment, code review, and isolated systems for scientific exploration.

@jar398

This comment has been minimized.

Show comment
Hide comment
@jar398

jar398 Sep 29, 2014

Member

I spent some time on this over the weekend. The user name for the user
running the services is no longer hardwired as 'opentree'; you can set it
in the configuration file. I tested this and it seems to work. (I missed
one spot in one of the apache configuration files, need to get back to
that.)

There's currently an assumption that the files are located relative to that
user's home directory, but this should be easy to generalize (I'll make an
issue for this as I may not get to it this week).

We're very close to being able to use the deployment system to run locally
or in a VM. Not that I'm opposed to a rewrite of the deployment scripts,
though.

On Mon, Sep 29, 2014 at 12:48 PM, Jim Allman notifications@github.com
wrote:

@jar398 https://github.com/jar398, @kcranston
https://github.com/kcranston, @temi https://github.com/temi : I spent
most of this weekend tinkering with a fork of @temi 's Ansible deployment
repo https://github.com/OpenTreeOfLife/opentree-install. I cleaned up
some problems with the Ansible playbook (i.e., task and role definitions)
and learned enough about Ansible to confirm that it can handle all the
complications (virtualenv, etc) we would throw at it.

I'm setting this aside for now, but we might want to revisit this and come
up with a series of playbooks to handle use cases like routine deployment,
code review, and isolated systems for scientific exploration.


Reply to this email directly or view it on GitHub
#403 (comment)
.

Member

jar398 commented Sep 29, 2014

I spent some time on this over the weekend. The user name for the user
running the services is no longer hardwired as 'opentree'; you can set it
in the configuration file. I tested this and it seems to work. (I missed
one spot in one of the apache configuration files, need to get back to
that.)

There's currently an assumption that the files are located relative to that
user's home directory, but this should be easy to generalize (I'll make an
issue for this as I may not get to it this week).

We're very close to being able to use the deployment system to run locally
or in a VM. Not that I'm opposed to a rewrite of the deployment scripts,
though.

On Mon, Sep 29, 2014 at 12:48 PM, Jim Allman notifications@github.com
wrote:

@jar398 https://github.com/jar398, @kcranston
https://github.com/kcranston, @temi https://github.com/temi : I spent
most of this weekend tinkering with a fork of @temi 's Ansible deployment
repo https://github.com/OpenTreeOfLife/opentree-install. I cleaned up
some problems with the Ansible playbook (i.e., task and role definitions)
and learned enough about Ansible to confirm that it can handle all the
complications (virtualenv, etc) we would throw at it.

I'm setting this aside for now, but we might want to revisit this and come
up with a series of playbooks to handle use cases like routine deployment,
code review, and isolated systems for scientific exploration.


Reply to this email directly or view it on GitHub
#403 (comment)
.

@temi

This comment has been minimized.

Show comment
Hide comment
@temi

temi Oct 31, 2014

great! will check it out.

temi commented Oct 31, 2014

great! will check it out.

@jar398 jar398 self-assigned this Oct 7, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment