Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use docker-compose to run crates.io inside Travis #1814

Closed
wants to merge 1 commit into from

Conversation

@kzys
Copy link
Contributor

commented Aug 27, 2019

No description provided.

@rust-highfive

This comment has been minimized.

Copy link

commented Aug 27, 2019

r? @carols10cents

(rust_highfive has picked a reviewer for you, use r? to override)

@kzys

This comment has been minimized.

Copy link
Contributor Author

commented Aug 27, 2019

This is the first step towards #868. Right now this new "Integration Tests" task is really slow, but there are a few things that would make the task faster.

1. Move more build tasks into Docker

Currently the existing Travis CI tasks download all of dependencies, but the new task also downloads them again inside a container. Moving all of them inside the container would allow us to cache the artifacts between the tasks.

2. Have intermediate Docker images on a repository

The Rust team apparently have some Docker images under https://hub.docker.com/u/rustops already. We can have crates.io's images on the repository to cache intermediate artifacts.

3. Use Docker in Production!

Okay. This might be too far. But Heroku supports Docker. Using Docker in Heroku would make reproducing Heroku's environment straightforward for developers. In other words, we can achieve production-development parity.

Heroku's Docker environment is a bit weird. The web container would be a fat container that contains frontend, backend and Nginx. But we are doing that anyway through Buildpacks. So we don't have to be a container idealist just because we run Docker in production :)

What do you think?

@smarnach

This comment has been minimized.

Copy link
Contributor

commented Aug 30, 2019

The point I don't understand here is why we need docker-compose on Travis for end-to-end tests. We could also run both the frontend and the backend directly in the Travis container, without the additional level of containers.

(I'm not disagreeing with you, just curious how you got to this design.)

@kzys

This comment has been minimized.

Copy link
Contributor Author

commented Aug 30, 2019

Good point. We can!

The biggest reason I choose docker-compose here is that we already have docker-compose.yml and 2 Dockerfiles to run backend and frontend together. In other words, someone has chosen that before me :)

We also have a completely different set of configuration files (Procfile and buildpacks) for the production environment.

So, for me, the options were;

Reusing Procfile and buildpacks

Pros:

  • Don't increase the number of configurations

Cons:

  • Running them without Heroku should be technically possible, but I'm not so sure how we can do it.

Reusing docker-compose.yml

Pros:

  • Don't increase the number of configurations

Cons:

  • Run-time overhead
  • Needs Docker

Write a small shell script to run frontend and backend together

Pros:

  • Simple
  • Low run-time overhead

Cons:

  • Increase the number of configurations.
  • Anyone who want to run the integration tests need to install PostgreSQL and other dependencies on the host.
@smarnach

This comment has been minimized.

Copy link
Contributor

commented Sep 2, 2019

@kzys Thanks for the additional explanation.

I doubt it is feasible to make the production, test and development environments much more similar. In production we don't "run" the frontend – it's just a bunch of compiled static files, so I've got no idea what the "frontend" container would be supposed to be doing in production. In the development environment, we want to run the Ember test server, to detect changes and automatically recompile the JavaScript files. The test environment will have yet slightly different requirements.

I'd personally tend to avoid the complexities of using Docker containers for the tests, and certainly in production, since I don't see enough advantages, but I don't really know enough about our deployment for my opinion to bear much weight.

@sgrif

This comment has been minimized.

Copy link
Contributor

commented Sep 5, 2019

I agree with @smarnach here:

I'd personally tend to avoid the complexities of using Docker containers for the tests, and certainly in production, since I don't see enough advantages

It looks like you're working on an alternative PR, so I'm going to close this in favor of that one

@sgrif sgrif closed this Sep 5, 2019

@kzys

This comment has been minimized.

Copy link
Contributor Author

commented Sep 5, 2019

@sgrif Can we keep this one open? #1825 is still work in progress and not working correctly. I'd like to have two working solutions first.

@sgrif

This comment has been minimized.

Copy link
Contributor

commented Sep 5, 2019

I'm not sure there's much reason to keep this open, given the consensus from two team members we don't want to go in this direction

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.