Provide a way to recreate containers

[carlosms]

From #20 (review)
@smacker said:

But we definitely need to think a bit more about how to re-create containers.
It is easy with only update command. But we also have sourced compose download and sourced compose set which right now don't work as expected. (At least not as I expect)

Our wrapper for docker-compose up is install/init. But it requires an argument with the working directory.
So right now we don't have a command for "run docker-compose up without changing the current workdir".

This should be discussed in the next team meeting.
Maybe we just need to add a sourced up command in addition to the current sourced install/init.

[smacker]

We have a solution for re-creation of the containers in the PR above but it's not clear if re-creations also mean re-sync or not.
With current implementation it does resync. We need to find out if we can properly disable re-sync only for this case without breaking anything else before moving forward.

[smacker]

how to avoid refresh on restart

gitcollector

gitcollector support --no-updates flag. It would stop gitcollector from updating existing repos but still would download new ones.

1. Up gitcollector with --no-updates by default always. Then restart won't cause a refresh (at least for existing repos)
2. For manual update run gitcollector without --no-updates using docker-compose run so it won't be restarted by up

ghsync

1. Multiple restarts don't refresh data
2. We don't have any implementation for a manual or automatic refresh but most probably for a manual one we can also run it with docker-compose run. And for automatic one, it should be possible to disable.

result

1. We should request the ability to skip any download from gitcollector similar to ghsync
2. We can use the PR from above with a different command name. I suggest restart instead of up because without knowledge of how docker-compose works up name is confusing.

agree? @se7entyse7en @carlosms

[smacker]

Another option is to call it "update". So it can be used for update from a manual change in docker-compose.yml or after a user downloads new binary (we will need to add logic to check that docker-compose.yml matches binary version)

[carlosms]

Your proposal makes sense. Especially to make the ghsync and gitcollector behavior as close as possible. What you describe does not only affect this new up/restart command right?
For example:

• you call sourced init orgs src-d today
• this evening a new repo is added to the org
• tomorrow you call init again, or restart the laptop and the containers are restarted. ghsync will skip the organization src-d, but gitcollector will clone the new repository.

The result is that the dashboards will show inconsistent data from gitbase and metadata.

[smacker]

yes. Init and up/restart/update should work absolutely the same.

Let's agree on the name then and I'll refactor the PR with these changes.
My vote +2 to update and +1 to restart. (the reason: fewer number of commands is better in my opinion)

[carlosms]

I'd prefer restart or up for the command that only recreates the containers.

Should we treat the gitcollector - ghsync difference on restart something independent to this command, and open an issue somewhere to discuss it further?

[smacker]

Should we treat the gitcollector - ghsync difference on restart something independent to this command, and open an issue somewhere to discuss it further?

do you propose implementing restart command that doesn't only recreates the containers?
or I didn't understand you....

[se7entyse7en]

I'd prefer restart or up for the command that only recreates the containers.

I vote for restart for the command to restart containers, and I'd expect an error if restart is called before an init.

[smacker]

Anyway, the PR is updated according to the discussion.

[carlosms]

Should we treat the gitcollector - ghsync difference on restart something independent to this command, and open an issue somewhere to discuss it further?

do you propose implementing restart command that doesn't only recreates the containers?
or I didn't understand you....

What I meant is that #83 initially was proposed as a way to do 2 things: recreate the containers + refresh the data in the future.

In here we agreed to make ghsync & gitcollector work in a similar way: if the container is restarted and in a previous run it finished the download, this time it should not look for updates or new repositories.

So if we rework #83 it will still have partial gicollector refresh, it will try to download new repositories.

By

Should we treat the gitcollector - ghsync difference on restart something independent to this command, and open an issue somewhere to discuss it further?

What I mean is that probably it makes sense to go ahead with it and not block restart until we have gitcollector that does not refresh automatically. But I was asking in case some of you think differently, and we should first have the gitcollector support discussed with DR team before working on restart.

[smacker]

Ah. I see. Thank you for the explanation. With d3ff64a the only update that would happen is downloading new repositories which doesn't happen often. I don't think we should block because of it.
I have created an issue in gitcollector: src-d/gitcollector#53