Syncing across multiple envs is a bit difficult

[josephjclark]

(not sure if I have an issue for already)

Using CLI sync in multiple environments is quite hard.

You need two things to sync: an endpoint (a URL to a lightning instance) and an API key.

If you have a project on github that connects to two instances, it's hard to manage these two variables.

Usually you'll have an environment set up with a single endpoint and api key, so you can just do:

openfn pull main

And it'll update your main from the environment.

But on instance two, you have to do:

openfn pull main --api-key $OTHER_API_KEY --endpoint $OTHER_ENDPOINT

Which is just a pain.

After you've fetched the project once, the project.yaml file will remember the endpoint. So with aliasing you can at least do:

openfn pull prod --api-key $OTHER_API_KEY

Which is one less thing, but still not great.

The problem really is that the API hey is sensitive and we don't want to risk checking it in to github.

Solutions:

• In the cli section of project file, store an env var name for the API key.
• Or in the cli section, put a path to an env file generally
• Set up some global CLI config which binds an API key to an endpoint. Now whenever the CLI calls one endpoint, it'll use the same API key

[josephjclark]

Not sure if this is related or not, but deploy can be a bit weird:

openfn deploy prod

Will take what you have currently checked out and deploy it to the prod alias.

Similarly

openfn deploy --endpoint www

Will take what is checked out and deploy it to the endpoint you feed it (if this is a new deploy you also need --new)

This doesn't mean "deploy the prod project", it means "deploy my filesystem to prod" which is a bit subtle and I'm not totally sold it works.

We might be better off doing:

openfn deploy <alias> --target <urlOrAlias>

Which will first check out the project you want to deploy (bit wierd?) and then deploy it to the endpoint you specifiy.