[WIP] Added circle-ci artifacts to prometheus-builder

[geekodour]

Closes #83
Closes #163

With this PR, prometheus-builder no longer builds prometheus, instead it downloads the artifact for that PR and uses that to copy it to the mounted volume.

prometheus-builder will no longer be needed to be updated any dependency change once this gets merged.

This is working locally, yet to test this on the cluster.

Another issue I faced was that, circle-ci api endpoint is not working as expected in few cases:

• https://circleci.com/api/v1.1/project/github/prometheus/prometheus/tree/pull/6326?filter=successful
• https://circleci.com/api/v1.1/project/github/prometheus/prometheus/tree/pull/6325?filter=successful

For example, api works as expected with prometheus/prometheus#6325 but not with prometheus/prometheus#6326

cc @krasi-georgiev @simonpasquier

Signed-off-by: Hrishikesh Barman hrishikeshbman@gmail.com

[krasi-georgiev]

hm , I am not sure what is the problem here, did you try to contact the circle CI support?

[krasi-georgiev]

one other thing for how long are these artifacts kept? I mean what would happen if the last run was 2 weeks ago, would the artifact be persisted?

[krasi-georgiev]

just re-read the original discussion.
I still think that we need to find some temporary place to build and put prometheus docker images from the PR and than delete when not needed. Than just pass the docker image as a param. This would allow to still run prombench locally without relying on circle ci, but since we discussed this a long time ago and I still can't think how to cleanup these PR images when not needed I would say that lets move forward with this suggestion and can refactor later if/when we have a better idea.

[geekodour]

@krasi-georgiev should we implement some kind of fall back? eg. if it cannot get the artifact from circleCI, attempt building it like what it does currently?

[simonpasquier]

The CI job triggered after prometheus/prometheus#6326 got merged hasn't completed though it's been started more than 30 hours ago.

[geekodour]

@krasi-georgiev

how long are these artifacts kept?

I could get get artifacts from a year old PR but they do not make any grantees as mentioned here

@simonpasquier

The CI job triggered after prometheus/prometheus#6326 got merged hasn't completed though it's been started more than 30 hours ago.

The issue seems to be something else from circleci's side, I tried getting api responses when the builds were successful as well. I'll open a ticket on circleci forum about it.

[krasi-georgiev]

From what I see using circle will be flaky so I really don't want to depend on circle for this, can you try to come up with a different solution?

[simonpasquier]

The issue seems to be something else from circleci's side, I tried getting api responses when the builds were successful as well. I'll open a ticket on circleci forum about it.

I don't know. Maybe your requests were rate-limited?

[geekodour]

can you try to come up with a different solution?

This currently tries to use the circleCI artifact first, if it fails to get the artifact then it tries to build it like what the current prometheus-builder does.

[geekodour]

I opened a question here: https://discuss.circleci.com/t/circleci-api-not-responding-correctly/35763

Putting it here to keep track.

[geekodour]

Since github actions now support artifacts, I think we could experiment with github actions artifacts instead of circleci artifacts here.

[simonpasquier]

it really looks like the CI run from prometheus/prometheus#6326 got stuck and never completed. Maybe it's happening from time to time but I don't remember anything like this before...