New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow public read-only access #137

Open
essen opened this Issue Dec 20, 2015 · 33 comments

Comments

@essen
Copy link

essen commented Dec 20, 2015

I would like to give read-only access to my users, at least for builds coming from pull requests. Right now they can't check the reasons for failure from the link in the PR.

@imbriaco

This comment has been minimized.

Copy link

imbriaco commented Feb 4, 2016

This is also an issue for us. We need the ability to share build output with unauthenticated users who may be able to see PR build failures but not have an account on Buildkite.

@keithpitt

This comment has been minimized.

Copy link
Member

keithpitt commented Feb 23, 2016

Over the last few months we've been refactoring our data access layer to support public builds. Most of this is done now, and it's now possible for us to add this feature (before it was close to impossible). The only things for us to figure out now is the scope of the feature. Here are some things that may be in scope for this feature:

  • Ability to delete a log, just in case something sensitive is outputted
  • Being able to delete an artifact in the event you upload something sensitive
  • ?
  • If someone who isn't part of your organization triggers a build through a Fork + Pull Request, they won't be able to cancel it, since they don't have access.
  • How do we educate users that turn on "public builds" and "build forked repos" about the security of their agents?

That's a bit of brain dump about public builds, and the things we need to think about. I'm keen to open up the discussion to you folks to see what you think! Obviously, the less that's in scope for this feature, the easier it is to ship, but since this can cause some security issues, I don't want to rush it.

@essen

This comment has been minimized.

Copy link
Author

essen commented Feb 23, 2016

In my case all the PR builds require my intervention before they run (a no-op followed by the continue/lock). I have separate projects/pipelines for PRs/pushes. I have no sensitive thing whatsoever. I just want them to have read-only access to the build pages (even for non-PRs), no canceling needed or anything.

About educating users, I suppose the same kind of warning as the one for building third party PRs is required. The most sensitive is the building part anyway, that's where anyone could put any code and run it on your machines.

Perhaps make a mandatory manual confirmation (like what I already do) and/or have a whitelist would be useful.

@keithpitt

This comment has been minimized.

Copy link
Member

keithpitt commented Feb 23, 2016

How are you keep those builds secure on your machines out of interest? Happy to talk more about it in Slack in a DM if you don't want to talk about it here.

The manual confirmation when they turn this feature on I think is a good idea! A bit like how when you delete a repo on GItHub, they require you to re-enter the name of the repo.

Another thing I just though of is the pipeline upload implication. Someone can fork a repo, run buildkite-agent pipeline upload bad.yml and completely change the running pipeline. We'd probably need a way to turn that off or something...

@essen

This comment has been minimized.

Copy link
Author

essen commented Feb 23, 2016

I read the PR before I run the builds. :-) Most PRs are small so any malicious intent is quickly detected. The repos do not contain any BuildKite configuration either it's all done from the site so there's little that can be broken.

@toolmantim

This comment has been minimized.

Copy link
Member

toolmantim commented Feb 24, 2016

@lox has also done work on the Buildkite AWS for securely running third party PRs, we have some technology

@toolmantim

This comment has been minimized.

Copy link
Member

toolmantim commented Feb 24, 2016

*Buildkite AWS Stack

@johanneskoester

This comment has been minimized.

Copy link

johanneskoester commented Apr 13, 2017

Any update on this? In the bioconda project (https://github.com/bioconda), we need people to see the build log publicly.

@pquerna

This comment has been minimized.

Copy link

pquerna commented Apr 17, 2017

We'd also like to use Buildkite for an open source project we have coming out; We're less worried about full features, but mostly being able to see a build-log via a public url. Is a solution for open source projects something coming "soon"? Thanks!

@keithpitt

This comment has been minimized.

Copy link
Member

keithpitt commented Apr 19, 2017

@johanneskoester @pquerna howdy! Sorry for the reply. This is on our roadmap and we're working on it in an upcoming cycle. Watch this space! 🔜

@johanneskoester

This comment has been minimized.

Copy link

johanneskoester commented Apr 19, 2017

@keithpitt That's great, thanks! Can you give us a rough estimate how soon? Within weeks or months? I am asking because I will work on a workaround if it takes more than weeks.

@prudhomm

This comment has been minimized.

Copy link

prudhomm commented Jun 24, 2017

what is the status of this ? being able to provide read-only access would be most helpful !

@toolmantim

This comment has been minimized.

Copy link
Member

toolmantim commented Jun 28, 2017

@johanneskoester @prudhomm this is still high on the cards, but nothing to show yet sorry!

@sj26 sj26 added the feature label Jul 4, 2017

@schickling

This comment has been minimized.

Copy link

schickling commented Sep 28, 2017

Any update on this @toolmantim? :)

@toolmantim

This comment has been minimized.

Copy link
Member

toolmantim commented Sep 28, 2017

@schickling not yet sorry, but we’re working on it

@domenkozar

This comment has been minimized.

Copy link

domenkozar commented Jan 24, 2018

One of main challenges here is to provide a good security story when builds use some sensitive information from agents.

@KevinGrandon

This comment has been minimized.

Copy link

KevinGrandon commented Jan 24, 2018

One of main challenges here is to provide a good security story when builds use some sensitive information from agents.

You probably shouldn't be logging output in that case?

I do think that the environment variables tab will probably need to be hidden for unauthenticated builds, and also probably something like #293 solved to enable this in a secure way for pull requests.

@nh2

This comment has been minimized.

Copy link

nh2 commented Jan 24, 2018

I think there are different stages at which this feature can be provided.

I believe the simplest to start would be to make it suitable only for builds that

  • do not use any form of secrets
  • do not build PRs, so that only code from trusted branches is built

I believe this would already cover 95% of open-source builds.

More sophisticated use cases could be added later, but right now many people cannot use buildkite because it is essentially useless for public builds of open-source projects.

@domenkozar

This comment has been minimized.

Copy link

domenkozar commented Jan 24, 2018

What @nh2 said. Next set could be buildkite github bot that triggers builds per github authd users through comments.

@essen

This comment has been minimized.

Copy link
Author

essen commented Jan 24, 2018

I agree it can easily be done for builds that do not use any form of secrets, but I think that should include PRs.

I already run public PRs automatically and have a step that requires me to click something in BuildKite to make it continue. This click should be only for users with the right credentials (it probably already checks them) but everything else can be public.

@avtar

This comment has been minimized.

Copy link

avtar commented Jan 24, 2018

Next set could be buildkite github bot that triggers builds per github authd users through comments.
@domenkozar #288 :)

@keithpitt

This comment has been minimized.

Copy link
Member

keithpitt commented Jan 25, 2018

Hey everyone, thanks for the ideas/thoughts :)

The good news is, we're actively working on this and having to have beta out soon. You're right in that it's a little tricky because we want to ensure we've got a good store to tell around security and abuse, and we're hoping to have all the rough edges ironed out for a beta soon.

We'll drop everyone a line here when were' looking for testers!

@buchgr

This comment has been minimized.

Copy link

buchgr commented Feb 7, 2018

Yay! We would also love to have this!

@johnae

This comment has been minimized.

Copy link

johnae commented Mar 22, 2018

Looking forward to the beta.

@petemounce

This comment has been minimized.

Copy link

petemounce commented Apr 27, 2018

I would really like it if there were some piece of documentation that told me something like "if you set your secret inside an environment variable with SECRET as part of its name, it will be obfuscated within all build logs".

We're intending to allow external contributions to some open source codebases, and so while we absolutely want those contributions to benefit from CI and reading the logs so they can self-service debugging, we'd prefer to minimise the amount of contribution-reading it's necessary to do before allowing a build to run.

@schani schani referenced this issue Jun 29, 2018

Merged

Dart #928

@sqs

This comment has been minimized.

Copy link

sqs commented Jul 3, 2018

We (https://sourcegraph.com) would like to open-source some repositories that we use Buildkite for, including our popular Sourcegraph browser extension. We are comfortable with beta stuff and putting in some extra work to do this. We would also be happy to blog about it and help spread the word about how awesome Buildkite is if this is possible soon. The alternative is that we'd have to eventually switch to Travis CI or CircleCI. You can email me at sqs@sourcegraph.com to hash out details.

@toolmantim

This comment has been minimized.

Copy link
Member

toolmantim commented Jul 6, 2018

In the meantime, one idea could be to use something like https://github.com/uw-ipd/github-checks-buildkite-plugin to publish checks for your builds? 🤔

@wilzbach wilzbach referenced this issue Jul 8, 2018

Merged

Add buildkite config script #225

3 of 6 tasks complete
@SuperQ

This comment has been minimized.

Copy link

SuperQ commented Jul 16, 2018

👍 The Prometheus open source monitoring project would be very interested in being able to share build output. We have many non-core contributors that can't see why their builds fail so we have to proxy build output for them. This wastes everyone's time. 😢

@mvines

This comment has been minimized.

Copy link

mvines commented Jul 16, 2018

FWIW I recently wrote a little heroku-based front end to buildkite that enables 3rd party contributors to view buildkite logs/artifacts from their pull requests: https://github.com/mvines/ci-gate

Additionally project maintainers need to approve these 3rd party PRs for CI by adding a label (presumably once they’ve confirmed the PR isn't malicious).

Maybe others will find my project useful until buildkite can support this use case natively.

@petemounce

This comment has been minimized.

Copy link

petemounce commented Jul 16, 2018

@mvines that sounds like one of the capabilities I want to give prow, which the kubernetes team built to handle their GitHub workflows.

Thanks for sharing. Relevant to #360 too.

@plasticine

This comment has been minimized.

Copy link
Member

plasticine commented Nov 22, 2018

Hey there everyone! 👋🏼 Just a quick one with some news on Buildkite public builds!

We’ve been busy working away on getting an initial version ready for prime time, and are currently ramping up towards making it generally available. However, given the amount of community interest in this feature, we’re also really keen to make sure we have time for feedback, and allow interested parties to kick the tyres a bit beforehand.

If you’re interested in getting into the beta group checking public builds out, please jump over to our community forum, sign up and shoot through a message asking for access — we’ll help get you set up from there! 🎉

@KevinGrandon

This comment has been minimized.

Copy link

KevinGrandon commented Nov 22, 2018

Hey there everyone! 👋🏼 Just a quick one with some news on Buildkite public builds!

Great start! Any information on what's planned for gating third party builds? ci-gate has some great ideas behind it, it would be really great if Buildkite could adopt something like this.

@plasticine

This comment has been minimized.

Copy link
Member

plasticine commented Nov 22, 2018

@KevinGrandon Hey, thanks! We’re just getting started with support for public builds so the focus right now is just starting to open things up. The longer term goal absolutely being awesome support and tooling around testing third-party contributions and supporting OSS. We’d be very interested in hearing about specific workflows or use-cases in this space on the forum to help us shape this work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment