Automate website staging of in-progress PRs

[LarkAscending]

Currently travis.sh deploys automatically to flutter.io when a branch is pushed to master:

if [ "$TRAVIS_EVENT_TYPE" = "push" ] && [ "$TRAVIS_BRANCH" = "master" ]; then
  # Deploy pushes to master to Firebase hosting.
  # TODO: deploy to a personal staging site, based on github ID, when not
  #       merging into master
  echo "Deploying to Firebase."
  npm install --global firebase-tools@3.0.0
  firebase -P sweltering-fire-2088 --token "$FIREBASE_TOKEN" deploy

(which is great, BTW)
This issue describes and tracks the TODO, which has become a high priority. This feature will allow us to greatly scale our doc efforts internally and externally as we push toward I/O.

We'd like to:

1. Automatically stage for each branch where a PR has commits and has not yet been merged into master.
2. Stage on each successful Travis run and display a URL (on success). If that is too often, we can discuss a plan B.
3. Could we create a new instance for each PR/branch, rather than for each user? When a user deletes their branch, that would be the trigger to automatically clean up and delete the staging deployment. Also, this means the user would not be overwriting their own staged builds when multiple PRs are in-progress...

[sethladd]

Thanks! This is very helpful.

I see: I was using "staging" mean "a place before production", and it looks like you were using "staging" as "deployed". Ah, pesky semantics :)

yes, we can help improve this for you :)

[LarkAscending]

We were on the same page semantically - today's workflow was not typical :-)
Sorry for the turbulence.

Happy to help test/debug this feature, if needed.

[mit-mit]

How many staging areas do we need? Would one per developer be enough (i.e., on every PR, we deploy to something like https://flutter-website-staging.firebaseapp.com/mit-mit/). That would mean that you can always only see the staging content for the most recent PR you have uploaded.

I'm asking as if not we would need to find some way of cleaning up older staging builds, and then it quickly becomes complicated.

[sethladd]

I'm asking as if not we would need to find some way of cleaning up older staging builds, and then it quickly becomes complicated.

Yeah, we were talking about this offline. It's a shared concern.

[FaisalAbid]

I can take on this task -
Let me know any specific requirements, what I can gather is basically

1. Someone sends a PR
2. Create a firebase staging based on that PR, deploy the PR if the build passes
3. Update the PR on github with a URL to the staging link
4. Delete the staging when the PR is closed or the branch is deleted
5. Delete stages that are > 15 days old (and have not been updated in 15 days or more)

[sethladd]

Thanks! I added a few items.

[FaisalAbid]

Great sounds good, I'll start on this tonight or tomorrow and have something shortly. Will keep this updated

[sethladd]

Very cool. Let me know what permissions you need, if any.

[LarkAscending]

@mit-mit @sethladd One instance per developer means when I'm working on multiple PRs at once, I'm overwriting my own changes.

In the per-branch or -PR model, we can know to delete the staging build when the branch gets deleted. That's the idea there. No old builds are hanging around, and there's no ambiguity on when it should be cleaned up.

[FaisalAbid]

I think having a one per branch model makes sense vs a one per dev model. Def we dont want to override the previous PR stuff

[sethladd]

One branch per PR helps developers who work on multiple, concurrent patches to the website.

Branches don't get auto-deleted when a PR is merged, so I think we can look at multiple signals for when to delete/remove the staged code. Some signals: too old (> 15 days is a strawman), branch is deleted, PR is merged.

[FaisalAbid]

Agreed

[Hixie]

@LarkAscending what's the code name for the customer who is requesting this? (Or should this be in "no milestone necessary"?)

[FaisalAbid]

Happy Presidents Day or Family Day!

I've started work on this now. Will update soon

[FaisalAbid]

hope everyone had a good day off!

Here is what I'm proposing for this. There are a few issues with what we oriignally wanted, mainly because firebase hosting does not support "diff" or partial updates. If we had a staging firebase deployment it would get overriden each time when someone sends a pr.

https://stackoverflow.com/questions/30677351/upload-single-file-to-firebase-hosting-via-cli-or-other-without-deleting-existin

Which led me to come up with another solution which is to use Heroku for the stage builds.

Thoughts?

Currently i have a simple flow setup with Travis, that creates a folder based on the branch thats in pull request and deploys it when it runs ./doc.sh

See #8294

[FaisalAbid]

I might have confused myself with the issue being created in this repo and being meant for flutter/website.

I thought it was for API docs website which is generated through docs.sh. Either case the change is very small to get it working for the main website, just looking for feedback on the heroku approach and I'll make the change.

[mit-mit]

No, we don't need this for API docs as the formatting rarely changes there. The real need is for PRs for https://github.com/flutter/website.

[FaisalAbid]

Okay got it! So this repo is used for issues for both website and SDK etc?

Will make the changes shortly and open the PR at website.

Thanks!

[sethladd]

Apologies for the confusion. Thanks for making this work for the website.

[sethladd]

If we can make this work with Google Cloud Storage and their hosting of static docs, that would be preferable. But, we're flexible.

[FaisalAbid]

Hey, that's a good idea, okay let me investigate Google cloud storage

[LarkAscending]

@FaisalAbid Not your fault - it's confusing. All the issues are with flutter/flutter, and this started w/an issue.
It's the Label dev: docs- website that differentiates.

[FaisalAbid]

Hey guys, update on this issue. I got it working with firebase deploy, we create a few firebase projects and cycle over them, deploy, check old branches, all that fun stuff!

We even have a PR Bot that updates the pull request with the link to the staged project.

I'm cleaning up and testing, hope to send a PR by end of day today if all goes well.

[sethladd]

We have turned this on. Woohoo!

[LarkAscending]

Travis website builds are failing for #431 and #432.

Error: Command requires authentication, please run firebase login

upstream error:

Unauthorized request.
#0      FirebaseClient.send (package:firebase/firebase_io.dart:95:11)

Could this be a dependency issue? Neither @cbracken nor @LarkAscending has the dependencies installed for FB CLI.

[FaisalAbid]

Taking a look, thats bizzare

[sethladd]

Declaring success!

Thanks @FaisalAbid !!

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.