Deprecate and remove dev branch

[jdangerx]

Tasks

Beta Give feedback

1. scour documentation for references to dev branch
2. remove dev from RTD
3. Update nightly build infrastructure to run from main instead of dev
4. Set update-conda-lockfile to run against main rather than dev
5. Set pre-commit-ci to run against main rather than dev
6. Set dependabot to run against main rather than dev
7. Update nightly build script to output to /nightly rather than /dev when running on a schedule
8. Explain the new git workflow to internal folks
9. Reset main to dev so that they share history, protect the dev branch so nobody can push to it
10. Set up appropriate branch protections for main, nightly, and stable
11. Delete the old gs://nightly-build-outputs.catalyst.coop bucket
12. Update header information in Datasette to point at nightly not dev
13. Update PUDL dataset on Kaggle to point at nightly outputs
14. Tell external people to pin to main, nightly, or stable, instead of dev branch
15. Get Start EIA-176 pipelines: company data #2949 reopened against main
16. Remove /dev outputs from GCS and S3 in favor of /stable and /nightly outputs

[zaneselvans]

What do we think the appropriate branch protections are for main, nightly, and stable?

It seems like nightly and stable should only every be modified by the automated processes (unless something goes awry).

The protections we have on dev right now seem like they would probably get moved to main? Namely that it has to be changed by a PR with at least one reviewer, and all the "required" CI checks need to pass?

[zaneselvans]

Verifying that our first successful main/nightly/stable build behaved as expected:

• nightly-YYYY-MM-DD tag was created
• nightly branch was updated to point at main
• build outputs were written to gs://builds.catalyst.coop/BUILD_ID
• appropriate build outputs were copied to gs://pudl.catalyst.coop/nightly
• appropriate build outputs were copied to s3://pudl.catalyst.coop/nightly
• Datasette was redeployed with build outputs
• PAT is not present in published logfile

[zaneselvans]

Issues:

• We're not capturing logging output from all of the commands inside the composite functions, like when the files are copied to S3, but the files are being copied and if the copy failed, I think we'd get a build failure.