Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deployment Strategies #473

Open
GandaG opened this issue Oct 11, 2019 · 6 comments
Open

Deployment Strategies #473

GandaG opened this issue Oct 11, 2019 · 6 comments
Labels
A-scripts Area: Scripts (Everything in the scripts folder) C-discussion Category: Discussion (among developers trying to find consensus on a design etc.)
Milestone

Comments

@GandaG
Copy link
Member

GandaG commented Oct 11, 2019

Now that we're halfway to a proper environment setup and building we can stop deploying like cavemen and automate everything!

  1. Dropbox - Nightly only

    • Currently we manually copy files to a shared folder. Like cavemen. Disgusting.
    • Automated deployment would necessarily include creating a dropbox app with "Full Dropbox" permissions ("App Folder" does not allow folder sharing).
    • The dropbox app provides "client id" and "secret" that should be kept stored in a secure location - we currently only have the discord dev channels as such a place.
    • Dev dropbox credentials would be required to deploy via the dropbox app.
    • The deployment should default to the root of the shared folder, with an optional cli arg to deploy to a branch folder (maybe for testing purposes? Could be useful).
  2. Nexus - Production only

  3. GitHub Releases - Production only

    • Currently only fully stable production releases (non-beta) are uploaded.
    • Requires a tagged commit to create.
    • Should be simple enough to implement, either pygithub can do it or the Releases API will work, they both look simple to work with.
    • We have a script to generate the changelog.
  4. Discord - Nightly only

    • Currently @Infernio toils away at updating the post in the wip-builds channel with a beautiful changelog for every nightly - is he not entitled to the sweat of his brow? "No!", says @Sharlikran, "it belongs to the users". "No!", says @Arthmoor, "it belongs to the community". "No!", says @FelesNoctis, "it belongs to the devs". I rejected those answers; instead, I chose something different. I chose to automate.
    • If we want to keep a single post on the wip-builds channel we'll need a bot to edit it - this requires a place to host the bot I believe, unless it can auto-add itself.
    • If we're going with new posts on that channel we can do something like LOOT does, with a webhook, which is substantially easier.
    • The main issue are the changelogs - maybe start keeping a written changelog? Not sure what to do here. See also "What's New" / "Recent Changes" popup when launching Wrye Bash #465.
  5. Linux repositories - Production only

    • Currently deploying to the AUR on new production releases.
  6. Chocolatey - unplanned - Nightly and Production

    • No current deployment here.
    • Not sure how feasible this is, but something we should keep in mind.
    • If we're allowed to set package dependencies, we could on the python package and run the source code directly - this would bypass some freezing issues like LOOT problems.

General Considerations:

  • All credentials used when deploying should be saved somewhere to avoid devs having to reintroduce them every time. Ideally, the could also be provided by cli args to help with CI integration.
  • All deployments should be a one-liner that required minimal to no interaction from the dev - some of these could be lengthy and forcing the dev to watch a log is no fun.
  • Since these deployments are all meant to be used every time (with exceptions for nightly/production-only) we could use an umbrella script to facilitate deployments.

Related Branches: ganda-473-deploy

I think you all know my thoughts on this.

Discuss!

@GandaG GandaG added C-enhancement Category: Enhancement, a request to add or enhance a feature C-discussion Category: Discussion (among developers trying to find consensus on a design etc.) labels Oct 11, 2019
@Utumno Utumno added this to the 308 milestone Oct 13, 2019
@Utumno
Copy link
Member

Utumno commented Oct 13, 2019

Currently we only upload production files manually. It's a ton of work to do this manually, this is probably the highest priority to automate.

I would agree let's have 307 out that way - assigning the rest to 308 although they may be done sooner - priority remains wx3/phoenix/py3 update for 307

@GandaG
Copy link
Member Author

GandaG commented Oct 15, 2019

let's have 307 out that way

307 Beta 4 was uploaded successfully via the scripts in ganda-260-deploy. There was a small hitch that is being worked on due to Infernio's Chrome freaking out a bit :P

@GandaG
Copy link
Member Author

GandaG commented Oct 22, 2019

Small update, deployment branch has been split per target

@Infernio Infernio removed the C-enhancement Category: Enhancement, a request to add or enhance a feature label Mar 2, 2020
@Infernio
Copy link
Member

Infernio commented Mar 2, 2020

(Dropped C-enhancement since this is primarily a discussion issue for now - readd enhancement in 308 once discussion is decided)

@Infernio Infernio added the A-scripts Area: Scripts (Everything in the scripts folder) label Mar 18, 2020
@Infernio Infernio modified the milestones: 309, 310 Feb 22, 2021
@Infernio Infernio modified the milestones: 310, 311 May 18, 2021
@Infernio
Copy link
Member

For tracking purposes, the upstream issue that's blocking us from just uploading our nice, script-built 7z files is this: actions/upload-artifact#39

Utumno added a commit that referenced this issue Dec 19, 2021
Very useful so we can quickly deploy a test exe to users - especially if
you don't have access to a windows machine.

WIP:
- better glob pattern/ do cp in same step or even better
- try using already packed one (renaming it) -> nope:
actions/upload-artifact#39

This works but does not create the Apps foler (duh) and does not put
everything in mopy - see

https://github.com/actions/upload-artifact#upload-using-multiple-paths-and-exclusions
search for "flattened"

https://docs.github.com/en/actions/advanced-guides/storing-workflow-data-as-artifacts

@@ -37,5 +37,5 @@ jobs:
         python -m pytest
-#    - name: Copy executable
-    - name: Standalone Executable
+    - name: Copy executable
       run: cp "scripts/build/standalone/dist/Wrye Bash.exe" Mopy
+    - name: Standalone Executable
       uses: actions/upload-artifact@v2

a step cannot have both the `uses` and `run` keys

https://docs.github.com/en/actions/learn-github-actions/expressions#format
https://docs.github.com/en/actions/learn-github-actions/contexts
format trick from
https://github.community/t/concatenate-env-variables-and-strings/128201/3

scripts/build.py: (empty) env folder was included in standalone
List of ignores from that pack_standalone.ignores

Under #473
@Infernio Infernio modified the milestones: 311, 312 Jul 4, 2022
@Infernio
Copy link
Member

@Infernio Infernio modified the milestones: 312, 313 Jan 5, 2023
@Infernio Infernio modified the milestones: 313, 314 Dec 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-scripts Area: Scripts (Everything in the scripts folder) C-discussion Category: Discussion (among developers trying to find consensus on a design etc.)
Projects
None yet
Development

No branches or pull requests

3 participants