Instructions for GitHub Actions workflow

[heitorlessa]

Hello again :-) I've been trying to use Mike with GH Actions to deploy a develop version upon pushes to the develop branch, and version specific upon a new release (i.e. 1.20.1), however pushing to develop is wiping out existing versions file assets.

The version file looks correct but every time develop branch is updated it wipes all sub-directories for other versions.

Am I missing a special parameter, sequence, or perhaps in this case I should always have an alias tied to a version like mike deploy --push develop unreleased?

Appreciate the help!

---

Here's a screenshot of the file structure before and after, and the GitHub action workflow:

New release works as expected

New develop release wipes out the previous version

Develop version workflow

Always builds the latest doc when pushing to develop branch

name: Docs

on:
  push:
    branches:
      - develop

jobs:
  api-docs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          fetch-depth: 0
      - name: Set up Python
        uses: actions/setup-python@v1
        with:
          python-version: "3.8"
      - name: Install dependencies
        run: make dev
      - name: Setup doc deploy
        run: |
          git config --global user.name Docs deploy
          git config --global user.email docs@dummy.bot.com
      - name: Build docs website and API reference
        run: poetry run mike deploy --push develop

New release version workflow

Creates a new version and update aliases

name: Docs new version release

on:
  release:
    types: [published]

jobs:
  upload:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
      with:
        fetch-depth: 0
    - name: Set up Python
      uses: actions/setup-python@v1
      with:
        python-version: "3.8"
    - name: Set release notes tag
      run: |
        export RELEASE_TAG_VERSION=${{ github.event.release.tag_name }}
        echo "RELEASE_TAG_VERSION=${RELEASE_TAG_VERSION:1}" >> $GITHUB_ENV
    - name: Install dependencies
      run: make dev
    - name: Setup doc deploy
      run: |
        git config --global user.name Docs deploy
        git config --global user.email docs@dummy.bot.com
    - name: Release docs website and API reference
      run: |
        poetry run mike deploy --push --update-aliases ${RELEASE_TAG_VERSION} latest
        poetry run mike set-default --push latest

[heitorlessa]

Update: It must be something else indeed as I've just tried mike deploy --push develop stage and it also removed a previous version I had. I suspect it must be something with the checkout@v2 action

[jimporter]

That's probably this issue. Short version: GH actions use --depth 1, which means you only get the latest commit on the current branch. Setting fetch_depth: 0 gets you a proper clone so that it has a local copy of gh-pages to commit to; otherwise you'll lose the previous commits since you're creating a fresh gh-pages branch locally, and then pushing it.

On the other hand, I'd expect that your code would fail unless you pass --force to mike deploy, so maybe something else is happening. It certainly should fail, since mike deploy shouldn't normally overwrite history.

There's probably a better way to do this that doesn't involve making a full clone of the repo, but I'm not familiar enough with all the various GH Actions features to know for sure.

[heitorlessa]

Thanks @jimporter I saw that issue the first time it failed hence why I added the depth: 0 in the workflow, since I figured if it works locally but not in GH there was something else at play.

If it's not that jumps to mind immediately, I'll try forcefully fetching the latest gh-pages branch as part of the workflow to see if that helps

Thank you!

[jimporter]

Ah, so you did. I should probably make a habit of having my coffee first and then replying to issues. :)

Ok, I think I figured it out... kind of. Somehow, the develop branch is getting an alias of "", i.e. the empty string. That gets splattered all over the root directory of your gh-pages branch and clobbers everything else. I don't know how that happened, and I don't see anything obvious in your config that would trigger this though. I'll be sure to add a check for this in mike so that empty version names/aliases trigger an error.

[heitorlessa]

Haha no worries, I do the same thing on my repos too. I can explain the empty string - I was testing how I could use Mike to build and deploy separately, so the first attempt was to use —update-aliases with two variables where the last could be empty. Later I saw I couldn’t separate build from deployment - I have a need to use Mike + include an API folder that gets generated separately. that versions.json is only updated when I use —push after looking at the source code, but I never cleaned that up as I kept trying multiple ways before I cut an issue out ;)

…

On Mon, 29 Mar 2021 at 19:40, Jim Porter ***@***.***> wrote: Ah, so you did. I should probably make a habit of having my coffee *first* and *then* replying to issues. :) Ok, I think I figured it out... kind of. Somehow, the develop branch is getting an alias of "", i.e. the empty string. That gets splattered all over the root directory of your gh-pages branch and clobbers everything else. I don't know how that happened, and I don't see anything obvious in your config that would trigger this though. I'll be sure to add a check for this in mike so that empty version names/aliases trigger an error. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#49 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZPQBDUOP3HISJBXEHMJGDTGC3PPANCNFSM4Z7INARQ> .

[jimporter]

Cool, I'm glad the random empty string was a bit less mysterious than I'd feared!

Once the tests pass and I merge it, ab52395 (updated hash) will raise an exception if it sees an empty version/alias, since it's a pretty disastrous failure. Fixing the invocation of mike deploy isn't enough: you have to delete the bad alias from versions.json too, or the problem will keep happening as mike tries to keep the "" alias up-to-date.

I'll close this now since it sounds like you just need to tweak your GH action, but if there are still problems, just let me know and I can reopen the issue.

[heitorlessa]

Gotcha! I’ll nuke everything, and start over as I already have a fetch depth 0 that in theory should fetch all branches and commits. If that doesn’t work I’ll try one last thing by forcefully using git commands ;)

…

On Mon, 29 Mar 2021 at 20:04, Jim Porter ***@***.***> wrote: Closed #49 <#49>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#49 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZPQBHB2RX2MO3IGBLCQX3TGC6KRANCNFSM4Z7INARQ> .

[jimporter]

Oh I forgot to mention: generally, you don't need to run mike set-default during your CI. It's something you can set up initially beforehand and then not have to touch it, since it generally won't change. If the default version is latest, then that'll always point to the latest release.

Post-1.0, I'm thinking I might change this around so the latest version is specified in the mike plugin config in mkdocs.yml. That way, there's one fewer subcommand to worry about, and mike can keep the root index.html up-to-date no matter what.

[heitorlessa]

I had challenge with that actually, as redirects stopped working without it on subsequent deploys. But to be honest I tried so many things before I cut an issue as I thought it was something silly, but I now suspect that empty string had side effects. The plug-in extra key would be great indeed I’ll update tomorrow ;-)

…

On Mon, 29 Mar 2021 at 20:23, Jim Porter ***@***.***> wrote: Oh I forgot to mention: generally, you don't need to run mike set-default during your CI. It's something you can set up initially beforehand and then not have to touch it, since it generally won't change. If the default version is latest, then that'll always point to the latest release. Post-1.0, I'm thinking I might change this around so the latest version is specified in the mike plugin config in mkdocs.yml. That way, there's one fewer subcommand to worry about, and mike can keep the root index.html up-to-date no matter what. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#49 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZPQBEJK477Y5GNJIGO34DTGDASJANCNFSM4Z7INARQ> .

[jimporter]

The index.html redirect file was probably getting clobbered by the "" alias. Just another example of how that error can really mess things up. :)

[heitorlessa]

WORKS!!! By always using an alias, develop [version] -> stage [alias], this works beautifully. There is only a glitch of redirects not kicking in at first due to GH CDN caching I reckon but overall all new releases & develop commits are working just fine now.

I'd be happy to contribute with a sample workflow if you see there's a place here, as I'm sure it'll be a common one. Alternatively, I could contribute to mkdocs-material docs to have that as an example

[jimporter]

Great! If you have the config available for me to look at, I can adapt that into some docs. I have a couple of ideas for ways to improve the configuration besides just doing fetch-depth: 0, so I want to try those out before I publish an "official" way of doing it, but a starting config would definitely make things simpler for me!

[prcr]

Chiming in just to say that I was actually exploiting the fact that the version/alias could be "." to deploy the "Latest" version of the documentation directly to the root of the gh-pages branch. 🤓 😅

https://github.com/codacy/docs/blob/master/.github/workflows/mkdocs.yml#L143
https://github.com/codacy/docs/tree/gh-pages

I can't update to mike 1.0.0 because of this, as I would need to reconsider the strategy that I'm using to deploy the documentation.

[jimporter]

@prcr Hmm, I'm not sure how you managed to avoid clobbering all the old versions; maybe Git was treating "." slightly differently from "" and it didn't remove the old files. (That's a bug too, since it means that stale docs can stick around forever if you move things around.)

You could partially solve your issue by moving everything to example.com/latest (or /vX.Y.Z) and then creating a custom 404 page that redirects users to the correct page. You'd just need to take window.location and insert "/latest" in the correct spot. It would take a bit of cleverness to ensure that this doesn't put users in an infinite redirection loop, though. This is the solution I'd prefer, and I've already been looking into making a 404 page that does something like this as a new mike feature.

Another option would be to simply add a step after mike deploy where you copy the files from /latest (or wherever) to the root directory. Something like this:

git checkout gh-pages
cp -r latest .
git add .
git commit -m "message"
git push

This might cause issues with the site_url attribute, but it's worth a try at least...

I think having doc versions as directory siblings is the only safe way to avoid path collisions or other issues with files getting clobbered, so I don't think I'd want to add official support for doing things the way you were. It's just too prone to breakage.

Edit: I suppose if no other solution works, it would theoretically be ok to allow deploying to the root directory via a special command-line flag. That way it's impossible to do it accidentally, and I could add special handling to make sure only the appropriate files are deleted. This would take quite a bit of work though, since mike would need to keep track of what files to keep around at the root level.

[prcr]

Thanks for your detailed answer and proposed workarounds @jimporter. Yes, at the time I set up things this way I was already aware this was a "hack" and wasn't the recommended way of using mike. I tested with "" first and that didn't do the trick, but I used "." to trick mike. 😉

I'm actually liking the suggestion of moving the files to the root myself, as it will make more obvious what's happening and would keep mike simple. 👍 I could even use this workaround to explicitly solve the issue of stale files piling up.

Doing this refactoring is not a high priority for me at the moment, but I'll let you know the solution that I eventually settle down with.

[jimporter]

Oh, now that I think about it, another wrinkle with moving files to the root dir manually is that you'll have to make sure the JS for the version selector can find versions.json. That's probably not too hard to fix, but it might require editing the contents of the files slightly. Perhaps it would make sense to add a patch or two to the version selector code to make this easier, but I haven't thought through all the implications of moving files around like this...

[prcr]

I'm already using some custom logic to generate the version selector drop-down, so this shouldn't be a big change for my particular scenario:

https://github.com/codacy/docs/blob/master/docs/assets/javascripts/version-select.js#L90

But indeed it's some extra complexity that mike could probably do without! 🙂

[airtonix]

So can we TL;DR the problem and solution please ?

[JPHutchins]

Thanks for the example and discussion! Here's what I did.

Initialize the gh-pages branch to forward to latest:

mike set-default latest --allow-undefined

Push it

git push origin gh-pages

Release workflow excerpt:

  publish-docs:
    name: Publish documentation 📚 to GitHub Pages
    needs:
      - github-release
    runs-on: ubuntu-latest

    permissions:
      contents: write # IMPORTANT: mandatory for deploying to GitHub Pages

    steps:
      - uses: actions/checkout@v4

      - run: git fetch --prune --unshallow --tags

      - run: pipx install poetry

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.x"
          cache: "poetry"

      - run: poetry install --only doc

      - name: Configure git for gh-pages
        run: |
          git config --global user.name "SMP Docs Bot"
          git config --global user.email "docs@dummy.bot.com"

      - name: Set release version
        run: echo "GIT_TAG=${{ github.event.release.tag_name }}" >> $GITHUB_ENV

      - name: Build and deploy documentation
        run: poetry run mike deploy --push --update-aliases ${GIT_TAG} latest

It's live! Link to repo for reference: https://github.com/JPHutchins/smp