Caching Jekyll-deploy-action Docker

[datapolitical]

Now that I've got my build time under 6 minutes the next big time sink is the 2+ minutes it takes to build the action. Caching seems to be possible using this: https://github.com/docker/build-push-action/issues but it's well outside my depth.

Figured I would pass it along in case you had thoughts.

[datapolitical]

Bumping this as the action currently takes 2.5 minutes to build which seems very long.

[jeffreytse]

Hi @datapolitical,

This is a build based on docker, we can not cache this image for speeding up it, the large size of caching would exceed the limitation the GitHub allowed. I will try to investigate another way to speed up it.

Thanks and regards.

[datapolitical]

Maybe this will help: https://github.com/marketplace/actions/build-docker-images-using-cache

[jeffreytse]

Thanks for the help. : D

[datapolitical]

Does this seem like a good lead? I could to set it up myself but I’d have to do it on a fork I think.

[jeffreytse]

I think you can fork this project and have a try, and welcome to give me a feedback. : D

[darioblanco]

@datapolitical with the new setup-ruby action you can improve the speed greatly. I could reduce my workflow from 4-5 minutes to ~30 seconds thanks to this:

name: Deploy to Github Pages

on:
  push:
    branches:
      - main
    paths:
      - '**.md'
      - '.github/workflows/jekyll.yml'
      - '_config.yml'
      - 'CNAME'
      - 'Gemfile'
  workflow_dispatch:

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout main branch
        uses: actions/checkout@v2
      - name: Checkout gh-pages branch
        uses: actions/checkout@v2
        with:
          path: gh-pages
          ref: gh-pages
      - name: Setup ruby
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true
      - name: Install Jekyll dependencies
        run: bundle
      - name: Build jekyll site
        run: JEKYLL_ENV=production bundle exec jekyll build -c _config.yml -d ./build
      - name: Generate build into gh-pages checkout
        run: |
          cd ./build
          touch .nojekyll
          echo "my.website" > CNAME
          cp -r . ${{ github.workspace }}/gh-pages
      - name: Commit jekyll build changes
        run: |
          cd ${{ github.workspace }}/gh-pages
          git config --global user.email "myemail"
          git config --global user.name "myuser"
          git add .
          git commit -m "ci: jekyll build from action ${GITHUB_SHA}"
      - name: Push changes to gh-pages
        uses: ad-m/github-push-action@v0.6.0
        with:
          directory: gh-pages
          github_token: ${{ secrets.YOUR_GITHUB_PAT }}
          branch: gh-pages
          force: true

However, this approach does not use the jekyll-deploy-action at all, but if the speed improvement is important and you do not care handling a bit more for yourself, it is a good alternative.

[jeffreytse]

Hi @darioblanco,thanks for the suggestion and I think now this one has helped to @datapolitical .

[datapolitical]

Yeah this is great I'm gonna have to experiment to figure out if it's compatible with what I've been doing and isn't going to cause any problems but it definitely looks interesting and thankfully I have a test setup I can use to verify that it works OK.

[jeffreytse]

@datapolitical, so sorry for no settling this issue, I will be happy to have a research on this no docker solution and try to refactor jekyll-deploy-action, so that it can be as fast as you expected.

[datapolitical]

Oh awesome

[jeffreytse]

Hi @datapolitical

Firstly, thanks for your patience, now there is a good news to tell you, the running time of the action is mainly impacted by recompiling the gem native extensions every time. To solve this issue, we should cache the gems as well as their native extensions completely, the latest commit f9e971d has settled it. With this enhancement, the speed has been improved almost over 70%.

Before (3m5s): https://github.com/jeffreytse/jekyll-deploy-action/actions/runs/1142065563
After (50s): https://github.com/jeffreytse/jekyll-deploy-action/actions/runs/1142144941

Thanks and Regards

[jeffreytse]

@datapolitical Now the new version v0.3.0 has been released, you can just use it by jeffreytse/jekyll-deploy-action@v0.3.0.

[datapolitical]

This update is causing my build to break with the following error:

Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
285

286
current directory:
287
/github/workspace/vendor/bundle/ruby/2.7.0/gems/rmagick-4.2.2/ext/RMagick
288
/usr/bin/ruby-2.7 -I /usr/lib/ruby/2.7.0 -r ./siteconf20210820-214-112vgkl.rb
289
extconf.rb
290
checking for brew... no
291
checking for pacman... yes
292
checking for Ruby version >= 2.3.0... yes
293
checking for pkg-config... yes
294
Package MagickCore was not found in the pkg-config search path.
295
Perhaps you should add the directory containing `MagickCore.pc'
296
to the PKG_CONFIG_PATH environment variable
297
Package 'MagickCore', required by 'virtual:world', not found
298

299

300
ERROR: Can't install RMagick 4.2.2.
301
Can't find the ImageMagick library or one of the dependent libraries.
302
Check the mkmf.log file for more detailed information.

[jeffreytse]

Hi @datapolitical

With the new release, you should manually install additional dependencies according to your gems by the pre_build_commands option. According to your gem RMagick, below is the code snippet for you.

# Use GitHub Deploy Action to build and deploy to Github
jobs:
  github-pages:
    runs-on: ubuntu-latest
    steps:
      - uses: jeffreytse/jekyll-deploy-action@v0.3.0
        with:
          provider: 'github'
          token: ${{ secrets.GH_TOKEN }} # It's your Personal Access Token(PAT)
          repository: ''             # Default is current repository
          branch: 'gh-pages'         # Default is gh-pages for github provider
          jekyll_src: './'           # Default is root directory
          jekyll_cfg: '_config.yml'  # Default is _config.yml
          jekyll_baseurl: ''         # Default is according to _config.yml
          bundler_ver: '>=0'         # Default is latest bundler version
          cname: ''                  # Default is to not use a cname
          actor: ''                  # Default is the GITHUB_ACTOR
          pre_build_commands: 'pacman -S --noconfirm imagemagick'     # Installing additional dependencies (Arch Linux)

Thanks and Regards

[datapolitical]

I'm using Ubuntu 20.04, is the command different?

[jeffreytse]

@datapolitical Since the action is based on Arch Linux, you should use pre-build commands based on Arch Linux, the package manager on Arch Linux is Pacman.

[datapolitical]

Something is wrong with the imagemagick install:

Liquid Exception: magick convert -resize 400x -quality 75 /tmp/mini_magick20210820-1332-1sdhtie.jpg[0] /tmp/mini_magick20210820-1332-5ugwfz.webp failed with error: convert: delegate failed 'cwebp' -quiet %Q '%i' -o '%o'' @ error/delegate.c/InvokeDelegate/1913. in /github/workspace/vendor/bundle/ruby/2.7.0/bundler/gems/contrast-76b6aa921e7b/_layouts/recipes.html 397 ------------------------------------------------ 398 Jekyll 4.2.0 Please append --traceto thebuildcommand 399 for any additional information or backtrace. 400 ------------------------------------------------ 401 /github/workspace/vendor/bundle/ruby/2.7.0/gems/mini_magick-4.11.0/lib/mini_magick/shell.rb:17:inrun': magick convert -resize 400x -quality 75 /tmp/mini_magick20210820-1332-1sdhtie.jpg[0] /tmp/mini_magick20210820-1332-5ugwfz.webp failed with error: (MiniMagick::Error)
402
convert: delegate failed `'cwebp' -quiet %Q '%i' -o '%o'' @ error/delegate.c/InvokeDelegate/1913.

[jeffreytse]

@datapolitical Accordingly, additional dependency libwebp you should install.

[jeffreytse]

@datapolitical For better user experience, I will add a new feature to automatically checking gems used and installing relative system dependencies. For example, if you use RMagick gem, it will auto install packages imagemagick and libwebp.