Jack McKew's Blog
jackmckew.dev Powered by Pelican + Travis CI + Netlify
What tipped me over the edge was when I was looking back on previous posts (as this blog is mainly for storing previous projects, concepts and ideas), I was finding that I would go through the post and then have no idea whatsoever on where the project actually lived, this had to be fixed.
I started noticing more and more people online had moved to Github Pages, which is primarily used with Jekyll. This rabbit hole went on as follows:
- Static served websites (generate HTML pages and serve them)
- Numerous static site generators:
- So on
- Blogging with Jupyter notebooks
- Concept of CI/CD
- Travis CI
What I settled on was a bit of a concoction of services, such that I can both get my feet wet with these new tools and still stay in the land of snakes (Python).
Pelican + Travis CI + Netlify + Github
Before we get into all 4 services in conjunction, let's separate and step through the process for each of them.
Right off the bat, the first milestone I wanted to hit was to be able to generate a locally hosted static site from a single post converted to markdown. Luckily, there is an exact guide for going through this process in the documentation for Pelican and using the tool pelican-quickstart.
The next step was to decide on a theme for the website, while the intentions were to develop a theme from scratch, I shall leave this for a later date. An easy way of previewing themes was the website:
Which lets you scroll through the various themes, and even links to the repository on github for the theme if you wish to use it. The theme I decided on was Flex by Alexandre Vicenzi.
Apply the the theme was as simple as cloning the repo (or using git submodules), and adding one line of code in pelicanconf.py (generated automatically by pelican-quickstart).
THEME = "./themes/Flex"
Admittedly, I just tried out all the plugins in the Pelican Plugins Repository until I found the combination that works for me, this ended up being:
PLUGINS = [ "sitemap", "better_codeblock_line_numbering", "better_code_samples", "bootstrapify", "deadlinks", "more_categories", "neighbors", "pelican-ert", "liquid_tags.notebook", "liquid_tags.include_code", "representative_image", "share_post", 'show_source', 'tipue_search', "dateish", "post_stats", "render_math", "autostatic", "clean_summary" ]
For tipue_search in particular, as this wasn't supported by the theme yet, I created a pull request on the original repository, with the functionality integrated https://github.com/alexandrevicenzi/Flex/pull/193.
Now that I had the skeleton of the website set up, I needed to bring in all the existing posts from wordpress. By following another guide within the Pelican documentation, this was a relatively simple task http://docs.getpelican.com/en/3.6.3/importer.html. However, I did spend the time to go through and edit each markdown to remove redundant 'wordpress' formatting tags manually.
Linking to Content
As one of the main tasks of this project was to consolidate articles with the content/code/analysis in one spot, initially in development following the guide in http://docs.getpelican.com/en/3.6.3/content.html.
content ├── articles │ └── article.md ├── images │ └── han.jpg ├── pdfs │ └── menu.pdf └── pages └── test.md
I ended up with a structure like above, which annoyed me a bit as now the content was in one place, but still divided into 3 folders with little-to-no link between them, my goal was to have the structure like:
content ├── articles │ ├── test-article │ | ├── img │ │ | └── icon.png │ │ | └── photo.jpg │ | ├── notebooks │ │ | └── test-notebook.ipynb │ │ └── article.md └── files └── archive.zip
To be honest, I was actually surprised at how easy it was to turn Travis CI and that I could spin up a virtual machine, install all the dependencies and re-build the website. However, I had a lot of trouble trying to get Travis CI to push back to the repository such that Netlify could build from it.
This was later remedied by setting a repository secret variable on Travis CI as I couldn't get the secret token encrypted by Travis CI CLI (Ruby application).
In essence, all that was needed was a .travis.yml file in the root directory which ended up like this:
language: python branches: only: - master install: - pip install --upgrade pip - pip install -r requirements.txt script: - pelican content deploy: provider: pages skip_cleanup: true github_token: $GITHUB_TOKEN keep_history: true local_dir: output on: branch: master
Admittedly, I feel as if I'm not using Netlify for all it can do.
Essentially, all for this project, it just detects a change in the gh-pages branch (for Github Pages), and redeploys the website out to a custom domain.
Github is the repository location for all the code, and I use Git for version control and interaction with the repository.
All I need to do now to create a new post is:
- Push a new markdown file (and any other linked content) to the master branch of the repository,
- This will fire up Travis CI to build the site with Pelican for me,
- Travis CI will then push the created site to the gh-pages branch of the repository,
- Netlify will detect the change and process the new site,
- The new site is deployed with updated posts!