Azure Synapse Notebook Tests

This repository contains 2 workflows:

• Notebook Linting & Unit Tests (unit-tests.yml): runs pylint and unit tests for the notebooks in the workspace/notebooks folder
• Synapse Workspace Deployment (workspace-deployment.yml): creates the templates for the Synapse workspace.

Notebook Linting & Unit Tests

Prerequisites:

• nbQA: Run isort, pyupgrade, mypy, pylint, flake8, black, blacken-docs, and more on Jupyter Notebooks
• testbook: Unit test your Jupyter Notebooks.

How it works

The workflow usese github actions to preform the following steps:

• Checks out the code.
• Installs nbQA & testbook.
• Extracts and fixes the ipynb notebooks from the workspace/notebooks folder. Resulting files are stored in the ipynb_to_test folder.
• Using nbQA runs pylint on the ipynb_to_test folder.
• Using testbook run pytest on the ipynb_to_test folder.

Synapse Workspace Deployment

How it works

The workflow uses github actions to preform the following steps:

• Checks out the code.
• Runs the validate operation of the Azure/Synapse-workspace-deployment action.
• Uploads the template files created in th ExportedArtifacts folder as artifacts.

References

• Automating the Publishing of Workspace Artifacts in Synapse CICD
• Continuous integration and delivery for an Azure Synapse Analytics workspace
• Azure Synapse Workspace Deployment
• nbQA
• testbook