Note
Since writing this post, I moved to using Ruff for Python linting.
Intro
Github Actions test and lint all code changes before they are pushed into a branch. You can see this ‘in-action' on the actions tab of the blogthedata repo. Actions can be added to any repo by placing a .yaml into the root directory.
name: Blogthedata Tests
on:
push:
pull_request:
branches:
- main
jobs:
build:
runs-on: ubuntu-latest
env:
SECRET_KEY: ${{secrets.SECRET_KEY}}
steps:
- uses: actions/checkout@v2
- name: Setup Python 3.9.7
uses: actions/setup-python@v2
with:
python-version: 3.9.7
- name: Install dependencies
run: |
pip install --upgrade pip
pip install wheel
pip install -r config/requirements.txt
- name: Lint with Flake8
run: |
flake8 django_project
- name: Coverage + run unit tests
run: |
coverage run -m pytest django_project
coverage report -m --skip-empty --skip-covered
Lines 2 - 6 - Specify the action trigger. I run the workflow whenever I attempt a push or PR with master as the target branch.
Lines 7 - 9 - Spin up an Ubuntu container
Lines 10 - 11 Set environment variables to be used by the container. These contain things like Django's SECRET_KEY and any other sensitive tokens. When you use the {{secrets.<name>}} syntax, you’re accessing encrypted secrets set within your repo's settings page. This actually forced me to change my implementation because I was loading secrets with a configuration file not committed to source control. I switched to environment variables because that appeared to be easier and safer to use with Github Actions.
Line 13 - Check out code from the blogthedata repo
Lines 14 - 22 - Install Python and its dependencies within the Ubuntu container created on line 9
Lines 23 - 25 - Lint the code using flake8. Flake8’s configuration is stored in a .flake8 file. I exclude the Python virtual environment (venv), and db migrations from being linted. On the ignore line, I tell Flake8 to disregard E501, Line too long
[flake8]
exclude = venv migrations
ignore = E501
Lines 26 - 29 - Run code coverage using the coverage module and unit tests with PyTest. I store the configuration in a .coveragerc file. The configuration loads the django-coverage plugin which I talk more about in my post about how to get perfect PEP 8 compliance. It also excludes test files from coverage..it's a little overkill to have unit tests for my unit tests!
[run]
plugins = django_coverage_plugin
omit =
*/tests/*
Conclusion
The whole process takes about 2 minutes from start to finish. One way I could speed this up is by implementing mocks and stubs instead of relying on a database to complete unit tests.
A week ago, I had never used Github Actions and was simply trying to close this issue. My next step is to develop a CD (Continuous Deployment) workflow where production updates without any manual intervention. Today, I perform these three steps to update prod.
- SSH into the production server
- Perform a git pull to get the latest code
- Restart Apache2 web server
It's not cumbersome, but it's good practice to automate, as anything done manually is prone to human error. You can track blogthedata.com CD progress in this issue.
Add a CI workflow to your project, today!
![John Solly Profile Picture](/static/jsolly.98b1e6cb9051.webp)
John Solly
Hi, I'm John, a Software Engineer with a decade of experience building, deploying, and maintaining cloud-native geospatial solutions. I currently serve as a senior software engineer at HazardHub (A Guidewire Offering), where I work on a variety of infrastructure and application development projects.
Throughout my career, I've built applications on platforms like Esri and Mapbox while also leveraging open-source GIS technologies such as OpenLayers, GeoServer, and GDAL. This blog is where I share useful articles with the GeoDev community. Check out my portfolio to see my latest work!
Comments