Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unit tests #35

Merged
merged 46 commits into from
Mar 27, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
9c6f009
Remove need for virtualenv package
GjjvdBurg Feb 23, 2021
5bc4352
Remove Python 2 remnants
GjjvdBurg Feb 23, 2021
5a29ef7
code formatting
GjjvdBurg Feb 23, 2021
ffcda4c
Add unit tests for Photo class
GjjvdBurg Feb 23, 2021
de968b0
Add preliminary github actions workflow
GjjvdBurg Feb 24, 2021
d4c4871
Add unit testing data
GjjvdBurg Feb 24, 2021
b98c25d
Don't use venv on CI
GjjvdBurg Feb 24, 2021
c5924c7
fix leftover from removing python 2
GjjvdBurg Feb 24, 2021
fbc63d5
Deal with memory errors by closing open images
GjjvdBurg Feb 24, 2021
49da0a7
code formatting
GjjvdBurg Feb 24, 2021
0b28046
Use python for running unit tests on CI
GjjvdBurg Feb 24, 2021
aa895f9
remove old test image
GjjvdBurg Feb 24, 2021
52e303a
Add unit tests for Album
sebastiaan-lampo Feb 23, 2021
4c8d190
Merge branch 'album-test' of https://github.com/sebastiaan-lampo/Hugo…
GjjvdBurg Mar 6, 2021
16cd8fd
Merge branch 'sebastiaan-lampo-album-test' into feature/unit_tests
GjjvdBurg Mar 6, 2021
348b293
Remove __init__ file from test dir
GjjvdBurg Mar 20, 2021
1bdffe2
Clean up yaml writing function
GjjvdBurg Mar 20, 2021
e3e2e04
Handle memory free better
GjjvdBurg Mar 20, 2021
567af9d
Change hash algorithm to sha256
GjjvdBurg Mar 20, 2021
35b68c1
Code formatting and minor fixes
GjjvdBurg Mar 20, 2021
23e4925
Add unit tests for album class
GjjvdBurg Mar 20, 2021
178b5cd
Handle photos without name
GjjvdBurg Mar 20, 2021
a3d8dcf
Add tests for output format
GjjvdBurg Mar 20, 2021
ab31798
Clean up yaml_field_to_file
GjjvdBurg Mar 20, 2021
2ac9aa6
Remove unneeded file
GjjvdBurg Mar 20, 2021
0632793
Close open image
GjjvdBurg Mar 20, 2021
21d2e76
Update test image to remove warning
GjjvdBurg Mar 20, 2021
5b535bc
Debug build failures on windows
GjjvdBurg Mar 20, 2021
eacadff
Update hash
GjjvdBurg Mar 20, 2021
164d752
Makefile fixes
GjjvdBurg Mar 20, 2021
62ec2f7
Use abspath instead of realpath
GjjvdBurg Mar 20, 2021
abc8a3d
Use logging for warnings/errors
GjjvdBurg Mar 20, 2021
01a2dfe
Robustify test
GjjvdBurg Mar 20, 2021
ea30bfd
Rename conf to config
GjjvdBurg Mar 20, 2021
8febef7
Add unit tests for utils
GjjvdBurg Mar 20, 2021
ac85b52
Add tests for config
GjjvdBurg Mar 20, 2021
2edd9d5
Remove unnecessary code
GjjvdBurg Mar 20, 2021
7715b7c
Organise imports
GjjvdBurg Mar 20, 2021
4dc7039
Split update and clean functions
GjjvdBurg Mar 22, 2021
3ed86b0
Simplify init
GjjvdBurg Mar 22, 2021
0f02b98
Code formatting
GjjvdBurg Mar 22, 2021
328c618
Exit cleanly if config file not found
GjjvdBurg Mar 22, 2021
55191db
Unit tests for HugoPhotoSwipe class
GjjvdBurg Mar 22, 2021
6efe0b9
Add coverage target to makefile
GjjvdBurg Mar 23, 2021
f7012df
Add bagde to readme
GjjvdBurg Mar 23, 2021
c534c75
Add pre-commit config for code formatting
GjjvdBurg Mar 23, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
27 changes: 27 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: build

on:
push:
pull_request:
schedule:
- cron: 23 11 * * */14

jobs:
python:
name: Testing HugoPhotoSwipe
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ 'ubuntu-latest', 'macos-latest', 'windows-latest' ]
py: [ '3.6', '3.9' ]
steps:
- name: Install Python ${{ matrix.py }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.py }}

- name: Checkout
uses: actions/checkout@v2

- name: Test
run: make test_direct
6 changes: 6 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
repos:
- repo: https://github.com/psf/black
rev: 20.8b1
hooks:
- id: black
language_version: python3
18 changes: 14 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -46,26 +46,36 @@ dist: ## Make Python source distribution

.PHONY: test

test: venv ## Run nosetests using the default nosetests command
source $(VENV_DIR)/bin/activate && green -a -vv ./tests
test: venv ## Run unit tests in virtual environment
source $(VENV_DIR)/bin/activate && green -a -s 1 -vv ./tests

test_direct: ## Run unit tests without virtual environment (typically for CI)
pip install .[tests] && python -m unittest discover -v ./tests

cover: venv
source $(VENV_DIR)/bin/activate && green -a -r -s 1 -vv ./tests


#######################
# Virtual environment #
#######################

.PHONY: venv

venv: $(VENV_DIR)/bin/activate
venv: $(VENV_DIR)/bin/activate ## Create a virtual environment

$(VENV_DIR)/bin/activate:
test -d $(VENV_DIR) || virtualenv $(VENV_DIR)
test -d $(VENV_DIR) || python -m venv $(VENV_DIR)
source $(VENV_DIR)/bin/activate && pip install -e .[dev]
touch $(VENV_DIR)/bin/activate

############
# Clean up #
############

clean_venv: ## Clean up the virtual environment
rm -rf $(VENV_DIR)

clean: ## Clean build dist and egg directories left after install
rm -rf ./dist
rm -rf ./build
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# HugoPhotoSwipe

[![build](https://github.com/GjjvdBurg/HugoPhotoSwipe/actions/workflows/build.yml/badge.svg)](https://github.com/GjjvdBurg/HugoPhotoSwipe/actions/workflows/build.yml)
[![PyPI version](https://badge.fury.io/py/hugophotoswipe.svg)](https://pypi.org/project/hugophotoswipe)
[![Downloads](https://pepy.tech/badge/hugophotoswipe)](https://pepy.tech/project/hugophotoswipe)

Expand Down
74 changes: 45 additions & 29 deletions hugophotoswipe/album.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,18 @@

"""

from __future__ import print_function

import logging
import os
import shutil
import yaml

from tqdm import tqdm

from .conf import settings
from .config import settings
from .photo import Photo
from .utils import yaml_field_to_file, modtime, question_yes_no, mkdirs
from .utils import modtime
from .utils import question_yes_no
from .utils import yaml_field_to_file


class Album(object):
Expand Down Expand Up @@ -73,8 +73,8 @@ def names_unique(self):
@property
def markdown_file(self):
""" Path of the markdown file """
md_dir = os.path.realpath(settings.markdown_dir)
mkdirs(md_dir)
md_dir = os.path.abspath(settings.markdown_dir)
os.makedirs(md_dir, exist_ok=True)
return os.path.join(md_dir, self.name + ".md")

@property
Expand All @@ -89,10 +89,12 @@ def output_dir(self):
# #
################

def clean(self):
""" Clean up the processed images and the markdown file
def clean(self, force=False):
"""Clean up the processed images and the markdown file

Ask the user for confirmation and only remove if it exists

If ``force = True``, don't ask for confirmation.
"""
output_dir = os.path.join(settings.output_dir, self.name)
have_md = os.path.exists(self.markdown_file)
Expand All @@ -106,7 +108,7 @@ def clean(self):
q += ". Is this okay?"
if (not have_md) and (not have_out):
return
if not question_yes_no(q):
if not force and not question_yes_no(q):
return

if have_md:
Expand Down Expand Up @@ -160,7 +162,7 @@ def create_markdown(self):
fid.write("\n".join(txt))
print("Written markdown file: %s" % self.markdown_file)

def dump(self):
def dump(self, modification_time=None):
""" Save the album configuration to a YAML file """
if self._album_file is None:
raise ValueError("Album file is not defined.")
Expand All @@ -184,8 +186,9 @@ def dump(self):
yaml_field_to_file(
fid, self.creation_time, "creation_time", force_string=True
)
modification_time = modification_time or modtime()
yaml_field_to_file(
fid, modtime(), "modification_time", force_string=True
fid, modification_time, "modification_time", force_string=True
)

fid.write("\n")
Expand All @@ -204,7 +207,9 @@ def dump(self):
for photo in self.photos:
fid.write("\n")
yaml_field_to_file(fid, photo.filename, "file", indent="- ")
yaml_field_to_file(fid, hash(photo), "hash", indent=" ")
yaml_field_to_file(
fid, "sha256:" + photo.sha256sum(), "hash", indent=" "
)
print("Updated album file: %s" % self._album_file)

@classmethod
Expand All @@ -216,8 +221,9 @@ def load(cls, album_dir):
with open(album_file, "r") as fid:
data.update(yaml.safe_load(fid))
else:
print("Skipping non-album directory: %s" % album_dir)
logging.warning("Skipping non-album directory: %s" % album_dir)
return None

album = cls(**data)
album.cover_path = os.path.join(
settings.output_dir, album.name, settings.cover_filename
Expand All @@ -226,12 +232,15 @@ def load(cls, album_dir):
all_photos = []
for p in album.photos:
photo_path = os.path.join(album_dir, settings.photo_dir, p["file"])
caption = "" if p["caption"] is None else p["caption"].strip()
caption = (
"" if p.get("caption", None) is None else p["caption"].strip()
)
alt = "" if p.get("alt", None) is None else p["alt"].strip()
photo = Photo(
album_name=album.name,
original_path=photo_path,
name=p["name"],
alt=p["alt"],
alt=alt,
caption=caption,
copyright=album.copyright,
)
Expand All @@ -240,16 +249,21 @@ def load(cls, album_dir):
album.photos = []
for photo in all_photos:
if photo.name is None:
print("No name defined for photo %r. Using filename." % photo)
continue
logging.warning(
"No name defined for photo %r. Using filename." % photo
)
photo.name = os.path.basename(photo.original_path)
album.photos.append(photo)
return album

def update(self):
def update(self, modification_time=None):
""" Update the processed images and the markdown file """
if not self.names_unique:
print("Photo names for this album aren't unique. Not processing.")
logging.error(
"Photo names for this album aren't unique. Not processing."
)
return

# Make sure the list of photos from the yaml is up to date with
# the photos in the directory, simply add all the new photos to
# self.photos
Expand All @@ -258,13 +272,13 @@ def update(self):
missing = [f for f in os.listdir(photo_dir) if not f in photo_files]
missing.sort()
for f in missing:
pho = Photo(
photo = Photo(
album_name=self.name,
original_path=os.path.join(photo_dir, f),
name=f,
copyright=self.copyright,
)
self.photos.append(pho)
self.photos.append(photo)
logging.info(
"[%s] Found %i photos from yaml and photos dir"
% (self.name, len(self.photos))
Expand Down Expand Up @@ -296,7 +310,7 @@ def update(self):
for photo in self.photos:
hsh = next(
(
h["hash"]
str(h["hash"]).split(":")[-1]
for h in self.hashes
if h["file"] == photo.filename
),
Expand All @@ -305,10 +319,12 @@ def update(self):
photo_hashes[photo] = hsh

to_process = []
for p in self.photos:
if not (p.has_sizes() and (hash(p) == photo_hashes[p])):
to_process.append(p)
del p.original_image
for photo in self.photos:
if not photo.has_sizes():
to_process.append(photo)
elif not photo.sha256sum() == photo_hashes[photo]:
to_process.append(photo)
photo.free()

logging.info(
"[%s] There are %i photos to process."
Expand All @@ -317,20 +333,20 @@ def update(self):
if to_process:
iterator = (
iter(to_process)
if settings.verbose
if not settings.verbose
else tqdm(to_process, desc="Progress")
)
for photo in iterator:
photo.create_sizes()
del photo.original_image
photo.free()

# Overwrite the markdown file
logging.info("[%s] Writing markdown file." % self.name)
self.create_markdown()

# Overwrite the yaml file of the album
logging.info("[%s] Saving album yaml." % self.name)
self.dump()
self.dump(modification_time=modification_time)

####################
# #
Expand Down