Skip to content

Commit

Permalink
chore(CI/CD): bump version 0.7.3 -> 0.8.0 (#102)
Browse files Browse the repository at this point in the history
* chore(CI/CD): bump version 0.7.3 -> 0.8.0

* docs: update CHANGELOG.md

* chore: simplify tests and bump minimal libraries versions

* chore: change python version for smallest dependencies

* ci: disable pre-commit for manual tests

* chore: bump pytest-xdist

* chore: bump geopandas version

* chore: bump typeguard version

* chore: bump pytest version

* ci: remove doctests from oldest dependencies test

* chore: bump pytest-mock version

* chore: bump tqdm version

* chore: bump s2 version

* chore: remove h3ronpy from dependencies

* chore: bump osmnx version

* fix: change h3 to polygon transformation

* chore: bump psutil version

* chore: bump polars version

* chore: bump pyarrow and pooch version

* chore: change readme

* chore: change changelog

---------

Co-authored-by: Kamil Raczycki <raczyckikamil@gmail.com>
  • Loading branch information
kraina-cicd and RaczeQ committed May 9, 2024
1 parent 8ecaea0 commit 271d671
Show file tree
Hide file tree
Showing 7 changed files with 63 additions and 68 deletions.
29 changes: 6 additions & 23 deletions .github/workflows/manual_tests.yml
Expand Up @@ -8,15 +8,8 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
python-version: ["3.9", "3.10", "3.11", "3.12"]
include:
- os: macos-latest
python-version: "3.12"
- os: macos-13
python-version: "3.12"
- os: windows-latest
python-version: "3.12"
os: [ubuntu-latest, macos-13, macos-latest, windows-latest]
python-version: ["3.12"]
env:
OS: ${{ matrix.os }}
PYTHON: ${{ matrix.python-version }}
Expand All @@ -31,29 +24,21 @@ jobs:
- name: Generate lock with newest dependencies
run: pdm lock --lockfile pdm.newest.lock --strategy no_cross_platform -dG:all
- name: Install quackosm and tests dependencies
run: pdm install --lockfile pdm.newest.lock -dG:all
run: pdm install --lockfile pdm.newest.lock -dG:all --skip=post_install
- name: Run tests with pytest
run: |
pdm run pytest -v -s --durations=20 --doctest-modules --doctest-continue-on-failure quackosm
pdm run pytest -v -s --durations=20 tests/base
pdm run pytest -v -s --durations=20 tests/optional_imports
pdm run pytest -v -s --durations=20 tests/benchmark
run-tests-oldest:
name: Run tests 🛠️ on multiple systems 🖥️ and Python 🐍 versions (oldest dependencies)
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
python-version: ["3.9", "3.10", "3.11", "3.12"]
include:
- os: macos-latest
python-version: "3.12"
- os: macos-13
python-version: "3.12"
- os: windows-latest
python-version: "3.12"
os: [ubuntu-latest, macos-13, macos-latest, windows-latest]
python-version: ["3.9"]
env:
OS: ${{ matrix.os }}
PYTHON: ${{ matrix.python-version }}
Expand All @@ -68,10 +53,8 @@ jobs:
- name: Generate lock with oldest dependencies
run: pdm lock --lockfile pdm.oldest.lock --strategy no_cross_platform,direct_minimal_versions -dG:all
- name: Install srai and tests dependencies
run: pdm install --lockfile pdm.oldest.lock -dG:all
run: pdm install --lockfile pdm.oldest.lock -dG:all --skip=post_install
- name: Run tests with pytest
run: |
pdm run pytest -v -s --durations=20 --doctest-modules --doctest-continue-on-failure quackosm
pdm run pytest -v -s --durations=20 tests/base
pdm run pytest -v -s --durations=20 tests/optional_imports
pdm run pytest -v -s --durations=20 tests/benchmark
14 changes: 12 additions & 2 deletions CHANGELOG.md
Expand Up @@ -7,6 +7,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.8.0] - 2024-05-08

### Added

- Polars library to the main dependencies
Expand All @@ -16,14 +18,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Refactored ways grouping logic from duckdb to polars `LazyFrame` API for faster operations
- Default result file extension from `geoparquet` to `parquet` [#99](https://github.com/kraina-ai/quackosm/issues/99)
- Moved `rich` to the main dependencies [#95](https://github.com/kraina-ai/quackosm/issues/95)
- Set minimal versions of multiple dependencies
- Added tests for minimal dependencies versions

### Fixed

- Steps numbering after encountering `MemoryError`

### Removed

- `h3ronpy` from dependencies and replaced logic with pure `h3` calls

### Deprecated

- Usage of `geoparquet` files as default extension, reusing existing `geoparquet` files from cache will be supported with deprecation warning [#99](https://github.com/kraina-ai/quackosm/issues/99)
- Reusing existing `geoparquet` files from cache will be supported, but will result in deprecation warning [#99](https://github.com/kraina-ai/quackosm/issues/99)

## [0.7.3] - 2024-05-07

Expand Down Expand Up @@ -254,7 +262,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Created QuackOSM repository
- Implemented PbfFileReader

[Unreleased]: https://github.com/kraina-ai/quackosm/compare/0.7.3...HEAD
[Unreleased]: https://github.com/kraina-ai/quackosm/compare/0.8.0...HEAD

[0.8.0]: https://github.com/kraina-ai/quackosm/compare/0.7.3...0.8.0

[0.7.3]: https://github.com/kraina-ai/quackosm/compare/0.7.2...0.7.3

Expand Down
22 changes: 11 additions & 11 deletions README.md
Expand Up @@ -72,23 +72,25 @@ Required:

- `duckdb (>=0.10.2)`: For all DuckDB operations on PBF files

- `pyarrow (>=13.0.0)`: For parquet files wrangling
- `pyarrow (>=14.0.0)`: For parquet files wrangling

- `pyarrow-ops`: For easy removal of duplicated features in parquet files

- `geoarrow-pyarrow (>=0.1.2)`: For GeoParquet IO operations

- `geopandas`: For returning GeoDataFrames and reading Geo files
- `geopandas (>=0.6)`: For returning GeoDataFrames and reading Geo files

- `shapely (>=2.0)`: For parsing WKT and GeoJSON strings and fixing geometries

- `typeguard`: For internal validation of types
- `polars (>=0.19.4)`: For faster OSM ways grouping operation

- `psutil`: For automatic scaling of parameters based on available resources
- `typeguard (>=3.0)`: For internal validation of types

- `pooch`: For downloading `*.osm.pbf` files
- `psutil (>=5.6.2)`: For automatic scaling of parameters based on available resources

- `tqdm`: For showing progress bars
- `pooch (>=1.6.0)`: For downloading `*.osm.pbf` files

- `rich (>=10.11.0)` & `tqdm (>=4.42.0)`: For showing progress bars

- `requests`: For iterating OSM PBF files services

Expand All @@ -99,15 +101,13 @@ Optional:

- `typer[all] (>=0.9.0)` (click, colorama, rich, shellingham): For CLI

- `osmnx`: For geocoding of strings in CLI
- `osmnx (>=1.3.0)`: For geocoding of strings in CLI

- `h3 (>=4.0.0b1)`: For reading H3 strings in CLI

- `h3ronpy (>=0.18.0)`: For transforming H3 indexes into geometries

- `s2`: For transforming S2 indexes into geometries
- `s2 (>=0.1.9)`: For transforming S2 indexes into geometries

- `python-geohash`: For transforming GeoHash indexes into geometries
- `python-geohash (>=0.8)`: For transforming GeoHash indexes into geometries

## Usage

Expand Down
16 changes: 8 additions & 8 deletions pdm.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

35 changes: 17 additions & 18 deletions pyproject.toml
@@ -1,22 +1,22 @@
[project]
name = "QuackOSM"
version = "0.7.3"
version = "0.8.0"
description = "An open-source tool for reading OpenStreetMap PBF files using DuckDB"
authors = [{ name = "Kamil Raczycki", email = "kraczycki@kraina.ai" }]
dependencies = [
"geopandas",
"geopandas>=0.6",
"shapely>=2",
"pyarrow>=13.0.0",
"pyarrow>=14.0.0",
"duckdb>=0.10.2",
"geoarrow-pyarrow>=0.1.2",
"typeguard",
"psutil",
"pooch",
"tqdm",
"typeguard>=3.0.0",
"psutil>=5.6.2",
"pooch>=1.6.0",
"tqdm>=4.42.0",
"beautifulsoup4",
"pyarrow-ops",
"requests",
"polars",
"polars>=0.19.4",
"rich>=10.11.0",
]
requires-python = ">=3.9"
Expand Down Expand Up @@ -48,10 +48,9 @@ Changelog = "https://github.com/kraina-ai/quackosm/blob/main/CHANGELOG.md"
[project.optional-dependencies]
cli = [
"typer[all]>=0.9.0",
"osmnx",
"osmnx>=1.3.0",
"h3>=4.0.0b1",
"h3ronpy>=0.18.0",
"s2",
"s2>=0.1.9",
"python-geohash>=0.8",
]

Expand All @@ -73,16 +72,16 @@ dev = [
"setuptools>=45.0.0",
]
# pdm add -dG lint <library>
lint = ["pre-commit", "mypy", "docformatter[tomli]", "ruff"]
lint = ["pre-commit", "mypy>=1", "docformatter[tomli]", "ruff>=0.1.0"]
# pdm add -dG test <library>
test = [
"pytest",
"pytest>=7.0.0",
"tox-pdm",
"pytest-mock",
"pytest-mock>=3.3.0",
"requests-mock",
"pytest-check",
"pytest-parametrization",
"pytest-xdist",
"pytest-xdist>=3.4.0",
"pytest-doctestplus",
"srai>=0.6.2",
]
Expand All @@ -96,15 +95,15 @@ docs = [
"ipykernel",
"mkdocs-gen-files",
"mkdocs-awesome-pages-plugin",
"mike<2",
"mike>=1,<2",
"black",
"jupyter",
"nbconvert",
"nbformat",
"notebook",
"osmnx",
"matplotlib",
"seaborn",
"seaborn>=0.10.0",
"py-cpuinfo",
"adjustText",
]
Expand Down Expand Up @@ -180,7 +179,7 @@ close-quotes-on-newline = true
wrap-one-line = true

[tool.bumpver]
current_version = "0.7.3"
current_version = "0.8.0"
version_pattern = "MAJOR.MINOR.PATCH[PYTAGNUM]"
commit_message = "chore(CI/CD): bump version {old_version} -> {new_version}"
commit = true
Expand Down
2 changes: 1 addition & 1 deletion quackosm/__init__.py
Expand Up @@ -14,7 +14,7 @@
from quackosm.pbf_file_reader import PbfFileReader

__app_name__ = "QuackOSM"
__version__ = "0.7.3"
__version__ = "0.8.0"

__all__ = [
"PbfFileReader",
Expand Down
13 changes: 8 additions & 5 deletions quackosm/cli.py
Expand Up @@ -14,7 +14,6 @@
from click import Argument
from click.exceptions import MissingParameter
from geohash import bbox as geohash_bbox
from h3ronpy.arrow.vector import cells_to_wkb_polygons
from s2 import s2
from shapely import from_geojson, from_wkt
from shapely.geometry import Polygon, box
Expand Down Expand Up @@ -145,10 +144,14 @@ def convert(self, value, param=None, ctx=None): # type: ignore
return None

try:
h3_int_indexes = [h3.str_to_int(h3_cell.strip()) for h3_cell in value.split(",")]
return gpd.GeoSeries.from_wkb(cells_to_wkb_polygons(h3_int_indexes)).unary_union
except Exception:
raise typer.BadParameter(f"Cannot parse provided H3 values: {value}") from None
geometries = [] # noqa: FURB138
for h3_cell in value.split(","):
geometries.append(
Polygon([coords[::-1] for coords in h3.cell_to_boundary(h3_cell.strip())])
)
return gpd.GeoSeries(geometries).unary_union
except Exception as ex:
raise typer.BadParameter(f"Cannot parse provided H3 values: {value}") from ex


class S2GeometryParser(click.ParamType): # type: ignore
Expand Down

0 comments on commit 271d671

Please sign in to comment.