| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
| FROM postgis/postgis:10-2.5-alpine | ||
|
|
||
| COPY ./scripts/tune-postgis.sh /docker-entrypoint-initdb.d/tune-postgis.sh |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,139 +1,127 @@ | ||
| # Installation | ||
|
|
||
| This document describes how to manually configure your system for running OpenStreetMap Carto. If you prefer quick, platform independent setup for a development environment, without the need to install and configure tools by hand, follow a Docker installation guide in [DOCKER.md](DOCKER.md). | ||
|
|
||
| ## OpenStreetMap data | ||
| You need OpenStreetMap data loaded into a PostGIS database (see below for [dependencies](#dependencies)). These stylesheets expect a database generated with osm2pgsql using the flex backend. | ||
|
|
||
| Start by creating a database, we are using the database name `gis` here: | ||
|
|
||
| ```sh | ||
| sudo -u postgres createuser -s $USER | ||
| createdb gis | ||
| ``` | ||
|
|
||
| Enable PostGIS and hstore extensions with | ||
|
|
||
| ```sh | ||
| psql -d gis -c 'CREATE EXTENSION postgis; CREATE EXTENSION hstore;' | ||
| ``` | ||
|
|
||
| Then, grab some OSM data; It's probably easiest to grab an PBF of OSM data from [Geofabrik](https://download.geofabrik.de/). Once you've done that, import with osm2pgsql: | ||
|
|
||
| ```sh | ||
| osm2pgsql -O flex -S openstreetmap-carto-flex.lua -d gis ~/path/to/data.osm.pbf | ||
| ``` | ||
|
|
||
| You can find a more detailed guide to setting up a database and loading data with osm2pgsql at [switch2osm.org](https://switch2osm.org/serving-tiles/manually-building-a-tile-server-16-04-2-lts/). | ||
|
|
||
| ### Disable JIT | ||
|
|
||
| We do not recommend [PostgreSQL JIT](https://www.postgresql.org/docs/current/jit-reason.html), which is on by default in PostgreSQL 12 and higher. JIT is benifitial for slow queries where executing the SQL takes substantial time and that time is not spent in function calls. This is not the case for rendering, where most time is spent either fetching from disk, in PostGIS functions, or the query is fast. In theory, the query planner will only use JIT on slower queries, but it is known to get the type of queries map rendering requires wrong. | ||
|
|
||
| Disabling JIT is **essential** for use with Kosmtik and other style development tools. | ||
|
|
||
| JIT can be disabled with `psql -d gis -c 'ALTER SYSTEM SET jit=off;' -c 'SELECT pg_reload_conf();'` or any other means of adjusting the PostgreSQL config. | ||
|
|
||
| ### Custom indexes | ||
| Custom indexes are required for rendering performance and are essential on full planet databases. These are generated by the `scripts/indexes.py` script, see `scripts/indexes.py --help` for various advanced options, but the command below will work to create the indexes on a new installation: | ||
|
|
||
| ```sh | ||
| psql -d gis -f indexes.sql | ||
| ``` | ||
|
|
||
| The indexes can be created in parallel with | ||
|
|
||
| ```sh | ||
| scripts/indexes.py -0 | xargs -0 -P0 -I{} psql -d gis -c "{}" | ||
| ``` | ||
|
|
||
| ### Database functions | ||
| Some functions need to be loaded into the database for current versions. These can be added / re-loaded at any point using: | ||
|
|
||
| ```sh | ||
| psql -d gis -f functions.sql | ||
| ``` | ||
|
|
||
| ## Scripted download | ||
| Some features are rendered using preprocessed shapefiles. | ||
|
|
||
| To download them and import them into the database you can run the following script: | ||
|
|
||
| ```sh | ||
| scripts/get-external-data.py | ||
| ``` | ||
|
|
||
| The script downloads shapefiles, loads them into the database, and sets up the tables for rendering. Additional script options and corresponding documentation is available by invoking `scripts/get-external-data.py --help`. | ||
|
|
||
| ## Fonts | ||
| The stylesheet uses Noto, an openly licensed font family from Google with support for multiple scripts. The stylesheet uses Noto's "Sans" style where available. If not available, this stylesheet uses another appropriate style from the Noto family. The "UI" version is used where available, as its vertical metrics fit better with Latin text. | ||
|
|
||
| Hanazono is used a fallback for seldom used CJK characters that are not covered by Noto. | ||
|
|
||
| For more details, see the documentation at [fonts.mss](style/fonts.mss). | ||
|
|
||
| To download the fonts, run the following script | ||
|
|
||
| ``` | ||
| scripts/get-fonts.sh | ||
| ``` | ||
|
|
||
| ## Dependencies | ||
|
|
||
| To display *any* map, a database containing OpenStreetMap data and some utilities are required: | ||
| * [PostgreSQL](https://www.postgresql.org/) | ||
| * [PostGIS](https://postgis.net/) | ||
| * [`osm2pgsql`](https://github.com/openstreetmap/osm2pgsql#installing)>=`1.8.0` to [import your data](https://switch2osm.org/serving-tiles/updating-as-people-edit-osm2pgsql-replication/) into a PostGIS database | ||
| * [Python 3](https://www.python.org/downloads/) with the `psycopg2`, `pyyaml`, and `requests` libraries (`python3-psycopg2`, `python3-pyyaml`, `python3-requests` packages on Debian-derived systems) | ||
| ```bash | ||
| python3 -m pip install --break-system-packages --user pyyaml requests psycopg2 | ||
| ``` | ||
| * [`ogr2ogr`](https://gdal.org/en/latest/download.html) for loading shapefiles into the database (`gdal-bin` on Debian-derived systems) | ||
|
|
||
| ### For development (style design studio) | ||
|
|
||
| * [Kosmtik](https://github.com/kosmtik/kosmtik) | ||
| * Install it: | ||
| ```bash | ||
| git clone https://github.com/kosmtik/kosmtik | ||
| cd kosmtik | ||
| npm install | ||
| ``` | ||
| * _The 0.0.17 release of Kosmtik wouldn't be enough because we need up-to-date CartoCSS and Mapnik versions._ | ||
| * Launch it: | ||
| ```bash | ||
| node index.js serve path/to/openstreetmap-carto/project.mml | ||
| ``` | ||
| * [TileMill](https://tilemill-project.github.io/tilemill/) | ||
| * Not officially supported, but you may be able to use a recent TileMill version by copying or symlinking the project directly into your Mapbox/project directory. | ||
| #### Optional | ||
| Some colours, SVGs and other files are generated with helper scripts. Not all users will need these dependencies. | ||
| * [Python](https://www.python.org/downloads/) | ||
| * [`generate_road_colours.py`](./scripts/generate_road_colours.py) and [`generate_unpaved_patterns.py`](./scripts/generate_unpaved_patterns.py) depend on [Color Math](https://github.com/gtaylor/python-colormath) and [`numpy`](https://numpy.org/). To install these, run: | ||
| ```bash | ||
| python3 -m pip install --break-system-packages --user colormath numpy | ||
| ``` | ||
| ### For deployment | ||
| CartoCSS and Mapnik are required for deployment. | ||
| * [CartoCSS](https://github.com/mapbox/carto) >= `1.2.0` *(we're using YAML)* | ||
| * [Mapnik](https://github.com/mapnik/mapnik/wiki/Mapnik-Installation) >= `3.0.22` | ||
|
|
||
| With CartoCSS, these sources are compiled into a Mapnik compatible XML file. When running CartoCSS, specify the Mapnik API version you are using (at least 3.0.22: `carto -a "3.0.22"`). |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| settings: | ||
| temp_schema: loading | ||
| schema: public | ||
| data_dir: data | ||
| database: gis | ||
| metadata_table: external_data | ||
| sources: | ||
| simplified_water_polygons: | ||
| # The type of file this source is | ||
| type: shp | ||
| # Where to get it | ||
| url: https://osmdata.openstreetmap.de/download/simplified-water-polygons-split-3857.zip | ||
| # The location within the archive | ||
| file: simplified-water-polygons-split-3857/simplified_water_polygons.shp | ||
| archive: | ||
| format: zip | ||
| # Files to extract from the archive | ||
| files: | ||
| - simplified-water-polygons-split-3857/simplified_water_polygons.cpg | ||
| - simplified-water-polygons-split-3857/simplified_water_polygons.dbf | ||
| - simplified-water-polygons-split-3857/simplified_water_polygons.prj | ||
| - simplified-water-polygons-split-3857/simplified_water_polygons.shp | ||
| - simplified-water-polygons-split-3857/simplified_water_polygons.shx | ||
| water_polygons: | ||
| type: shp | ||
| url: https://osmdata.openstreetmap.de/download/water-polygons-split-3857.zip | ||
| file: water-polygons-split-3857/water_polygons.shp | ||
| archive: | ||
| format: zip | ||
| files: | ||
| - water-polygons-split-3857/water_polygons.cpg | ||
| - water-polygons-split-3857/water_polygons.dbf | ||
| - water-polygons-split-3857/water_polygons.prj | ||
| - water-polygons-split-3857/water_polygons.shp | ||
| - water-polygons-split-3857/water_polygons.shx | ||
| icesheet_polygons: | ||
| type: shp | ||
| url: https://osmdata.openstreetmap.de/download/antarctica-icesheet-polygons-3857.zip | ||
| file: antarctica-icesheet-polygons-3857/icesheet_polygons.shp | ||
| archive: | ||
| format: zip | ||
| files: | ||
| - antarctica-icesheet-polygons-3857/icesheet_polygons.cpg | ||
| - antarctica-icesheet-polygons-3857/icesheet_polygons.dbf | ||
| - antarctica-icesheet-polygons-3857/icesheet_polygons.prj | ||
| - antarctica-icesheet-polygons-3857/icesheet_polygons.shp | ||
| - antarctica-icesheet-polygons-3857/icesheet_polygons.shx | ||
| icesheet_outlines: | ||
| type: shp | ||
| url: https://osmdata.openstreetmap.de/download/antarctica-icesheet-outlines-3857.zip | ||
| file: antarctica-icesheet-outlines-3857/icesheet_outlines.shp | ||
| ogropts: | ||
| - "-explodecollections" | ||
| archive: | ||
| format: zip | ||
| files: | ||
| - antarctica-icesheet-outlines-3857/icesheet_outlines.cpg | ||
| - antarctica-icesheet-outlines-3857/icesheet_outlines.dbf | ||
| - antarctica-icesheet-outlines-3857/icesheet_outlines.prj | ||
| - antarctica-icesheet-outlines-3857/icesheet_outlines.shp | ||
| - antarctica-icesheet-outlines-3857/icesheet_outlines.shx | ||
|
|
||
| ne_110m_admin_0_boundary_lines_land: | ||
| type: shp | ||
| url: https://naturalearth.s3.amazonaws.com/110m_cultural/ne_110m_admin_0_boundary_lines_land.zip | ||
| file: ne_110m_admin_0_boundary_lines_land.shp | ||
| ogropts: &ne_opts | ||
| - "--config" | ||
| - "SHAPE_ENCODING" | ||
| - "WINDOWS-1252" | ||
| - "-explodecollections" | ||
| # needs reprojecting | ||
| - '-t_srs' | ||
| - 'EPSG:3857' | ||
| archive: | ||
| format: zip | ||
| files: | ||
| - ne_110m_admin_0_boundary_lines_land.dbf | ||
| - ne_110m_admin_0_boundary_lines_land.prj | ||
| - ne_110m_admin_0_boundary_lines_land.shp | ||
| - ne_110m_admin_0_boundary_lines_land.shx |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| /* Additional database functions for openstreetmap-carto */ | ||
|
|
||
| /* Access functions below adapted from https://github.com/imagico/osm-carto-alternative-colors/tree/591c861112b4e5d44badd108f4cd1409146bca0b/sql/roads.sql */ | ||
|
|
||
| /* Simplified 'yes', 'destination', 'no', 'unrecognised', NULL scale for access restriction | ||
| 'no' is returned if the rendering for highway category does not support 'restricted'. | ||
| NULL is functionally equivalent to 'yes', but indicates the absence of a restriction | ||
| rather than a positive access = yes. 'unrecognised' corresponds to an uninterpretable | ||
| access restriction e.g. access=unknown or motorcar=occasionally */ | ||
| CREATE OR REPLACE FUNCTION carto_int_access(accessvalue text, allow_restricted boolean) | ||
| RETURNS text | ||
| LANGUAGE SQL | ||
| IMMUTABLE PARALLEL SAFE | ||
| AS $$ | ||
| SELECT | ||
| CASE | ||
| WHEN accessvalue IN ('yes', 'designated', 'permissive') THEN 'yes' | ||
| WHEN accessvalue IN ('destination', 'delivery', 'customers') THEN | ||
| CASE WHEN allow_restricted = TRUE THEN 'restricted' ELSE 'yes' END | ||
| WHEN accessvalue IN ('no', 'permit', 'private', 'agricultural', 'forestry', 'agricultural;forestry') THEN 'no' | ||
| WHEN accessvalue IS NULL THEN NULL | ||
| ELSE 'unrecognised' | ||
| END | ||
| $$; | ||
|
|
||
| /* Try to promote path to cycleway (if bicycle allowed), then bridleway (if horse) | ||
| This duplicates existing behaviour where designated access is required */ | ||
| CREATE OR REPLACE FUNCTION carto_path_type(bicycle text, horse text) | ||
| RETURNS text | ||
| LANGUAGE SQL | ||
| IMMUTABLE PARALLEL SAFE | ||
| AS $$ | ||
| SELECT | ||
| CASE | ||
| WHEN bicycle IN ('designated') THEN 'cycleway' | ||
| WHEN horse IN ('designated') THEN 'bridleway' | ||
| ELSE 'path' | ||
| END | ||
| $$; | ||
|
|
||
| /* Return int_access value which will be used to determine access marking. | ||
| Return values are documented above for carto_int_access function. | ||
| Note that the code handling the promotion of highway=path assumes that | ||
| promotion to cycleway or bridleway is based on the value of bicycle or | ||
| horse respectively. A more general formulation would be, for example, | ||
| WHEN 'cycleway' THEN carto_int_access(COALESCE(NULLIF(bicycle, 'unknown'), "access"), FALSE) */ | ||
| CREATE OR REPLACE FUNCTION carto_highway_int_access(highway text, "access" text, foot text, bicycle text, horse text, motorcar text, motor_vehicle text, vehicle text) | ||
| RETURNS text | ||
| LANGUAGE SQL | ||
| IMMUTABLE PARALLEL SAFE | ||
| AS $$ | ||
| SELECT | ||
| CASE | ||
| WHEN highway IN ('motorway', 'motorway_link', 'trunk', 'trunk_link', 'primary', 'primary_link', 'secondary', | ||
| 'secondary_link', 'tertiary', 'tertiary_link', 'residential', 'unclassified', 'living_street', 'service', 'road') THEN | ||
| carto_int_access( | ||
| COALESCE( | ||
| NULLIF(motorcar, 'unknown'), | ||
| NULLIF(motor_vehicle, 'unknown'), | ||
| NULLIF(vehicle, 'unknown'), | ||
| "access"), TRUE) | ||
| WHEN highway = 'path' THEN | ||
| CASE carto_path_type(bicycle, horse) | ||
| WHEN 'cycleway' THEN carto_int_access(bicycle, FALSE) | ||
| WHEN 'bridleway' THEN carto_int_access(horse, FALSE) | ||
| ELSE carto_int_access(COALESCE(NULLIF(foot, 'unknown'), "access"), FALSE) | ||
| END | ||
| WHEN highway IN ('pedestrian', 'footway', 'steps') THEN carto_int_access(COALESCE(NULLIF(foot, 'unknown'), "access"), FALSE) | ||
| WHEN highway = 'cycleway' THEN carto_int_access(COALESCE(NULLIF(bicycle, 'unknown'), "access"), FALSE) | ||
| WHEN highway = 'bridleway' THEN carto_int_access(COALESCE(NULLIF(horse, 'unknown'), "access"), FALSE) | ||
| ELSE carto_int_access("access", TRUE) | ||
| END | ||
| $$; |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,42 +1,18 @@ | ||
| -- These are indexes for rendering performance with OpenStreetMap Carto. | ||
| -- This file is generated with scripts/indexes.py | ||
| CREATE INDEX planet_osm_line_ferry ON planet_osm_line USING GIST (way) WHERE route = 'ferry' AND osm_id > 0; | ||
| CREATE INDEX planet_osm_line_label ON planet_osm_line USING GIST (way) WHERE name IS NOT NULL OR ref IS NOT NULL; | ||
| CREATE INDEX planet_osm_line_river ON planet_osm_line USING GIST (way) WHERE waterway = 'river'; | ||
| CREATE INDEX planet_osm_line_waterway ON planet_osm_line USING GIST (way) WHERE waterway IN ('river', 'canal', 'stream', 'drain', 'ditch'); | ||
| CREATE INDEX planet_osm_point_place ON planet_osm_point USING GIST (way) WHERE place IS NOT NULL AND name IS NOT NULL; | ||
| CREATE INDEX planet_osm_polygon_admin ON planet_osm_polygon USING GIST (ST_PointOnSurface(way)) WHERE name IS NOT NULL AND boundary = 'administrative' AND admin_level IN ('0', '1', '2', '3', '4'); | ||
| CREATE INDEX planet_osm_polygon_military ON planet_osm_polygon USING GIST (way) WHERE (landuse = 'military' OR military = 'danger_area') AND building IS NULL; | ||
| CREATE INDEX planet_osm_polygon_name ON planet_osm_polygon USING GIST (ST_PointOnSurface(way)) WHERE name IS NOT NULL; | ||
| CREATE INDEX planet_osm_polygon_name_z6 ON planet_osm_polygon USING GIST (ST_PointOnSurface(way)) WHERE name IS NOT NULL AND way_area > 5980000; | ||
| CREATE INDEX planet_osm_polygon_nobuilding ON planet_osm_polygon USING GIST (way) WHERE building IS NULL; | ||
| CREATE INDEX planet_osm_polygon_water ON planet_osm_polygon USING GIST (way) WHERE waterway IN ('dock', 'riverbank', 'canal') OR landuse IN ('reservoir', 'basin') OR "natural" IN ('water', 'glacier'); | ||
| CREATE INDEX planet_osm_polygon_way_area_z10 ON planet_osm_polygon USING GIST (way) WHERE way_area > 23300; | ||
| CREATE INDEX planet_osm_polygon_way_area_z6 ON planet_osm_polygon USING GIST (way) WHERE way_area > 5980000; | ||
| CREATE INDEX planet_osm_roads_admin ON planet_osm_roads USING GIST (way) WHERE boundary = 'administrative'; | ||
| CREATE INDEX planet_osm_roads_admin_low ON planet_osm_roads USING GIST (way) WHERE boundary = 'administrative' AND admin_level IN ('0', '1', '2', '3', '4'); | ||
| CREATE INDEX planet_osm_roads_roads_ref ON planet_osm_roads USING GIST (way) WHERE highway IS NOT NULL AND ref IS NOT NULL; |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| #!/bin/sh | ||
| set -e | ||
|
|
||
| FONTDIR="./fonts" | ||
|
|
||
| mkdir -p "${FONTDIR}" | ||
|
|
||
| # download filename url | ||
| download() { | ||
| ## Download if newer, and if curl fails, clean up and exit | ||
| curl --fail --compressed -A "get-fonts.sh/osm-carto" -o "$1" -z "$1" -L "$2" || { echo "Failed to download $1 $2"; rm -f "$1"; exit 1; } | ||
| } | ||
|
|
||
| case "$1" in | ||
| ja) | ||
| download "${FONTDIR}/NotoSansCJKjp-Regular.otf" "https://github.com/googlefonts/noto-cjk/raw/main/Sans/OTF/Japanese/NotoSansCJKjp-Regular.otf" | ||
| download "${FONTDIR}/NotoSansCJKjp-Bold.otf" "https://github.com/googlefonts/noto-cjk/raw/main/Sans/OTF/Japanese/NotoSansCJKjp-Bold.otf" | ||
| sed -i "s/Noto Sans CJK \(JP\|KR\|SC\|TC\|HK\)/Noto Sans CJK JP/g" ./style/fonts.mss | ||
| ;; | ||
| ko) | ||
| download "${FONTDIR}/NotoSansCJKkr-Regular.otf" "https://github.com/googlefonts/noto-cjk/raw/main/Sans/OTF/Korean/NotoSansCJKkr-Regular.otf" | ||
| download "${FONTDIR}/NotoSansCJKkr-Bold.otf" "https://github.com/googlefonts/noto-cjk/raw/main/Sans/OTF/Korean/NotoSansCJKkr-Bold.otf" | ||
| sed -i "s/Noto Sans CJK \(JP\|KR\|SC\|TC\|HK\)/Noto Sans CJK KR/g" ./style/fonts.mss | ||
| ;; | ||
| zh-[Hh]ans) | ||
| download "${FONTDIR}/NotoSansCJKsc-Regular.otf" "https://github.com/googlefonts/noto-cjk/raw/main/Sans/OTF/SimplifiedChinese/NotoSansCJKsc-Regular.otf" | ||
| download "${FONTDIR}/NotoSansCJKsc-Bold.otf" "https://github.com/googlefonts/noto-cjk/raw/main/Sans/OTF/SimplifiedChinese/NotoSansCJKsc-Bold.otf" | ||
| sed -i "s/Noto Sans CJK \(JP\|KR\|SC\|TC\|HK\)/Noto Sans CJK SC/g" ./style/fonts.mss | ||
| ;; | ||
| zh-[Hh]ant-[Tt][Ww]) | ||
| download "${FONTDIR}/NotoSansCJKtc-Regular.otf" "https://github.com/googlefonts/noto-cjk/raw/main/Sans/OTF/TraditionalChinese/NotoSansCJKtc-Regular.otf" | ||
| download "${FONTDIR}/NotoSansCJKtc-Bold.otf" "https://github.com/googlefonts/noto-cjk/raw/main/Sans/OTF/TraditionalChinese/NotoSansCJKtc-Bold.otf" | ||
| sed -i "s/Noto Sans CJK \(JP\|KR\|SC\|TC\|HK\)/Noto Sans CJK TC/g" ./style/fonts.mss | ||
| ;; | ||
| zh-[Hh]ant-[Hh][Kk]) | ||
| download "${FONTDIR}/NotoSansCJKhk-Regular.otf" "https://github.com/googlefonts/noto-cjk/raw/main/Sans/OTF/TraditionalChineseHK/NotoSansCJKhk-Regular.otf" | ||
| download "${FONTDIR}/NotoSansCJKhk-Bold.otf" "https://github.com/googlefonts/noto-cjk/raw/main/Sans/OTF/TraditionalChineseHK/NotoSansCJKhk-Bold.otf" | ||
| sed -i "s/Noto Sans CJK \(JP\|KR\|SC\|TC\|HK\)/Noto Sans CJK HK/g" ./style/fonts.mss | ||
| ;; | ||
| *) | ||
| echo "Usage: change-fonts-cjk.sh ja|ko|zh-Hans|zh-Hant-TW|zh-Hant-HK" | ||
| exit 1 | ||
| ;; | ||
| esac | ||
|
|
||
| echo "Done." |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Thu, 01 Jan 1970 00:00:00 GMT |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Thu, 01 Jan 1970 00:00:00 GMT |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Thu, 01 Jan 1970 00:00:00 GMT |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Thu, 01 Jan 1970 00:00:00 GMT |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Thu, 01 Jan 1970 00:00:00 GMT |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,143 @@ | ||
| #!/usr/bin/env python3 | ||
|
|
||
| # Colours for unpaved roads | ||
|
|
||
| # This reads some color variables from some .mss files and also reads | ||
| # symbols/unpaved/unpaved.svg and generates colourized versions of | ||
| # the unpaved pattern for all road types and saves them in the symbols/unpaved | ||
| # folder. Existing files of the same name are overwritten! | ||
| # | ||
| # This script produces patterns that perceptually have the same overall | ||
| # brightness as the original road color. Therefor, the pattern foreground | ||
| # is darker than the original color, and the pattern background lighter than | ||
| # the original color. This script does its very best, but the same overall | ||
| # brightness is not always possible (for example for white roads) and also | ||
| # depends on the monitor gammut on which the pattern is finally displayed. | ||
| # | ||
| # Usage: Call this script in the main directory of openstreetmap-carto | ||
| # without further parameters. | ||
| # | ||
| # Customize: | ||
| # You can customize this script by changing the first variables of in the main() | ||
| # function (color_names, file_names, darken, brighten_darken_ratio). | ||
|
|
||
| from colormath.color_objects import LabColor, sRGBColor | ||
| from colormath.color_conversions import convert_color | ||
|
|
||
| # def get_color_value_by_name(variable_name, file_names): | ||
| # | ||
| # Searches in MSS files for variable values with the given name. Returns | ||
| # the first value it finds. Only supports very basic syntax like: | ||
| # @test: 12; # Comment | ||
| # which would return "12". | ||
| # | ||
| # Paramaters: | ||
| # variable_name: the name of the variable for which we search the value | ||
| # file_names: list of files where we search for the variable value | ||
| # | ||
| # Return value: the variable value (if any) | ||
| def get_color_value_by_name(variable_name, file_names): | ||
| for files in file_names: | ||
| with open(files) as f: | ||
| for line in f: | ||
| if line.startswith("@" + variable_name + ":"): | ||
| temp = line.strip("@" + variable_name + ":").split(";")[0].strip() | ||
| # test if the value length is okay (#abc or #aabbcc) | ||
| if (len(temp) == 4) or (len(temp)== 7): | ||
| # remove the first character (#) | ||
| temp = temp[1:] | ||
| # expand value like #abc to #aabbcc | ||
| if len(temp) == 3: | ||
| temp = temp[0] + temp [0] + temp [1] + temp [1] + temp [2] + temp [2] | ||
| # make sure that the content is really a (lowercase) hex value | ||
| if all(c in set("0123456789abcdef") for c in temp): | ||
| # if so, return the hex value with a leading "#" | ||
| return ("#" + temp) | ||
|
|
||
| # Takes an RGB hex values, applies the indicated Lab lightness change and returns the result as RGB hex value again | ||
| # def change_lightness(base_color_rgb_hex, lightness_change): | ||
| # | ||
| # Changes the lightness of a given color. This function | ||
| # tries to do a perceptual transformation. | ||
| # | ||
| # Paramaters: | ||
| # base_color_rgb_hex: An RGB hex value like #1212ab | ||
| # lightness_change: A number (positive or negative), interpreted as change to the lightness component like in Lab perceptual color space (range: 0..100) | ||
| # | ||
| # Return value: The RGB hex value with the lightness change applied. If the lightness change leaves | ||
| # us with an out-of-gammut value, it is clipped to make sure to be within the RGB gammut. So the return | ||
| # value is guarantied to be always a valid RGB value. | ||
| def change_lightness(base_color_rgb_hex, lightness_change): | ||
| base_color_lab = convert_color(sRGBColor.new_from_rgb_hex(base_color_rgb_hex), LabColor) | ||
| new_color_lab = LabColor( | ||
| base_color_lab.lab_l + lightness_change, # This value might be out of gammut and therefor invalid | ||
| base_color_lab.lab_a, | ||
| base_color_lab.lab_b, | ||
| base_color_lab.observer, | ||
| base_color_lab.illuminant) | ||
| new_color_rgb = convert_color(new_color_lab, sRGBColor) # This value might be out of gammut and therefor invalid | ||
| # use the "clamped" values which means they are within the gammut and therefor valid | ||
| new_color_rgb_clamped = sRGBColor( | ||
| new_color_rgb.clamped_rgb_r, | ||
| new_color_rgb.clamped_rgb_g, | ||
| new_color_rgb.clamped_rgb_b) | ||
| return new_color_rgb_clamped.get_rgb_hex() | ||
|
|
||
| def main(): | ||
|
|
||
| # List of names of color variables in mss code for which we will generate patterns | ||
| color_names = { | ||
| 'motorway-low-zoom', | ||
| 'trunk-low-zoom', | ||
| 'primary-low-zoom', | ||
| 'motorway-fill', | ||
| 'trunk-fill', | ||
| 'primary-fill', | ||
| 'secondary-fill', | ||
| 'platform-fill', | ||
| 'aeroway-fill', | ||
| 'road-fill', | ||
| 'pedestrian-fill', | ||
| 'living-street-fill', | ||
| 'raceway-fill', | ||
| 'residential-fill' | ||
| } | ||
|
|
||
| # List of names of mss files in which we search for color variables | ||
| file_names = { | ||
| 'style/roads.mss', | ||
| 'style/road-colors-generated.mss' | ||
| } | ||
|
|
||
| # The value by which the original color is darkened for the pattern foreground | ||
| # This value should always be negative. | ||
| darken = -60 | ||
|
|
||
| # The pattern foreground occupies less space than the background. So lightening | ||
| # the background has to be less intense than darkening the foreground. This | ||
| # value is multiplied with the negative value of "darken" to get a value for | ||
| # "brighten", so brighten_darken_ration must also be negative to make sure | ||
| # the "brighten" result is positive. This value should (only) be changed when | ||
| # the pattern itself is changed. | ||
| brighten_darken_ratio = -0.065 | ||
|
|
||
| # actual code | ||
| for color_name in color_names: | ||
| print("\nColor name: " + color_name) | ||
| original_color_value = get_color_value_by_name(color_name, file_names) | ||
| print("Plain color: " + original_color_value) | ||
| pattern_colors = [change_lightness(original_color_value, darken), | ||
| change_lightness(original_color_value, darken * brighten_darken_ratio)] | ||
| print("Colors for pattern: " + str(pattern_colors)) | ||
| if pattern_colors: | ||
| with open('symbols/unpaved/unpaved.svg', 'rt') as fin: | ||
| with open('symbols/unpaved/unpaved_' + color_name + '.svg', 'wt') as fout: | ||
| for line in fin: | ||
| temp = line | ||
| temp = temp.replace('#0000ff', pattern_colors[0]) | ||
| temp = temp.replace('fill:none', 'fill:' + pattern_colors[1]) | ||
| fout.write(temp) | ||
| print("Pattern file: " + 'symbols/unpaved/unpaved_' + color_name + '.svg') | ||
|
|
||
| if __name__ == "__main__": | ||
| main() |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,132 @@ | ||
| #!/bin/sh | ||
| set -e | ||
|
|
||
| FONTDIR="./fonts" | ||
|
|
||
| mkdir -p "${FONTDIR}" | ||
|
|
||
| # download filename url | ||
| download() { | ||
| ## Download if newer, and if curl fails, clean up and exit | ||
| curl --fail --compressed -A "get-fonts.sh/osm-carto" -o "$1" -z "$1" -L "$2" || { echo "Failed to download $1 $2"; rm -f "$1"; exit 1; } | ||
| } | ||
|
|
||
| # TTF Hinted Noto Fonts | ||
|
|
||
| # Fonts available in regular, bold, and italic | ||
| REGULAR_BOLD_ITALIC="NotoSans" | ||
|
|
||
| # Fonts available in regular and bold | ||
| REGULAR_BOLD="NotoSansAdlamUnjoined | ||
| NotoSansArabicUI | ||
| NotoSansArmenian | ||
| NotoSansBalinese | ||
| NotoSansBamum | ||
| NotoSansBengaliUI | ||
| NotoSansCanadianAboriginal | ||
| NotoSansCham | ||
| NotoSansCherokee | ||
| NotoSansDevanagariUI | ||
| NotoSansEthiopic | ||
| NotoSansGeorgian | ||
| NotoSansGujaratiUI | ||
| NotoSansGurmukhiUI | ||
| NotoSansHebrew | ||
| NotoSansJavanese | ||
| NotoSansKannadaUI | ||
| NotoSansKayahLi | ||
| NotoSansKhmerUI | ||
| NotoSansLaoUI | ||
| NotoSansLisu | ||
| NotoSansMalayalamUI | ||
| NotoSansMyanmarUI | ||
| NotoSansOlChiki | ||
| NotoSansOriyaUI | ||
| NotoSansSinhalaUI | ||
| NotoSansSundanese | ||
| NotoSansSymbols | ||
| NotoSansTaiTham | ||
| NotoSansTamilUI | ||
| NotoSansTeluguUI | ||
| NotoSansThaana | ||
| NotoSansThaiUI | ||
| NotoSerifTibetan" | ||
|
|
||
| # Fonts with regular and black, but no bold | ||
| REGULAR_BLACK="NotoSansSyriac" | ||
|
|
||
| # Fonts only available in regular | ||
| REGULAR="NotoSansBatak | ||
| NotoSansBuginese | ||
| NotoSansBuhid | ||
| NotoSansChakma | ||
| NotoSansCoptic | ||
| NotoSansHanunoo | ||
| NotoSansLepcha | ||
| NotoSansLimbu | ||
| NotoSansMandaic | ||
| NotoSansMongolian | ||
| NotoSansNewTaiLue | ||
| NotoSansNKo | ||
| NotoSansOsage | ||
| NotoSansOsmanya | ||
| NotoSansSamaritan | ||
| NotoSansSaurashtra | ||
| NotoSansShavian | ||
| NotoSansSymbols2 | ||
| NotoSansTagalog | ||
| NotoSansTagbanwa | ||
| NotoSansTaiLe | ||
| NotoSansTaiViet | ||
| NotoSansTifinagh | ||
| NotoSansVai | ||
| NotoSansYi" | ||
|
|
||
| # Download the fonts in the lists above | ||
|
|
||
| for font in $REGULAR_BOLD_ITALIC; do | ||
| regular="$font-Regular.ttf" | ||
| bold="$font-Bold.ttf" | ||
| italic="$font-Italic.ttf" | ||
| download "${FONTDIR}/${regular}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${regular}" | ||
| download "${FONTDIR}/${bold}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${bold}" | ||
| download "${FONTDIR}/${italic}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${italic}" | ||
| done | ||
|
|
||
| for font in $REGULAR_BOLD; do | ||
| regular="$font-Regular.ttf" | ||
| bold="$font-Bold.ttf" | ||
| download "${FONTDIR}/${regular}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${regular}" | ||
| download "${FONTDIR}/${bold}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${bold}" | ||
| done | ||
|
|
||
| for font in $REGULAR_BLACK; do | ||
| regular="$font-Regular.ttf" | ||
| black="$font-Black.ttf" | ||
| download "${FONTDIR}/${regular}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${regular}" | ||
| download "${FONTDIR}/${black}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${black}" | ||
| done | ||
|
|
||
| for font in $REGULAR; do | ||
| regular="$font-Regular.ttf" | ||
| download "${FONTDIR}/${regular}" "https://github.com/notofonts/noto-fonts/raw/main/hinted/ttf/${font}/${regular}" | ||
| done | ||
|
|
||
| # Other noto fonts which don't follow the URL pattern above | ||
| download "${FONTDIR}/NotoSansCJKjp-Regular.otf" "https://github.com/googlefonts/noto-cjk/raw/main/Sans/OTF/Japanese/NotoSansCJKjp-Regular.otf" | ||
| download "${FONTDIR}/NotoSansCJKjp-Bold.otf" "https://github.com/googlefonts/noto-cjk/raw/main/Sans/OTF/Japanese/NotoSansCJKjp-Bold.otf" | ||
|
|
||
| # Fonts in zipfiles need a temporary directory | ||
| TMPDIR=$(mktemp -d -t get-fonts.XXXXXXXXX) | ||
| trap "rm -rf ${TMPDIR} ${FONTDIR}/static" EXIT | ||
|
|
||
| # Noto Emoji B&W isn't available as a separate download, so we need to download the package and unzip it | ||
| curl --fail -A "get-fonts.sh/osm-carto" -o "${TMPDIR}/Noto_Emoji.zip" -L 'https://fonts.google.com/download?family=Noto%20Emoji' | ||
|
|
||
| unzip -oqq "${TMPDIR}/Noto_Emoji.zip" static/NotoEmoji-Regular.ttf static/NotoEmoji-Bold.ttf -d "${FONTDIR}" | ||
| mv "${FONTDIR}/static/NotoEmoji-Regular.ttf" "${FONTDIR}" | ||
| mv "${FONTDIR}/static/NotoEmoji-Bold.ttf" "${FONTDIR}" | ||
|
|
||
| curl --fail -A "get-fonts.sh/osm-carto" -o "${TMPDIR}/hanazono.zip" -L 'https://mirrors.dotsrc.org/osdn/hanazono-font/68253/hanazono-20170904.zip' | ||
|
|
||
| unzip -oqq "${TMPDIR}/hanazono.zip" HanaMinA.ttf HanaMinB.ttf -d "${FONTDIR}" |