customize postgres configuration #4

Closed
ian29 opened this Issue Dec 12, 2012 · 9 comments

Projects

None yet

4 participants

@ian29

would be great if you could configure the postgres connection parameters with a simple script - some people may be connection on different hosts, users, passwords, etc and it's a pain to use TileMill's GUI to change these, and scary to do this with sed. especially true if you are unfamiliar with a project.mml.

could be some very simple variable replacement magic.

@gravitystorm

I discussed this with a few guys at the Hack Weekend. A script would be great, but I'm strongly in favour of making the default path work without one - i.e. if you use the defaults in osm2pgsql (dname=gis, table prefix = planet_osm_ etc) then you shouldn't need to run a script.

So I think such a script should be able to read the project.mml, probably parse the json and iterate over the results changing whatever needs to be changed, and the write it back out again.

Of course, a better approach is to improve carto so that this is less necessary, in any stylesheet. Mapnik has (I've been told) support for datasource templates as children of the Map element within the xml format, but documentation on this concept seems to be scarce. It would then be possible to extend carto to define the database parameters in only one place, making fiddling with them much easier and perhaps removing the need for a script.

@gravitystorm

See mapnik/mapnik@37f49e2 for the commit that introduced the datasource templates to the XML format.

@springmeyer

Yes, <Datasource> and <FileSource> are XML properties that can be used in Mapnik XML to re-use settings. Examples are https://github.com/mapnik/mapnik/blob/master/tests/data/good_maps/datasource.xml and https://github.com/mapnik/mapnik/blob/master/tests/data/good_maps/filesource.xml.

I'm not sure how to add support for these in carto however.

@yohanboniface

FYI, I've set up for my needs a little tool for customizing the .mml file: https://github.com/yohanboniface/CartoCC
It's very simple, but can do the job in most of the common situations.

@gravitystorm
Owner

@yohanboniface that is excellent, exactly the approach I had in mind.

@ian29

@yohanboniface this is great work, thanks so much!

@yohanboniface

You guys are really welcome! :)
I think CartoCC can do a great job when you just need to locally test some style or deploy your style on some server. However I'm still thinking a way to integrate with Tilemill for a collaborative workflow, not to get every commit involving the .mml file polluted by env related config changes, and keeping the process as simple as possible.
One option could be to add a "remote_rules.json" in every style, which has to be processed before committing, to put back common values, and so limiting noise in commit diff.
One can also always use git add -p for .mml, but it can be painful, and source of errors.
I also have to look at the Tilemill plugin API, maybe something can be done in this way.
One option could be to find a way of making Tilemill call CartoCC at runtime, but always writing layers changes in the project.mml itself, so 1. the project.mml file is never polluted by local conf changes; 2. Tilemill doesn't need to handle internally all sorts of config overwrite needed.
Of course, all this could really better be addressed in Tilemill itself.

@springmeyer

@yohanboniface - One thing you might be interested in is the syntax GYP uses. GYP is a build system that basically uses a json format file to author build files for each platform. It has standardized various keywords it understands and mostly avoids the if/else trap of re-inventing what would be better to do in a programming language. http://code.google.com/p/gyp/wiki/GypLanguageSpecification

For example, for the mapnik-reference project, I'm needing to build out a json file while reducing duplication. I'm leveraging the gyp-like variables keyword to make this easy: mapnik/mapnik-reference@7e59b1b

@gravitystorm
Owner

While this is obviously not fixed, we have a useful tool (CartoCC) and discussions going on upstream in Tilemill. I don't think there's anything else we can do here for now.

@imagico imagico pushed a commit to imagico/openstreetmap-carto that referenced this issue Sep 27, 2016
Sven Geggus Rename the database views to match upstream table names.
This way we can use project.yaml directly and get rid of project-de.yaml

This will also allow the usage of tools like kosmtik

(closes #4)
72a9c8f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment