Enable mapfile strings and arbitrary file streams to be tokenized #4031

Open
mapserver-bot opened this Issue Apr 4, 2012 · 7 comments

Comments

Projects
None yet
5 participants
@mapserver-bot

Reporter: homme
Date: 2011/09/20 - 17:38
Trac URL: http://trac.osgeo.org/mapserver/ticket/4031
mapfile.c contains the function msTokenizeMap() which is useful in systems which adopt a more declarative style to manipulating mapfiles and generating maps than traditional mapscript applications. A declarative system commonly creates a mapfile which it then passes to the stock CGI mapserv application for rendering.

msTokenizeMap(), however, only allows tokenizing of mapfiles referenced on disk by file paths. Applications sometimes need more flexibility in sourcing mapfiles (e.g. from databases and alternative file streams) without the overhead of saving them to disk for use with msTokenizeMap().

The attached patch extends the mapserver API to provide msTokenizeMapStream() and msTokenizeMapString() which allow arbitrary FILE streams and strings representing mapfiles to be tokenized. It refactors the original msTokenizeMap() code to maximise code reuse and minimise any code maintenance.

Also attached is a small test program which demonstrates the API.

@mapserver-bot

This comment has been minimized.

Show comment
Hide comment
@mapserver-bot

mapserver-bot Apr 5, 2012

attachment http://trac.osgeo.org/mapserver/attachment/ticket/4031/msTokenize.patch :

   A patch for mapserver.h and mapfile.c against revision 12562

attachment http://trac.osgeo.org/mapserver/attachment/ticket/4031/msTokenize.patch :

   A patch for mapserver.h and mapfile.c against revision 12562
@mapserver-bot

This comment has been minimized.

Show comment
Hide comment
@mapserver-bot

mapserver-bot Apr 5, 2012

attachment http://trac.osgeo.org/mapserver/attachment/ticket/4031/tokenize-test.c :

   Example program demonstrating the use of msTokenizeMapStream() and msTokenizeMapString()

attachment http://trac.osgeo.org/mapserver/attachment/ticket/4031/tokenize-test.c :

   Example program demonstrating the use of msTokenizeMapStream() and msTokenizeMapString()

@ghost ghost assigned sdlime Apr 5, 2012

@jratike80

This comment has been minimized.

Show comment
Hide comment
@jratike80

jratike80 Nov 7, 2015

There has been no comments ever on this patch which is unfriendly for @homme.

There has been no comments ever on this patch which is unfriendly for @homme.

@geographika

This comment has been minimized.

Show comment
Hide comment
@geographika

geographika Jul 22, 2017

Contributor

@homme - I've been looking to do something similar with Python and came across this pull request. Is this approach still needed or did using the msLoadMapFromString function give you everything you needed in the end?

msLoadMapFromString used at https://github.com/geo-data/node-mapserv/blob/master/src/map.cpp#L264

Contributor

geographika commented Jul 22, 2017

@homme - I've been looking to do something similar with Python and came across this pull request. Is this approach still needed or did using the msLoadMapFromString function give you everything you needed in the end?

msLoadMapFromString used at https://github.com/geo-data/node-mapserv/blob/master/src/map.cpp#L264

@homme

This comment has been minimized.

Show comment
Hide comment
@homme

homme Jul 24, 2017

Contributor

@geographika It's been a while since I looked at this, but msLoadMapFromString was enough of a workaround for that particular use case. My original intention with this PR was to provide the low level functionality that could be used to integrate a declarative format that could be easily manipulated by third party clients (e.g. JSON) with the Mapserver core. I still think this would be useful for being able to drive Mapserver without needing to resort to language bindings. The icing on the cake would be to have Mapserver use the GDAL virtual file system API to access Mapfiles :)

Contributor

homme commented Jul 24, 2017

@geographika It's been a while since I looked at this, but msLoadMapFromString was enough of a workaround for that particular use case. My original intention with this PR was to provide the low level functionality that could be used to integrate a declarative format that could be easily manipulated by third party clients (e.g. JSON) with the Mapserver core. I still think this would be useful for being able to drive Mapserver without needing to resort to language bindings. The icing on the cake would be to have Mapserver use the GDAL virtual file system API to access Mapfiles :)

@geographika

This comment has been minimized.

Show comment
Hide comment
@geographika

geographika Jul 24, 2017

Contributor

@homme - many thanks for getting back on this after 5 years!

As I understand it the patch would allow a string to be parsed to mapserv.exe rather than just a path? E.g. mapserv -m "MAP NAME....END" and would generate an image? That would be a nice feature, and would avoid having to write temporary files to disk. I ran into the same problem with a Python library I'm working on (http://mappyfile.readthedocs.io/en/latest/).

I was discussing this with @rouault at the recent FOSS4GE codesprint in Paris. He may be able to comment further on the GDAL virtual file system.

Contributor

geographika commented Jul 24, 2017

@homme - many thanks for getting back on this after 5 years!

As I understand it the patch would allow a string to be parsed to mapserv.exe rather than just a path? E.g. mapserv -m "MAP NAME....END" and would generate an image? That would be a nice feature, and would avoid having to write temporary files to disk. I ran into the same problem with a Python library I'm working on (http://mappyfile.readthedocs.io/en/latest/).

I was discussing this with @rouault at the recent FOSS4GE codesprint in Paris. He may be able to comment further on the GDAL virtual file system.

@homme

This comment has been minimized.

Show comment
Hide comment
@homme

homme Jul 25, 2017

Contributor

@geographika as the patch stands I don't think it directly enables mapserv -m "MAP NAME....END" type functionality, but I believe it provides the ground work for that: the mapserv code would probably need to be updated to take advantage of it.

Mappy looks very cool, and just the kind of thing I had in mind! I started but didn't finish work on something similar as a bit of a side project in Golang a while back: geo-data/mapstruct. I moved it over to using its own scanner instead of the Tokenize bindings to reduce dependencies.

Contributor

homme commented Jul 25, 2017

@geographika as the patch stands I don't think it directly enables mapserv -m "MAP NAME....END" type functionality, but I believe it provides the ground work for that: the mapserv code would probably need to be updated to take advantage of it.

Mappy looks very cool, and just the kind of thing I had in mind! I started but didn't finish work on something similar as a bit of a side project in Golang a while back: geo-data/mapstruct. I moved it over to using its own scanner instead of the Tokenize bindings to reduce dependencies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment