I have a small shapefile (~1.5k) that I'm trying to edit in Quantum GIS and render in TileMill. But when I update the shapefile in QGIS, TileMill hangs.
Here's the process:
1. Starting condition: The shapefile is a layer in both my QGIS project and my TileMill project.
2. Enable editing on the layer in QGIS
3. Add a new polygon to the layer. (Editing an existing polygon works OK; I run into trouble adding a new polygon.)
4. Toggle editing off in QGIS. Save the shapefile when prompted.
5. Go back to TileMill, and try to get it to reload the data. (This could be changing the .mss file, or zooming to a scale that isn't cached.)
At step 5, several of my CPUs start running at 100%. I've waited 15 minutes for TileMill to come back; it doesn't. Normally this project refreshes in a few seconds. The only fix I've found is to restart TileMill.
The problem occurs in a project that consists of just the default data and my new shapefile.
I'm running TileMill 0.5.1.0-0ubuntu2 and QGIS 1.4.0-Enceladus.
tilemill under the hood uses mapnik which keeps open file references, both to the shapefile .dbf/.shp and to its index file (if you have one).
If you have an index then it would need to be manually regenerated (or removed) after shapefile editing and currently tilemill does not have mechanism for this, but we plan one in the future.
If you don't have an index then re-saving your stylesheet should refresh mapnik's reference to the shapefile. If you don't have a .index file and you are still seeing a hang after re-saving your project, then please post the shapefile (zipped up) somewhere.
In the past (older versions) of QGIS have actually created corrupt shapefiles after editing and the only solution I found when I saw this was to convert to a new shapefile with ogr2ogr on the command line.
By "index" do you mean using shapeindex to create a .index file? I'm not doing that.
I don't think the problem is with corrupt shapefiles. I'm able to use shapetosvg.pl to create an SVG file, with all my shapes in it.
My shapefile is available here: http://www.sfu.ca/~edickins/neighbourhood-type--tilemill-issue-714.zip
ya, that is what I meant by an index. So, could you try creating one? See if that fixes things?
I made the index, and Tilemill stopped hanging. But it also stopped updating the map data. Here's what I'm doing:
At step 3, my new polygons don't appear. Is this the problem you mentioned above, about re-generating the index? (I assumed you meant that TileMill doesn't re-create the .index file for me.)
right, the index will need to be recreated. So your steps above sounds right, and should work. I think what is happening is that the polygons you are creating in QGIS are invalid for mapnik (not your fault, maybe qgis's at fault maybe mapnik, hard to know right now). When you create the mapnik spatial index (.index) these polygons are effectively skipped and rendering works (without hangs). But the polygons don't show up of course.
What ideally you can do is post back here the exact details of your setup so, when I have time (which may not be for several weeks), I can be able to replicate.
Make sure to include:
1) exact details of how you installed tilemill
2) QGIS version, and how you installed it
3) operating system
4) anything else you can think of.
The actionable items are to:
1) ensure mapnik does not hang, no matter what, even without indexes built
2) figure out why/how these polygons you are creating are different
So, I installed TileMill with this install-tilemill.sh script. The version is 0.5.1.1-0ubuntu2. (I've updated it through apt-get since I installed it.)
DIST=`grep DISTRIB_CODENAME /etc/lsb-release | sed "s/DISTRIB_CODENAME=\(.*\)/\1/g"`
if [ -z "$DIST" ]; then
echo "Your Ubuntu distribution version could not be determined."
if [ "$DIST" != "maverick" && "$DIST" != "natty" ]; then
echo "Your distribution $DIST is not supported."
deb http://ppa.launchpad.net/developmentseed/mapbox/ubuntu $DIST main
deb-src http://ppa.launchpad.net/developmentseed/mapbox/ubuntu $DIST main
deb http://ppa.launchpad.net/chris-lea/node.js/ubuntu $DIST main
deb-src http://ppa.launchpad.net/chris-lea/node.js/ubuntu $DIST main
" > $LIST
# Mapbox key
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 0FB81AF3
# Node.js key
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C7917B12
apt-get install tilemill
Quantum GIS was also installed through apt-get. It's version QGIS 1.4.0-Enceladus
My OS is Ubuntu 11.04 with packages that are up-to-date as of today.
Okay, finally starting to look at this. The shapefile you posted above appears to work fine for me (no hangs). Didn't it hang for you?
Also, your QGIS version is quite old. It would be great if you would upgrade to the latest release and see if the problem persists.
Thanks for looking into this. I updated QGIS & Tilemill. They're now at 1.7.1-Wroclaw and 0.5.1.1-0ubuntu6.
But, the problem still persists. I made a new project, with a new shapefile, and got the same results. (You can download the project here.)
I tried the case both with & without the index, and got the same results:
After I restarted TileMill, the shape appeared.
I included a screen shot from this case, with two of the three shapes drawn. It's in the ZIP file.
k. thanks so much for the follow through. This catches me again on the road, but I'll give this proper testing attention as soon as possible - likely early next week and hopefully address for the 0.6.1 stable release.
update: I think that now that mapnik/mapnik#1022 is solved (mapnik exposes the ability to clear its shapefile cache, and tilemill now calls this function before map saving), this issue may be resolved.
However, stale index files will still be an issue. If you can test this again (once TileMill 0.9.1 is released) that would be great.
tracking the stale index issue at #1377, closing.
also clear the mapnik cache in the tile process - refs #1121,#714,#1210,
fix the request method - this finally fixes shapefile mapped memory c…
…learing - refs #1121,#714,#1210,#1278,#1335