Race condition with external resources

[strk]

Sometimes the localized external resources needed by CartoCSS are removed from disk before their corresponding style expires.

This seems to be due to the way filesystem path for the resources is determined.
Originally reported here:
https://groups.google.com/forum/#!topic/cartodb/m9t4iFdM-ok/discussion

Jira ref: https://cartodb.atlassian.net/browse/CDB-67

[saleiva]

Hi @strk I don't think that's the problem.
Without touching the original resource applying the same cartocss to a map several times retrieves some errors randomly.

[strk]

I tried locally and it looks like applying the same cartocss to a map doesn't really send any request to the tiler from the browser. Instead the browser performs a PUT to the maps API (rails). Indeed I do see a request to the deprecated API arriving to the tiler from the rails API host, so the problem could be there. Can you confirm there's no request to tiler from the browser on later requests ?

[strk]

I confirm there's no request from browser. Tried online. The call to deprecated API is really triggered by the PUT to maps api.

[strk]

So we have two options here:

• Fix the deprecated API trying to resolve the race condition
• Stop using the deprecated API from rails app (Rails using old tiler API cartodb#198)

[strk]

I'm seeing more weird things here. Specifically I'm seeing multilayer map styles in redis that reference resources downloaded to per-table cache dirs. This may very well be the cause of our troubles. It could be that while copying the style from a so-called "table" we end up getting the localized path rather than the original path, which is then unguaranteed as per lifetime. \cc @lorenzoplanas @javisantana

[strk]

@lorenzo, what about avoiding to POST to /style if the style didn't change ? According to @saleiva the problem can only be reproduced by continuously applying the same style, so avoiding the override might fix it and generally improve interaction speed. In any case, I cannot reproduce this bug :(

[strk]

So it looks to me that millstone is really not meant to be used the way we are, that is with different configurations within the same process. Look here:
https://github.com/mapbox/millstone/blob/v0.6.0/lib/millstone.js#L72

That global has is used to keep track of "in progress" downloads.
When more requests come in for the same url, the second request is put in a queue and called when download completes:
https://github.com/mapbox/millstone/blob/v0.6.0/lib/millstone.js#L104

Now I suspect that this means accepting the local path for the first request and discarding the one specified in the second. I filed a ticket to that extent: tilemill-project/millstone#105

[strk]

I don't see any easy way to force millstone to use local variables for those caches. This means we either fork millstone to avoid the cache or we change our approach to use a single sandbox with all localized resources together. If we take the "single sandbox" approach we'll have a problem deciding when it is safe to cleanup elements in the cache.

[strk]

I've sent a pull request for millstone, but it would be helpful if anyone could confirm the pull fixes the bug (I can't handle to reproduce myself): tilemill-project/millstone#107

[strk]

Testing this involves installing millstone from our fork: https://github.com/CartoDB/millstone/tree/master-sandboxes
millstone is a dependency of grainstore \cc @javisantana

[strk]

Pull request was merged, so it's a matter of testing against millstone master now

[javisantana]

will try it asap

[strk]

tested, finally ?

[strk]

I've added a test within grainstore: CartoDB/grainstore#52

[strk]

Reopening, as there might be a problem with the new millstone version:
https://travis-ci.org/CartoDB/Windshaft/builds/10977881#L1830

[strk]

never mind, it was a time-related issue.will raise the timeouts