Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Reduce cost of multiple req2param calls within the same request #165

Closed
strk opened this Issue · 2 comments

1 participant

@strk

As reported in CartoDB/Windshaft#157 there are many possible calls to req2param in a single request (when it's a request to create a map).

This ticket is to see if it's really needed for those multiple calls to perform the same work or if they could just do nothing on subsequent calls.

Given the work of req2params is to fill up req.params, one idea could be to store a "signature" of that object somewhere in a "protected space" and check if the signature is available at the time of a second call.

@strk strk added the enhancement label
@strk strk added this to the 1.8.2 milestone
@strk strk self-assigned this
@strk

This is a trace of where multiple calls come from during a POST to /layergroup with a MapConfig of 2 layers:

req2params call 0 from:
    at Object.module.exports.me.req2params (/usr/src/node/apps/Windshaft-cartodb/Windshaft-cartodb/lib/cartodb/server_options.js:663:21)
    at HTTPServer.test.only.serverOptions.req2params (/usr/src/node/apps/Windshaft-cartodb/Windshaft-cartodb/test/acceptance/multilayer.js:1161:33)
    at Function.setupParams (/usr/src/node/modules/Windshaft/lib/windshaft/server.js:1105:21)
Error: req2params call 1 from:
    at Object.module.exports.me.req2params (/usr/src/node/apps/Windshaft-cartodb/Windshaft-cartodb/lib/cartodb/server_options.js:663:21)
    at HTTPServer.test.only.serverOptions.req2params (/usr/src/node/apps/Windshaft-cartodb/Windshaft-cartodb/test/acceptance/multilayer.js:1161:33)
    at Function.module.exports.app.getTileOrGrid.renderer (/usr/src/node/modules/Windshaft/lib/windshaft/server.js:525:21)
    at next (/usr/src/node/modules/Windshaft/node_modules.download.node0.8/step/lib/step.js:51:23)
    at Step (/usr/src/node/modules/Windshaft/node_modules.download.node0.8/step/lib/step.js:122:3)
    at HTTPServer.module.exports.app.getTileOrGrid (/usr/src/node/modules/Windshaft/lib/windshaft/server.js:523:9)
    at HTTPServer.module.exports.app.tryFetchTileOrGrid (/usr/src/node/modules/Windshaft/lib/windshaft/server.js:826:13)
req2params call 2 from:
    at Object.module.exports.me.req2params (/usr/src/node/apps/Windshaft-cartodb/Windshaft-cartodb/lib/cartodb/server_options.js:663:21)
    at HTTPServer.test.only.serverOptions.req2params (/usr/src/node/apps/Windshaft-cartodb/Windshaft-cartodb/test/acceptance/multilayer.js:1161:33)
    at Function.module.exports.app.getTileOrGrid.renderer (/usr/src/node/modules/Windshaft/lib/windshaft/server.js:525:21)
    at next (/usr/src/node/modules/Windshaft/node_modules.download.node0.8/step/lib/step.js:51:23)
    at Step (/usr/src/node/modules/Windshaft/node_modules.download.node0.8/step/lib/step.js:122:3)
    at HTTPServer.module.exports.app.getTileOrGrid (/usr/src/node/modules/Windshaft/lib/windshaft/server.js:523:9)
    at HTTPServer.module.exports.app.tryFetchTileOrGrid (/usr/src/node/modules/Windshaft/lib/windshaft/server.js:826:13)
Error: req2params call 3 from:
    at Object.module.exports.me.req2params (/usr/src/node/apps/Windshaft-cartodb/Windshaft-cartodb/lib/cartodb/server_options.js:663:21)
    at HTTPServer.test.only.serverOptions.req2params (/usr/src/node/apps/Windshaft-cartodb/Windshaft-cartodb/test/acceptance/multilayer.js:1161:33)
    at Function.module.exports.app.getTileOrGrid.renderer (/usr/src/node/modules/Windshaft/lib/windshaft/server.js:525:21)
    at next (/usr/src/node/modules/Windshaft/node_modules.download.node0.8/step/lib/step.js:51:23)
    at Step (/usr/src/node/modules/Windshaft/node_modules.download.node0.8/step/lib/step.js:122:3)
    at HTTPServer.module.exports.app.getTileOrGrid (/usr/src/node/modules/Windshaft/lib/windshaft/server.js:523:9)
    at HTTPServer.module.exports.app.tryFetchTileOrGrid (/usr/src/node/modules/Windshaft/lib/windshaft/server.js:826:13)

Those are the very first call + one call for each "tested" tile: a PNG tile + 2 UTF8GRID tiles.

Here's the layergroup config:

      var layergroup =  {
        version: '1.0.0',
        layers: [
           { options: {
               sql: 'select cartodb_id, ST_Translate(the_geom_webmercator, 5e6, 0) as the_geom_webmercator from test_table limit 2',  
               cartocss: '#layer { marker-fill:red; marker-width:32; marker-allow-overlap:true; }',
               cartocss_version: '2.0.1',
               interactivity: 'cartodb_id'
             } },
           { options: {
               sql: 'select cartodb_id, ST_Translate(the_geom_webmercator, -5e6, 0) as the_geom_webmercator from test_table limit 2 offset 2',
               cartocss: '#layer { marker-fill:blue; marker-allow-overlap:true; }',
               cartocss_version: '2.0.2',
               interactivity: 'cartodb_id'
             } }
        ]
      };

\cc @rafacas @javisantana

@strk

So, one way to call req2params again while fetching tiles is because that method does the splitting of the maptoken between actual map identifier and the last_updated portion.
That splitting is really only needed IF the compound token is sent, which is the case with current Windshaft version. The compound token is the one that is returned by the afterLayergroupCreate hook.

We could maybe handle that specially and only do the splitting... have to think a bit more about that from the point of view of maintainability. If windshaft used the pre-hook token there might be not even any need to call req2params at all...

@strk strk modified the milestone: 1.8.3, 1.8.2
@strk strk added the performance label
@strk strk closed this issue from a commit
@strk strk Upgrades windshaft to 0.19.1 with many performance improvements
Among others:

- Improve speed of instanciating a map
- Give meaningful error on attempts to use map tokens with
  attribute service

Closes #156 -- CDB-1796 #resolve
Closes #147
Closes #159
Closes #165
cf01f01
@strk strk closed this in cf01f01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.