Flask Migration Roadmap

Adrià Mercader edited this page Sep 29, 2016 · 3 revisions
Clone this wiki locally

Note: As ever please provide any feedback or comments on the mailing list or this issue.

As of 26th September 2016 the Flask migration work is well underway, with code awaiting review for most of the foundation work needed as well as the API blueprint. There's still plenty to do though, these are the tasks sorted roughly in the order they should happen, although many of them could be work on simultaneously.

The following is a rough potential timeline with tasks matched to actual release. This is likely to be reviewed following discussions with the rest of the tech team and availability of resources:

Timeline (Bigger here)


  • Cross-app Template Rendering / Caching

    We need to support template rendering in a similar way that other common objects have been wrapped to work both on Pylons and Flask. Render functions are located in ckan.lib.base, and the main thing that needs to be investigated is caching. There is logic to set caching headers on the render function (on a response object that Flask does not have), so we need to check if that is being used and how to replicate it Flask. On a related note CacheMiddleware, PageCacheMiddleware are being added to the Pylons middleware stack, we need to check if these are used or useful and if so, add them to the Flask stack. A temporary workaround was added to the API Blueprint PR to allow template rendering on Flask (99911), but it needs to be properly investigated.

  • Reference Frontend controller

    As with the API blueprint, this will be the first frontend controller to be migrated to Flask blueprint. It doesn't need to be particularly complex but it should involve all (or most) functionality used by the frontend. Good candidates are the home or tag controllers, both with simple methods that involve rendering templates. This will allow us to identify issues that need fixing, like the use we do of the Pylons router to store styling configuration (ckan_icon, highlight_actions).

  • Replace error controller with error handlers

    Frontend controllers will make use of error pages, so we need to support the functionality of displaying error pages properly on Flask. This is currently done in Pylons with the ErrorHandler and StatusCodeRedirect middlewares and the error controller. The alternative in Flask are error handlers.

  • Documentation on migrating controllers

    Short documentation (maybe on the wiki page) describing the expected structure, conventions, how to register blueprints, what objects to use, etc. Mention passing explicit variables to templates and to not use c or g for that.

  • Migrate priority controllers

    These critical controllers are the ones more likely to affect extensions and require more work during the actual migration so it makes sense to tackle them first. A priority should be to make sure that extension interfaces like IDatasetForm / IGroupForm work as expected. All work done until now seems to suggest that there shouldn't be major issues but of course that won't be known until work starts.

    • package
    • group / organization
    • user
  • Migrate secondary controllers

    These controllers are less critical and much simpler than the priority ones. They are good candidates for developers that hadn't been involved in the migration so far to pick up at the beginning. Some like the feed one doesn't even involve templare rendering.

    • admin
    • feed
    • revision
    • util
    • home
    • tag
  • Deprecate controllers:

    The migration process will be the perfect time to deprecate and remove these old controllers. The storage one relates to the FileStore used on CKAN older than 2.2.

    • storage
    • template
  • Documentation for extension maintainers on how to upgrade

    It is really critical to have extensive documentation for extension maintainers on how to upgrade. This is likely to get more clear the more we start migrating actual controllers and extensions, but it would be great to publish it early so developers can start planning. Points it should cover:

    • Use of toolkit, no direct imports. ckantoolkit for multiple CKAN versions support
    • How to replace IRoutes + controllers with IBlueprint (also supporting both interfaces)
    • New url_for syntax
    • New url_for implications when writing tests (use test_request_context, etc)
  • Remove Pylons dependency for running tests

    Right now we are running our tests with the --with-pylons option. This loads the Pylons test plugin for nose, which essentially starts an instance of the application and registers some objects like the translators. All this should be migrated to a CKAN plugin (--with-ckan) that calls load_environment where appropiate (or make_app if we really need to create a full stack).

    Many legacy tests rely on Pylons classes or methods, we need to decide what to do with those, either rewrite them or remove them.

  • Remove Pylons controllers, other Pylons code and requirements

    Once the migration has been completed we can start thinking about removing the actual Pylons code and requirements. This would need to strike a balance between backwards compatible support and new features development.


  • Remove Paster
  • Remove Fanstatic
  • ...