Flask Migration Roadmap
Note: As ever please provide any feedback or comments on the mailing list or this issue.
As of 26th September 2016 the Flask migration work is well underway, with code awaiting review for most of the foundation work needed as well as the API blueprint. There's still plenty to do though, these are the tasks sorted roughly in the order they should happen, although many of them could be work on simultaneously.
The following is a rough potential timeline with tasks matched to actual release. This is likely to be reviewed following discussions with the rest of the tech team and availability of resources:
(Bigger here)
-
Cross-app Template Rendering / Caching
We need to support template rendering in a similar way that other common objects have been wrapped to work both on Pylons and Flask. Render functions are located in
ckan.lib.base
, and the main thing that needs to be investigated is caching. There is logic to set caching headers on the render function (on aresponse
object that Flask does not have), so we need to check if that is being used and how to replicate it Flask. On a related noteCacheMiddleware
,PageCacheMiddleware
are being added to the Pylons middleware stack, we need to check if these are used or useful and if so, add them to the Flask stack. A temporary workaround was added to the API Blueprint PR to allow template rendering on Flask (99911), but it needs to be properly investigated. -
Reference Frontend controller
As with the API blueprint, this will be the first frontend controller to be migrated to Flask blueprint. It doesn't need to be particularly complex but it should involve all (or most) functionality used by the frontend. Good candidates are the home or tag controllers, both with simple methods that involve rendering templates. This will allow us to identify issues that need fixing, like the use we do of the Pylons router to store styling configuration (ckan_icon
, highlight_actions
).
- Replace error controller with error handlers
Frontend controllers will make use of error pages, so we need to support the functionality of displaying error pages properly on Flask. This is currently done in Pylons with the ErrorHandler
and StatusCodeRedirect
middlewares and the error controller. The alternative in Flask are error handlers.
- Documentation on migrating controllers
Short documentation (maybe on the wiki page) describing the expected structure, conventions, how to register blueprints, what objects to use, etc. Mention passing explicit variables to templates and to not use c
or g
for that.
- Migrate priority controllers
These critical controllers are the ones more likely to affect extensions and require more work during the actual migration so it makes sense to tackle them first. A priority should be to make sure that
extension interfaces like IDatasetForm
/ IGroupForm
work as expected. All work done until now seems to suggest that there shouldn't be major issues but of course that won't be known until work starts.
- [ ] package
- [ ] group / organization
- [ ] user
- Migrate secondary controllers
These controllers are less critical and much simpler than the priority ones. They are good candidates for developers that hadn't been involved in the migration so far to pick up at the beginning. Some like the feed one doesn't even involve templare rendering.
- [ ] admin
- [ ] feed
- [ ] revision
- [ ] util
- [ ] home
- [ ] tag
- Deprecate controllers:
The migration process will be the perfect time to deprecate and remove these old controllers. The storage one relates to the FileStore used on CKAN older than 2.2.
- [ ] storage
- [ ] template
- Documentation for extension maintainers on how to upgrade
It is really critical to have extensive documentation for extension maintainers on how to upgrade. This is likely to get more clear the more we start migrating actual controllers and extensions, but it would be great to publish it early so developers can start planning. Points it should cover:
-
Use of toolkit, no direct imports. ckantoolkit for multiple CKAN versions support
-
How to replace
IRoutes
+ controllers withIBlueprint
(also supporting both interfaces) -
New
url_for
syntax -
New
url_for
implications when writing tests (usetest_request_context
, etc) -
Remove Pylons dependency for running tests
Right now we are running our tests with the --with-pylons
option. This loads the Pylons test plugin for nose, which essentially starts an instance of the application and registers some objects like the translators. All this should be migrated to a CKAN plugin (--with-ckan
) that calls load_environment
where appropiate (or make_app
if we really need to create a full stack).
Many legacy tests rely on Pylons classes or methods, we need to decide what to do with those, either rewrite them or remove them.
- Remove Pylons controllers, other Pylons code and requirements
Once the migration has been completed we can start thinking about removing the actual Pylons code and requirements. This would need to strike a balance between backwards compatible support and new features development.
Future
- Remove Paster
- Remove Fanstatic
- ...