Extract `cookiecutter` and `django` extensions into separated packages

[abravalheri]

Since we don't need extensions to live inside the same code repository anymore, we could extract some of the complex ones into separated repositories.

Those two in particular already depend on extra requirements to work anyway.
So instead of doing pip install pyscaffold[all], the users that want the functionalities would need to do pip install pyscaffold pyscaffoldext-django or pip install pyscaffold pyscaffoldext-cookiecutter, which is bigger, but at least we clean up the code base and speed up the tests...

Any thoughts?

[FlorianWilhelm]

This seems to be great idea since it would reduce the complexity inside PyScaffold. What about potential consequences? For instance how would we do unit testing? I guess it's best to also move those to a Travis job for the plugin. What about the documentation? I think it would be great to have a small explanation of official extensions directly on our main website.

Do you think this is something for version 3.1 or should we rather postpone it to version 3.2?

[abravalheri]

Thanks for the feedback @FlorianWilhelm :)

In my view, we should move the relevant unit/system testing to the new repositories and create Travis jobs for them as well.

The main (bad) consequence that I can imagine, is having changes in PyScaffold (e.g. from PRs) that break those extensions. Right now, since the test suite is unified, any breaking change would make PyScaffold Travis job to fail, and therefore would not be merged.

If we split, we should find our way of trigger builds in the other repositories and collect the feedback before merging. Of course this is a matter of confidence in PyScaffold test suite, if the extension API is well tested, the separation would ideally not be a concern, and this downfall trigger could be neglected.

Regarding the docs, I think we should have dedicated rtds, but mention the features in summary, reference the extensions and link their docs in the main page, so it is more or less transparent for the end user.

Versions, on the other hand, are something more polemic... How PyScaffold is doing semver ? Are we considering versions .0. to be unstable in the same way semver advocates about 0..? Do we consider this separation a break in the API?

(This question about semver is also related to other topics such as replacing the term "extension" by "plugin" - we can do those changes without any problem, is just that we have to plan the releases and have a deprecation period)

[FlorianWilhelm]

@abravalheri I am totally in favour of semantic versioning. This is why we should wait for version 4.0 to remove those two extensions since it would break the current API.

For the official supported extensions we need to make sure that they depend on compatible versions of PyScaffold. When we do semantic versioning we should be able to say e.g. pyscaffold>=3.0a0<4.0a0 in an extension's install_requires and pip should resolve everything correctly as long as we stick with semantic versioning. Regarding 0.x versions, I would say that an official extension is only considered stable when it's 1.x or higher otherwise anything can break at any time like stated in SemVer.

Regarding the term extension I think we can also just keep it. If I remember correctly I was preferring plugin but ... well it's not much of a difference and it seems we have already settled on extension. What do you think? Should we just keep it?

[abravalheri]

Let's do like that (I am completely aligned)

I will prepare some warning messages about the deprecation in v4.0.

Regarding extension vs plugin, my only reason to prefer "extension" is that pyscaffoldext is slightly shorter then pyscaffoldplug :)

[abravalheri]

Just to report some progress:

Since we have moved version (I think), we can start working on deprecations and changes of API!
Therefore I have created 2 new repositories: pyscaffold/pyscaffoldext-cookiecutter and pyscaffold/pyscaffoldext-django
I still have to work a bit in the documentation (by porting stuff from PyScaffold itself and wire the doc pages properly), create the readthedocs, upload a package to the staging index of PyPI, etc ...
On top of that it is necessary to remove old stuff from the main repository.
Last remark, for the moment the tests on the extension repos will fail -- argparse doesn't like repeated options.

[FlorianWilhelm]

Thanks for your work @abravalheri! I created a branch v4.0.x where all changes to PyScaffold itself can go during this transition. I would keep the master for the time being to track bugfix changes for the current version 3.2. Later on we just merge v4.0.x before a release.

[abravalheri]

Hi @FlorianWilhelm , I added the commits directly to the new branch, since they are more or less straightforward. Please let me know if you prefer PR's instead (I can revert the commits).

If I have the time to work in the other open issues for v4, I think I will use the same methodology, and submit PR's when the topic is more controversial or if we have not discussed the implementation details before...

[FlorianWilhelm]

Hi @abravalheri, thank you, that's perfectly fine, just commit to the new branch and a PR on controversial things.

Let me know if I should release some alpha version of the v4.0 branch to test the actual extracted extensions. With the help of the versioning we can make sure that those extensions will only work with 4.x releases.