Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for template dir in conan new #7215

Merged
merged 8 commits into from Jun 29, 2020
Merged

Added support for template dir in conan new #7215

merged 8 commits into from Jun 29, 2020

Conversation

TheQwertiest
Copy link
Contributor

@TheQwertiest TheQwertiest commented Jun 19, 2020

Changelog: Feature: Added support for template dir in conan new.
Docs: conan-io/docs#1752

Implements/Fixes #6346

Added a new --template-dir argument to conan new.

Usage:

conan new [-md TEMPLATE_DIR, --template-dir TEMPLATE_DIR] name
--template-dir    Use the given template directory from the local cache to generate a conan project

Template directory functions similarly to --template TEMPLATE, but takes directory as input instead. All files in this directory, their paths and names are processed via jinja.

  • Refer to the issue that supports this Pull Request.
  • If the issue has missing info, explain the purpose/use case/pain/need that covers this Pull Request.
  • I've read the Contributing guide.
  • I've followed the PEP8 style guides for Python code.
  • I've opened another PR in the Conan docs repo to the develop branch, documenting this one.

Note: By default this PR will skip the slower tests and will use a limited set of python versions. Check here how to increase the testing level by writing some tags in the current PR body text.

@TheQwertiest
Copy link
Contributor Author

@TheQwertiest TheQwertiest commented Jun 19, 2020

Things to improve in the future:

  • --overwrite-policy {overwrite, ask, skip, abort}: this will allow to seamlessly add conan support to existing projects.
  • some sort of config file to list files to copy without rendering.

@TheQwertiest
Copy link
Contributor Author

@TheQwertiest TheQwertiest commented Jun 19, 2020

Implementation limitation: currently all files are rendered and stored in memory (i.e. not written immediately after rendering), which is kinda dirty, and might be problematic if there are really big files.

@TheQwertiest
Copy link
Contributor Author

@TheQwertiest TheQwertiest commented Jun 19, 2020

Hm... Now that I think about it... This functionality could be merged in the existing --template arg: if TEMPLATE is file - copy it, otherwise use the --template-dir behaviour. Should I do that?

conans/client/cmd/new.py Outdated Show resolved Hide resolved
conans/client/cmd/new.py Outdated Show resolved Hide resolved
Copy link
Member

@jgsogo jgsogo left a comment

This is one of the improvements I had in mind after the search table and the graph 😉 did you read my mind?

I'd use the same --template argument, as you say it is perfectly compatible.

Probably we should use an Environment to load these templates, that's something I have also in mind but it will require more work (and some refactor), for this PR adding the feature is enough.

@TheQwertiest
Copy link
Contributor Author

@TheQwertiest TheQwertiest commented Jun 23, 2020

I'd use the same --template argument, as you say it is perfectly compatible.

The only confusing(?) thing about it would be that:

  • --template=file.py: copies file.py as conanfile.py to the current directory.
  • --template=dir: copies content of dir to the current directory and not dir itself,

Would that be alright?

@TheQwertiest
Copy link
Contributor Author

@TheQwertiest TheQwertiest commented Jun 23, 2020

Collapsed template-dir in template

- collapsed `template-dir` in `template`
- added `conan_version` rendering
- moved templates to `cmd_new`
Copy link
Member

@jgsogo jgsogo left a comment

Asking for some changes, but overall I really like it. Still I need to clone the branch and play with the feature, but 👌

conans/client/cmd/new.py Show resolved Hide resolved
conans/client/cmd/new.py Show resolved Hide resolved
conans/client/command.py Outdated Show resolved Hide resolved

def _get_files_from_template_dir(template_dir, name, version, package_name):
if not os.path.isfile(os.path.join(template_dir, "conanfile.py")):
raise ConanException("Template is missing 'conanfile.py': {}".format(template_dir))
Copy link
Member

@jgsogo jgsogo Jun 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, we shouldn't add this check, the conanfile can take any other name, or maybe there is a conanfile.txt...

Copy link
Contributor Author

@TheQwertiest TheQwertiest Jun 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would really like to keep this type of check to prevent the following scenario: user makes a typo or template dir is malformed (e.g. conan config install did not go through), so the command conan new --template=T finishes successfully, but there are no files or worse a whole load of random files from the wrong location (the classic cp -rf / path mistake).

Copy link
Member

@jgsogo jgsogo Jun 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If use enters a typo in conan new --template=<typo> it will fail because the folder/file doesn't exist. If there was a problem during the conan config install then, after this command runs successfully, you will realize about it in your working dir: missing files, a project that doesn't work...

If you are worried about someone entering a relative path like conan new --template=../../<typo>, we can limit the search in the cache to folders inside the <cache>/templates/cmd_new/ directory. It will work if it is found locally ($(pwd)/../../<typo>), but looking in the cache will raise.

Copy link
Contributor Author

@TheQwertiest TheQwertiest Jun 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, --template accepts absolute paths as well. So, conan new --template=/ path will work (it will take huge time to complete though :D). This check will also prevent users from (ab)using templates as a storage for random files.

Copy link
Member

@jgsogo jgsogo Jun 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that was my opinion, but we can wait for others to review and add their two cents 👍

Copy link
Contributor Author

@TheQwertiest TheQwertiest Jun 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, at the end of the day, you are the dev.
I.e. if you think that this check is not worth having then it's gotta go =)

conans/client/cmd/new.py Outdated Show resolved Hide resolved
conans/client/cmd/new.py Outdated Show resolved Hide resolved
TheQwertiest and others added 3 commits Jun 24, 2020
Co-authored-by: Javier G. Sogo <jgsogo@gmail.com>
Co-authored-by: Javier G. Sogo <jgsogo@gmail.com>
@TheQwertiest
Copy link
Contributor Author

@TheQwertiest TheQwertiest commented Jun 24, 2020

@jgsogo , btw are conan docs generated dynamically (e.g. https://docs.conan.io/en/latest/reference/commands/creator/new.html)?
In other words, do I have to edit command output manually (by replacing it with the output of conan new --help) or do I leave it as is?

@jgsogo
Copy link
Member

@jgsogo jgsogo commented Jun 24, 2020

The output is generated automatically with a script. We run it manually, but it is scripted (https://github.com/conan-io/docs/blob/develop/commands_help_update.py).

@jgsogo jgsogo self-assigned this Jun 24, 2020
@jgsogo jgsogo requested a review from memsharded Jun 24, 2020
conans/client/conan_api.py Outdated Show resolved Hide resolved
conans/client/cmd/new.py Outdated Show resolved Hide resolved
@TheQwertiest
Copy link
Contributor Author

@TheQwertiest TheQwertiest commented Jun 24, 2020

The output is generated automatically with a script. We run it manually

Uhm, so do I need to run it when making a doc PR? Or is it only invoked right before making a release?

@memsharded memsharded added this to the 1.27 milestone Jun 24, 2020
@jgsogo
Copy link
Member

@jgsogo jgsogo commented Jun 25, 2020

The output is generated automatically with a script. We run it manually

Uhm, so do I need to run it when making a doc PR? Or is it only invoked right before making a release?

Better when making the PR, but TBH, we usually forget about it and we run the script just before making a release 😅

Copy link
Member

@memsharded memsharded left a comment

This is great, thanks very much.

I would like to merge it on Monday to be included in next 1.27. There is just a minor thing to improve with the template path, if you can address it by then, it would be great, if not we can do it ourselves so we don't delay this contribution any more.

Thanks again for your contribution!

conans/client/cmd/new.py Outdated Show resolved Hide resolved
jgsogo
jgsogo approved these changes Jun 29, 2020
Copy link
Member

@jgsogo jgsogo left a comment

💯 !

We need docs for this new feature, they should document the fields that can be used in the template and include an example. Let us know if you want to write them ourselves or you will do it.

Thanks!

@TheQwertiest
Copy link
Contributor Author

@TheQwertiest TheQwertiest commented Jun 29, 2020

Done: conan-io/docs#1752
Note the note in that PR about cmd line output (pun intended :P)

@memsharded
Copy link
Member

@memsharded memsharded commented Jun 29, 2020

Great, thanks for this contribution!

@memsharded memsharded merged commit 6bddb45 into conan-io:develop Jun 29, 2020
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants