Script to produce Python packages for a coordinated release #5
Conversation
Create Pypi packages from git repos for the coordinated GrimoireLab releases.
Now the script can create and populate virtual environments, with everything needed to build GrimoireLab pacakges. Some clean of messages, and arguments.
The old build_pypi.py is now build_grimoirelab. The program was restructured and completed a bit. Now works with repositories that produce several Python packages, or that produce packages of names different to the name of the repository. A README.md was also produced, explaining (a bit) the tool.
README.md
Outdated
| @@ -9,3 +9,7 @@ This is for stuff relevant to GrimoireLab as a whole. For example: | |||
| Coordinated releases are snapshots (specific commits) | |||
| of most of the GrimoireLab components that are expected to work together. | |||
| See more information in the [releases README.md file](releases/README.md). | |||
|
|
|||
| * Utlis (directory [utlis](utils)) for doing stuff relevant to GrimoireLab | |||
There was a problem hiding this comment.
Utlis must be Utils and utlis must be utils
There was a problem hiding this comment.
Not sure what you are asking for here. The first Utils is uppercase because it is just a noun starting a sentence. The second and third utils are just the name and links of the utlis directory.
There was a problem hiding this comment.
OK, now I realize. Sorry, I had not noticed the spelling error. I'm fixing it in the new commit.
| {'name': 'grimoire-kidash', 'dir': 'kidash'}], | ||
| 'sortinghat': None | ||
| } | ||
|
|
There was a problem hiding this comment.
Maybe default_commits and repos could me merged to avoid the same entries (git repositories) duplicated in two places? Trying to follow the DRY pattern (don't repeat yoursefl).
There was a problem hiding this comment.
Thanks for the comment, @acs. This is a good idea. I split the information in two dictionaries because of their different origins. In one case (default_commits) it was just a fallback for the case when the user does not define a coordinated release file. In the other (repos) it was "structural" information about repos and packages.
Both could be joined together in a single dictionary, but I would like to explore another option. That would be using the format in default_commits for the coordinated release. In any case, both dictionaries could be merged in future versions of this script.
This said, if you prefer, I can merge dictionaries in this pull request, that shouldn't be that difficult.
There was a problem hiding this comment.
Just go ahead if you are going to change it. The repositories are not going to change frequently and both dictionaries are pretty close in the code, to maintain them. And this is a script, so it won't grow too much.
utils/build_grimoirelab
Outdated
|
|
||
| args_actions = parser.add_argument_group("Actions", "Actions to perform") | ||
| args_actions.add_argument("--build", action='store_true', | ||
| help="Build packges. Is the default action.") |
utils/build_grimoirelab
Outdated
| args_actions.add_argument("--build", action='store_true', | ||
| help="Build packges. Is the default action.") | ||
| args_actions.add_argument("--install", action='store_true', | ||
| help="Install pacakges.") |
utils/build_grimoirelab
Outdated
| def run_command(args, cwd='/'): | ||
|
|
||
| result = subprocess.run(args, cwd=cwd, | ||
| stdout=subprocess.PIPE, stderr=subprocess.STDOUT) |
utils/build_grimoirelab
Outdated
| setup_file = os.path.join(pkg_dir, 'setup.py') | ||
| if os.path.isfile(setup_file): | ||
| run_command([python, 'setup.py', 'sdist', | ||
| '--dist-dir=' + dist_dir], |
utils/build_grimoirelab
Outdated
| return True | ||
| else: | ||
| logging.info("Directory " + pkg_dir + "does not have a setup.py file.") | ||
| return False |
utils/build_grimoirelab
Outdated
| commit = 'HEAD' | ||
| cloned = clone_git(repo=repo_link, dir=repo_dir, commit=commit) | ||
| built = build_pypi(pkg_dir, building_env=venv_dir, | ||
| dist_dir=dist_dir) |
|
@acs, please, let me know if the last commit fixes everything, and if you find my proposal of leaving those two dictionaries for now as acceptable. If not, just let me know and I will make that change too. |
I've written a little script to help in the process of creating the Python packages for a coordinated release. In its current form, it can produce all packages for the repos that already support Python packages, and install them in a virtual environment where they could be tested.
To avoid problems with the Python installation in the host where the packages are produced, a virtual environment with installed dependencies for building is produced, and packages are built in it.
The script accepts options to build or install, to select only a certain set of modules, to define directories for building and installing virtual environments, for the temporary repositories, for where to produce the Python packages, and to select a coordinated release file.