Add new Managing Python Pipeline Dependencies page#147
Add new Managing Python Pipeline Dependencies page#147melap wants to merge 4 commits intoapache:asf-sitefrom melap:python2
Conversation
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id 0b74641 with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id e051736 with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
|
||
| The runner will use the `requirements.txt` file to install your additional dependencies onto the remote workers. | ||
|
|
||
| **Important:** Remote workers will install all packages listed in the `requirements.txt` file. Because of this, it's very important that you delete non-PyPI packages from the `requirements.txt` file, as stated in step 2. If you don't remove non-PyPI packages, the remote workers attempt to install packages from sources that are unknown to them, and an error occurs. |
There was a problem hiding this comment.
The final part of the sentence is confusing a little bit:
the remote workers attempt to install packages from sources that are unknown to them, and an error occurs
maybe ?
The remote workers will fail while attempting to install packages from sources unknown to them.
|
|
||
| ## <a name="multfiles"></a>Multiple File Dependencies | ||
|
|
||
| Often, your pipeline code spans multiple files. To run your project remotely, you must group these files as a Python package. When remote workers spin up, they look for this kind of package in the staging location and install it. To group your files as a Python package and make it available remotely, perform the following steps: |
There was a problem hiding this comment.
start instead of spin up
they look for this kind of package in the staging location and install it. -> they will install the Python package specified using this method.
|
|
||
| --setup_file /path/to/setup.py | ||
|
|
||
|
|
| **Note:** If you [created a requirements.txt file](#pypi) and your project spans multiple files, you can get rid of the `requirements.txt` file and instead, add all packages contained in `requirements.txt` to the `install_requires` field of the setup call (in step 1). | ||
|
|
||
|
|
||
| ## <a name="nonpython"></a>Non-Python Dependencies |
There was a problem hiding this comment.
I think this section and the next one (PyPI Dependencies with Non-Python Dependencies) can be merged.
We can remove this section and change the title of the next one to Non-Python Dependencies and PyPI Dependencies with Non-Python Dependencies. (Also maybe some adjustment to the first sentence to cover both cases.)
There was a problem hiding this comment.
did some tweaking, let me know what you think.
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id 0315d77 with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id 51202df with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
side note: I addressed the (unrelated to this PR) broken links in PR 151 |
|
LGTM. Thank you. |
3 participants
R: @aaltay