Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update hook for pyarrow #662

Merged
merged 4 commits into from
Nov 10, 2023
Merged

Update hook for pyarrow #662

merged 4 commits into from
Nov 10, 2023

Conversation

n-elie
Copy link
Contributor

@n-elie n-elie commented Nov 10, 2023

No description provided.

@rokm
Copy link
Member

rokm commented Nov 10, 2023

This will pull in pyarrow.tests, which we definitely do not want:

Example

>>> import pprint
>>> pprint.pprint(collect_submodules('pyarrow'))
193 WARNING: Failed to collect submodules for 'pyarrow.tests.parquet' because importing 'pyarrow.tests.parquet' raised: ModuleNotFoundError: No module named 'pytest'
['pyarrow',
 'pyarrow._acero',
 'pyarrow._compute',
 'pyarrow._compute_docstrings',
 'pyarrow._csv',
 'pyarrow._dataset',
 'pyarrow._dataset_orc',
 'pyarrow._dataset_parquet',
 'pyarrow._dataset_parquet_encryption',
 'pyarrow._feather',
 'pyarrow._flight',
 'pyarrow._fs',
 'pyarrow._gcsfs',
 'pyarrow._generated_version',
 'pyarrow._hdfs',
 'pyarrow._hdfsio',
 'pyarrow._json',
 'pyarrow._orc',
 'pyarrow._parquet',
 'pyarrow._parquet_encryption',
 'pyarrow._pyarrow_cpp_tests',
 'pyarrow._s3fs',
 'pyarrow._substrait',
 'pyarrow.acero',
 'pyarrow.benchmark',
 'pyarrow.cffi',
 'pyarrow.compute',
 'pyarrow.conftest',
 'pyarrow.csv',
 'pyarrow.cuda',
 'pyarrow.dataset',
 'pyarrow.feather',
 'pyarrow.filesystem',
 'pyarrow.flight',
 'pyarrow.fs',
 'pyarrow.hdfs',
 'pyarrow.interchange',
 'pyarrow.interchange.buffer',
 'pyarrow.interchange.column',
 'pyarrow.interchange.dataframe',
 'pyarrow.interchange.from_dataframe',
 'pyarrow.ipc',
 'pyarrow.json',
 'pyarrow.jvm',
 'pyarrow.lib',
 'pyarrow.libarrow_python',
 'pyarrow.libarrow_python_flight',
 'pyarrow.orc',
 'pyarrow.pandas_compat',
 'pyarrow.parquet',
 'pyarrow.parquet.core',
 'pyarrow.parquet.encryption',
 'pyarrow.substrait',
 'pyarrow.tests',
 'pyarrow.tests.arrow_16597',
 'pyarrow.tests.arrow_7980',
 'pyarrow.tests.conftest',
 'pyarrow.tests.interchange',
 'pyarrow.tests.interchange.test_conversion',
 'pyarrow.tests.interchange.test_interchange_spec',
 'pyarrow.tests.pandas_examples',
 'pyarrow.tests.pandas_threaded_import',
 'pyarrow.tests.read_record_batch',
 'pyarrow.tests.strategies',
 'pyarrow.tests.test_acero',
 'pyarrow.tests.test_adhoc_memory_leak',
 'pyarrow.tests.test_array',
 'pyarrow.tests.test_builder',
 'pyarrow.tests.test_cffi',
 'pyarrow.tests.test_compute',
 'pyarrow.tests.test_convert_builtin',
 'pyarrow.tests.test_cpp_internals',
 'pyarrow.tests.test_csv',
 'pyarrow.tests.test_cuda',
 'pyarrow.tests.test_cuda_numba_interop',
 'pyarrow.tests.test_cython',
 'pyarrow.tests.test_dataset',
 'pyarrow.tests.test_dataset_encryption',
 'pyarrow.tests.test_deprecations',
 'pyarrow.tests.test_exec_plan',
 'pyarrow.tests.test_extension_type',
 'pyarrow.tests.test_feather',
 'pyarrow.tests.test_filesystem',
 'pyarrow.tests.test_flight',
 'pyarrow.tests.test_flight_async',
 'pyarrow.tests.test_fs',
 'pyarrow.tests.test_gandiva',
 'pyarrow.tests.test_gdb',
 'pyarrow.tests.test_hdfs',
 'pyarrow.tests.test_io',
 'pyarrow.tests.test_ipc',
 'pyarrow.tests.test_json',
 'pyarrow.tests.test_jvm',
 'pyarrow.tests.test_memory',
 'pyarrow.tests.test_misc',
 'pyarrow.tests.test_orc',
 'pyarrow.tests.test_pandas',
 'pyarrow.tests.test_scalars',
 'pyarrow.tests.test_schema',
 'pyarrow.tests.test_sparse_tensor',
 'pyarrow.tests.test_strategies',
 'pyarrow.tests.test_substrait',
 'pyarrow.tests.test_table',
 'pyarrow.tests.test_tensor',
 'pyarrow.tests.test_types',
 'pyarrow.tests.test_udf',
 'pyarrow.tests.test_util',
 'pyarrow.tests.util',
 'pyarrow.types',
 'pyarrow.util',
 'pyarrow.vendored',
 'pyarrow.vendored.docscrape',
 'pyarrow.vendored.version']

Copy link
Member

@rokm rokm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the very least pyarrow.tests should be filtered out, e.g.:

hiddenimports = collect_submodules('pyarrow', filter=lambda x: not x.startswith('pyarrow.tests'))

@n-elie n-elie requested a review from rokm November 10, 2023 09:53
@n-elie
Copy link
Contributor Author

n-elie commented Nov 10, 2023

My bad, I updated the PR according to your suggestions

@n-elie n-elie requested a review from rokm November 10, 2023 10:47
@rokm
Copy link
Member

rokm commented Nov 10, 2023

And now for the administrative part: please add a news fragment describing the change for the changelog (the update category, and something along the lines of "Update hook for pyarrow to collect all of the package's submodules.").

Then we should be good to merge.

@n-elie
Copy link
Contributor Author

n-elie commented Nov 10, 2023

Thanks for your help

@bwoodsend bwoodsend merged commit 3fd6f13 into pyinstaller:master Nov 10, 2023
2 checks passed
github-actions bot pushed a commit to wxx9248/CIS-Game-Project-2023W that referenced this pull request Dec 20, 2023
…23.11 (#82)

Bumps
[pyinstaller-hooks-contrib](https://github.com/pyinstaller/pyinstaller-hooks-contrib)
from 2023.10 to 2023.11.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/releases">pyinstaller-hooks-contrib's
releases</a>.</em></p>
<blockquote>
<h2>2023.11</h2>
<p>Please see the <a
href="https://www.github.com/pyinstaller/pyinstaller-hooks-contrib/tree/master/CHANGELOG.rst">changelog</a>
for more details</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/blob/master/CHANGELOG.rst">pyinstaller-hooks-contrib's
changelog</a>.</em></p>
<blockquote>
<h2>2023.11 (2023-12-20)</h2>
<p>New hooks</p>
<pre><code>
* Add a hook for ``freetype`` that collects the shared library that is
bundled with ``freetype-py`` PyPI wheels.
(`[#674](pyinstaller/pyinstaller-hooks-contrib#674)

&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/674&gt;`_)
* Add a hook for ``z3c.rml`` that collects the required subset of
Bitstream
Vera TTF fonts from the ``reportlab`` package.
(`[#674](pyinstaller/pyinstaller-hooks-contrib#674)

&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/674&gt;`_)
* Add hook for ``eth_rlp``.
(`[#672](pyinstaller/pyinstaller-hooks-contrib#672)

&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/672&gt;`_)
* Add hook for ``eth_typing`` which requires its package metadata.
(`[#656](pyinstaller/pyinstaller-hooks-contrib#656)

&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/656&gt;`_)
* Add hook for ``eth_utils`` to collect its embedded JSON files.
(`[#656](pyinstaller/pyinstaller-hooks-contrib#656)

&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/656&gt;`_)
* Add hook for ``rlp``.
(`[#672](pyinstaller/pyinstaller-hooks-contrib#672)

&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/672&gt;`_)
* Add hook for ``sspilib`` that collects submodules of ``sspilib.raw``,
most of which are cythonized extensions.
(`[#669](pyinstaller/pyinstaller-hooks-contrib#669)

&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/669&gt;`_)
<p>Updated hooks
</code></pre></p>
<ul>
<li>Modernize the hook for <code>torch</code> and reduce the amount of
unnecessarily
collected data files (header files and static libraries). Requires
PyInstaller &gt;= 6.0.
(<code>[#666](pyinstaller/pyinstaller-hooks-contrib#666)
&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/666&gt;</code>_)</li>
<li>Update <code>pyarrow</code> hook to collect all of the package's
submodules.
(<code>[#662](pyinstaller/pyinstaller-hooks-contrib#662)
&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/662&gt;</code>_)</li>
<li>Update <code>rtree</code> hook for compatibility with <code>Rtree
&gt;= 1.1.0</code>.
(<code>[#657](pyinstaller/pyinstaller-hooks-contrib#657)
&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/657&gt;</code>_)</li>
<li>Update <code>sudachipy</code> hook for <code>sudachipy</code> 0.6.8.
(<code>[#673](pyinstaller/pyinstaller-hooks-contrib#673)
&lt;https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/673&gt;</code>_)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/3bd34d400863c8b0466bf1e84a57b6814e13476e"><code>3bd34d4</code></a>
Release v2023.11</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/11dff78740e2dc7b6685388c70c8e9ba0e07a03a"><code>11dff78</code></a>
hooks: add hook for freetype</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/7916492b83f98674529594083e2a6ba1e294ccbd"><code>7916492</code></a>
hooks: add hook for z3c.rml</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/937ca15b5f4d17ed0421e4a3fe2bcafd234954a9"><code>937ca15</code></a>
Scheduled weekly dependency update for week 51 (<a
href="https://redirect.github.com/pyinstaller/pyinstaller-hooks-contrib/issues/671">#671</a>)</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/6a303f6c0bd7c2c51adf0418ed0cef19d647bb76"><code>6a303f6</code></a>
tests: fix deprecation warnings in test_sudachipy</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/6cb34e6a78b8a1368297cbef80bf96605cafe15b"><code>6cb34e6</code></a>
hooks: sudachipy: add hiddenimports for v0.6.8</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/97800abfb16f3e4a54bcc38f752b924a9e93d9ba"><code>97800ab</code></a>
tests: repin sudachipy to latest version</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/23cd820137a8905b018cc1c56d4e2d0a347bd9f4"><code>23cd820</code></a>
ci: install sudachidict packages on sudachipy update</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/f8aaa39b4dac28b8da62f3b2f5633425ac5fd153"><code>f8aaa39</code></a>
tests: fix dependencies of sudachipy test</li>
<li><a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/commit/f9c592dfcd7491cf6a6d33adde89fdf413d0914f"><code>f9c592d</code></a>
hooks: add hook for eth_rlp</li>
<li>Additional commits viewable in <a
href="https://github.com/pyinstaller/pyinstaller-hooks-contrib/compare/2023.10...2023.11">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pyinstaller-hooks-contrib&package-manager=pip&previous-version=2023.10&new-version=2023.11)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@rokm rokm mentioned this pull request May 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants