BLD: Split out tests into pandas_tests package #53007

lithomas1 · 2023-04-29T22:38:30Z

closes Suggestion: remove tests from the distribution #30741 (Replace xxxx with the GitHub issue number)
Tests added and passed if fixing a bug or adding a new feature
All code checks passed.
Added type annotations to new arguments/methods/functions.
Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

This seems to work(I tested by making a new conda env and running tests via pd.test() ) but is kinda sketchy.
(Last thing to do is to update the absolute imports from within the test folder to relative ones).

This saves about 1-2MBs for me off the total wheel size for me.

jbrockmendel · 2023-04-30T20:01:06Z

MANIFEST.in

@@ -53,7 +53,11 @@ global-exclude *.pxi
 # GH 39321
 # csv_dir_path fixture checks the existence of the directory
 # exclude the whole directory to avoid running related tests in sdist
-prune pandas/tests/io/parser/data
+#prune pandas/tests/io/parser/data


Is this commented out on purpose? Remove?

jbrockmendel · 2023-04-30T20:02:21Z

So we wouldn’t need to make a separate repo? Would we need to set something up on pypi? If “no” to both then very neat

lithomas1 · 2023-04-30T22:43:38Z

So we wouldn’t need to make a separate repo? Would we need to set something up on pypi? If “no” to both then very neat

No, we wouldn't need to make a separate repo, but we'd have to set up a "pandas-tests" package on PyPI (unless we are removing ability to test altogether).

The idea is that "pandas-tests" would have a pinned pandas version as a dependency, and we would release pandas and pandas-tests at the same time.

The only thing left to do in this PR is to turn all the absolute imports into relative imports (since the tests may now live elsewhere), which is probably going to cause a lot of churn, and then run the tests to make sure that nothing else in the tests is depending on stuff in pandas proper.

mroeschke · 2023-05-01T17:38:02Z

Would also need some validation in pd.test to raise an error if pandas-test package isn't installed

github-actions · 2023-06-01T00:06:28Z

This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this.

lithomas1 · 2023-06-02T18:10:10Z

Planning on holding off on this until setuptools is completely gone.

mroeschke · 2023-08-01T17:26:42Z

Going to close to clear the queue but feel free to reopen when ready

…ve-tests

simonjayhawkins · 2024-02-06T17:46:27Z

Planning on holding off on this until setuptools is completely gone.

@lithomas1 what's the timescales on this?

lithomas1 · 2024-02-07T03:15:19Z

I'll take another look - thanks for the reminder.

…e-tests

lithomas1 · 2024-03-19T03:46:41Z

Finally green here 😌

Anyone want to take a look?

(NOTE: The diff is only big because I split the doctest stuff out of conftest. Actual changes are only ~200 lines or so)

alimcmaster1 · 2024-03-19T21:20:35Z

pandas/__init__.py

@@ -188,6 +188,19 @@
    __git_version__ = v.get("full-revisionid")
    del get_versions, v

+import sys


Does this break an API users potentially use. If so will we need a depr cycle?

Kinda.
You can still access pandas.tests as before (if you're using a wheel without the tests submodule) e.g. if you wanted to use the extension dtypes tests, but you'll need to install pandas-tests to do so.

This is technically breaking, but the next release also happens to be 3.0, so I think we'd be able to get away with a breaking change here :)

alimcmaster1 · 2024-03-19T21:21:15Z

Hey!

Thinks this is a decent idea.

How much does this decrease the dist size by? Would be good to know that before merging this as that's the main benefit of this change right? Shout if I'm missing something.

How will we ensure users don't have mismatching versions of pandas and pandas_test installed if these are to deployed to pypi as sep packages? I didn't see any logic for this. As will need to pin version of pandas used in pandas_test right?

lithomas1 · 2024-03-19T23:55:58Z

Hey!

Thinks this is a decent idea.

How much does this decrease the dist size by? Would be good to know that before merging this as that's the main benefit of this change right? Shout if I'm missing something.

It's about 15 MBs. The whole of pandas is around 45 MB, so it shaves off a third off the install side.

How will we ensure users don't have mismatching versions of pandas and pandas_test installed if these are to deployed to pypi as sep packages? I didn't see any logic for this. As will need to pin version of pandas used in pandas_test right?

I haven't figured this out yet (My original plan was to get this merged first and see if downstream is able to easily adapt to this before proceeding further). I was thinking I was going to put in a pin in the pyproject.toml for pandas_tests come release time. Open to suggestion on this, though.

ci/code_checks.sh

mroeschke

I think now it may be important to run the wheel build job on PRs again since more PRs will be updating pandas and pandas_tests and we need to make sure there's always compatibility between the two.

lithomas1 · 2024-03-20T19:47:41Z

I think now it may be important to run the wheel build job on PRs again since more PRs will be updating pandas and pandas_tests and we need to make sure there's always compatibility between the two.

I think it should be OK personally.

On a normal pandas build pandas_tests is just pandas.tests, we don't exclude the tests on a regular build, just on builds from sdist (which would be the case of a wheel build).

lithomas1 · 2024-03-27T19:09:24Z

@pandas-dev/pandas-core
any other feedback?

I'd like to merge this in otherwise.

mroeschke · 2024-03-27T19:26:33Z

pandas/pyproject.toml

+#dependencies=[
+#    "pandas==2.1.0"
+#]


Should this still be commented out?

Thanks for the reminder.

I planned on doing this in a followup (since technically this only needs to be set at release time), but then realized I can do this programmtically.

I'll make the changes to the wheel builder workflow soon.

mroeschke · 2024-03-27T19:35:11Z

I would still prefer to have every PR run 1 job (Ubuntu only) pandas wheel without tests + pandas_tests wheel to validate that the connection between the two is never broken.

Our CI jobs are relatively streamlined now and I think this job is definitely more important than some of the checks we run on every PR

WillAyd · 2024-03-27T21:16:40Z

Nice work, but I'm pretty lukewarm to the implementation. Seems like there are a lot of potential break points and CI is not the easiest to debug.

For me locally I see 20.7 MB / 33.6 MB in the test folder is from data files, with a good deal of that from SAS. I am unsure if all of the added complexity here is worth it versus a simpler solution to find a better home for data files

lithomas1 · 2024-03-27T21:33:50Z

Nice work, but I'm pretty lukewarm to the implementation. Seems like there are a lot of potential break points and CI is not the easiest to debug.

For me locally I see 20.7 MB / 33.6 MB in the test folder is from data files, with a good deal of that from SAS. I am unsure if all of the added complexity here is worth it versus a simpler solution to find a better home for data files

We already don't ship the data files in the test suite, so the 15 MBs of savings is from the actual Python pandas tests files, which lines up with the 13ish MBs of non test data that you're seeing.
(this is around a third of the unzipped wheel size)

Is there something in particular that worries you about the implementation?

(r.e. the breaking points: IMO, this does not affect regular CI at all and should only affect the wheel builders.

I do understand that this adds quite a bit of complexity to the build process, though.
Most of the added complexity should be contained to just to the wheel builders workflow and not to the core Python files of pandas itself.

Do let me know if there's something I can simplify more or clarify/document better.
)

lithomas1 · 2024-03-27T21:37:09Z

I would still prefer to have every PR run 1 job (Ubuntu only) pandas wheel without tests + pandas_tests wheel to validate that the connection between the two is never broken.

Our CI jobs are relatively streamlined now and I think this job is definitely more important than some of the checks we run on every PR

Gotcha.

I guess the easiest route would be to run a subset of the wheel builders on every PR. I'll do that in a followup.

WillAyd

Ah OK thanks @lithomas1 and sorry for the ignorance. I've left a few comments where I have concerns

WillAyd · 2024-03-27T21:37:31Z

.circleci/config.yml

+          # (unlike Github Actions there is no host access using cibuildwheel with CircleCI)
+          name: Build the sdist
+          command: |
+            pip3 install build setuptools-scm wheel


Is setuptools-scm a misnomer or does it have some kind of dependency with setuptools?

We use it to version the pandas-tests package?

Setting up versioneer is probably overkill for pandas-tests.

Hmm not sure I understand. So this does still require setuptools right? I think that is confusing given the work we have put into meson to replace that library

Yeah, pandas-tests is built using setuptools.

I figured it was much easier to use setuptools given this is a pure Python package.

(Note that if you're just doing regular development you'll never need setuptools, unless you want to build pandas_tests by hand for some reason)

.circleci/config.yml

.github/workflows/wheels.yml

WillAyd · 2024-03-27T21:46:17Z

pandas/conftest.py

@@ -1,120 +1,25 @@
 """
-This file is very long and growing, but it was decided to not split it yet, as
-it's still manageable (2020-03-17, ~1.1k LoC). See gh-31989
+Just a conftest file for doctest stuff


Sorry if I missed in the thread but why do we need a doctest conftest separate from the test confest? This doesn't work to move everything?

BLD: Split out tests into pandas_tests package

d3edcdf

lithomas1 added the Build Library building on various platforms label Apr 29, 2023

jbrockmendel reviewed Apr 30, 2023

View reviewed changes

github-actions bot added the Stale label Jun 1, 2023

lithomas1 added the Mothballed Temporarily-closed PR the author plans to return to label Jun 2, 2023

mroeschke closed this Aug 1, 2023

lithomas1 reopened this Aug 23, 2023

lithomas1 added 8 commits August 23, 2023 07:01

Merge branch 'main' of https://github.com/pandas-dev/pandas into remo…

8e0d861

…ve-tests

update for meson

bd83272

adjust cibuildwheel command?

339dacc

Merge branch 'main' of https://github.com/pandas-dev/pandas into remo…

eea0220

…ve-tests

update

a5a89b3

try again

48df264

fix more

d00e471

update

a43c9fe

lithomas1 added 7 commits February 11, 2024 12:21

Merge branch 'main' of github.com:pandas-dev/pandas into remove-tests

b2c2b8c

Update wheels.yml

d6ac87c

try fix paths

427ea4e

try something

7174249

try something

f50f110

typo

9cbbb66

try again

053ad48

lithomas1 added 8 commits March 18, 2024 11:13

meson changes

7ee6adc

try moving back stuff to the original conftest

19fa7f9

fix typing?

be64699

Merge branch 'main' into remove-tests

f1c18cc

add pyright ignore

e835993

Merge branch 'remove-tests' of github.com:lithomas1/pandas into remov…

5bdc0d1

…e-tests

last ignore

2cf3911

fix musl

e9ba935

lithomas1 requested a review from mroeschke March 19, 2024 16:23

alimcmaster1 reviewed Mar 19, 2024

View reviewed changes

mroeschke reviewed Mar 20, 2024

View reviewed changes

ci/code_checks.sh Outdated Show resolved Hide resolved

mroeschke reviewed Mar 20, 2024

View reviewed changes

lithomas1 added 2 commits March 20, 2024 15:03

Update code_checks.sh

e53ad08

Merge branch 'main' into remove-tests

23952aa

lithomas1 requested a review from mroeschke March 22, 2024 17:42

Merge branch 'main' into remove-tests

23b9827

mroeschke reviewed Mar 27, 2024

View reviewed changes

WillAyd reviewed Mar 27, 2024

View reviewed changes

agriyakhetarpal mentioned this pull request Apr 1, 2024

BLD, TST: Build and test Pyodide wheels for pandas in CI #57896

Merged

5 tasks

mroeschke mentioned this pull request Apr 12, 2024

RLS: 3.0 #57064

Open

agriyakhetarpal mentioned this pull request Apr 19, 2024

Making the numpy test suite separately installable numpy/numpy#26289

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BLD: Split out tests into pandas_tests package #53007

BLD: Split out tests into pandas_tests package #53007

lithomas1 commented Apr 29, 2023 •

edited

Loading

jbrockmendel Apr 30, 2023

jbrockmendel commented Apr 30, 2023

lithomas1 commented Apr 30, 2023

mroeschke commented May 1, 2023

github-actions bot commented Jun 1, 2023

lithomas1 commented Jun 2, 2023

mroeschke commented Aug 1, 2023

simonjayhawkins commented Feb 6, 2024

lithomas1 commented Feb 7, 2024

lithomas1 commented Mar 19, 2024

alimcmaster1 Mar 19, 2024

lithomas1 Mar 19, 2024

alimcmaster1 commented Mar 19, 2024 •

edited

Loading

lithomas1 commented Mar 19, 2024

mroeschke left a comment

lithomas1 commented Mar 20, 2024

lithomas1 commented Mar 27, 2024

mroeschke Mar 27, 2024

lithomas1 Mar 27, 2024

mroeschke commented Mar 27, 2024

WillAyd commented Mar 27, 2024

lithomas1 commented Mar 27, 2024 •

edited

Loading

lithomas1 commented Mar 27, 2024

WillAyd left a comment

WillAyd Mar 27, 2024

lithomas1 Mar 30, 2024

WillAyd Apr 1, 2024

lithomas1 Apr 1, 2024

WillAyd Mar 27, 2024

BLD: Split out tests into pandas_tests package #53007

Are you sure you want to change the base?

BLD: Split out tests into pandas_tests package #53007

Conversation

lithomas1 commented Apr 29, 2023 • edited Loading

Choose a reason for hiding this comment

jbrockmendel commented Apr 30, 2023

lithomas1 commented Apr 30, 2023

mroeschke commented May 1, 2023

github-actions bot commented Jun 1, 2023

lithomas1 commented Jun 2, 2023

mroeschke commented Aug 1, 2023

simonjayhawkins commented Feb 6, 2024

lithomas1 commented Feb 7, 2024

lithomas1 commented Mar 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alimcmaster1 commented Mar 19, 2024 • edited Loading

lithomas1 commented Mar 19, 2024

mroeschke left a comment

Choose a reason for hiding this comment

lithomas1 commented Mar 20, 2024

lithomas1 commented Mar 27, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mroeschke commented Mar 27, 2024

WillAyd commented Mar 27, 2024

lithomas1 commented Mar 27, 2024 • edited Loading

lithomas1 commented Mar 27, 2024

WillAyd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lithomas1 commented Apr 29, 2023 •

edited

Loading

alimcmaster1 commented Mar 19, 2024 •

edited

Loading

lithomas1 commented Mar 27, 2024 •

edited

Loading