Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Split tests in two groups by backend #2194

Merged
merged 15 commits into from
May 7, 2020

Conversation

datapythonista
Copy link
Contributor

Testing if splitting tests in this naive way solves the disk space problem. I guess docker images for the unused backends should not be download and storage consumption should be much less.

@jreback jreback added the ci Continuous Integration issues or PRs label Apr 23, 2020
@jreback jreback added this to the Next Bugfix Release milestone Apr 23, 2020
@datapythonista
Copy link
Contributor Author

@jreback this solves the disk space problem by splitting the tests in two groups of backends. All green, except the conda build build, which is a separate issue (see #2195).

I couldn't get this working with spark and omnisci, so those two are not being tested here (I'll create issues to add them back if we merge this).

@jreback
Copy link
Contributor

jreback commented May 7, 2020

yeah create issues for those (can run as separate backends entirely)

will look soon

BACKENDS: "clickhouse impala kudu-master kudu-tserver mysql omniscidb parquet postgres sqlite"
# TODO: omniscidb should be in BACKENDS_1, but it's not compatible with Python 3.8, so it needs to be set individually in py36 and py37 builds.
# It is not being added, because the conda solver takes forever to resolve when pymapd is present, so it's not being tested for now
BACKENDS_1: "mysql parquet postgres sqlite"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there someway you can actually put the names rather than backends_1, e.g.
mysql-parquet-postgres-sqlite (e.g. more informative in the displays / code).

@jreback
Copy link
Contributor

jreback commented May 7, 2020

i merged the feedstock conda job so in theory this should be all green if you rebase

@datapythonista
Copy link
Contributor Author

i merged the feedstock conda job so in theory this should be all green if you rebase

I changed the BACKEND_1... names. Let's see if we get a green build then.

@datapythonista
Copy link
Contributor Author

LinuxCondaBuild is still failing, I think using Python 3.7 is needed, checking in #2202.

@jreback
Copy link
Contributor

jreback commented May 7, 2020

kk happy to merge this and can proceed?

@datapythonista
Copy link
Contributor Author

datapythonista commented May 7, 2020

Yes, I'd merge this, and then we see the conda build problem. I guess just the change in #2202 is needed (probably better to add the parameter, as @xmnlab did).

@jreback jreback merged commit 8910d40 into ibis-project:master May 7, 2020
@jreback
Copy link
Contributor

jreback commented May 7, 2020

thanks @datapythonista

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci Continuous Integration issues or PRs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants