Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FEAT: Spark tests #1830

Merged
merged 25 commits into from
Jul 17, 2019
Merged

FEAT: Spark tests #1830

merged 25 commits into from
Jul 17, 2019

Conversation

amroid
Copy link
Contributor

@amroid amroid commented Jun 14, 2019

No description provided.

amroid added 18 commits June 5, 2019 15:57
@amroid
added spark client, compiler, some unit tests
bbb7244
@amroid
added kwargs for SparkClient and SparkContext
66067d9
@amroid
added Spark subclass of Backend
78f8fc1
@amroid
fixed PR changes
dc3608b
@amroid
Merge branch 'spark-client' into spark-tests
d55c64a
@amroid
fixed SparkUnion bug
0d7bb67
@amroid
test_string now works with Spark backend, added xfails and xpasses to…
54422ef 
… regex tests in test_string
@amroid
test_aggregation now works with Spark backend, changed base compiler …
bc9fff7 
…rewrite for any, notany, all, notall to use max and min instead of sum
@amroid
test_array now works with Spark backend, fixed Spark table creation a…
0560ada 
…nd tests
@amroid
added pyspark>=2.4.3 to requirements-3.x-dev.yml files
3b30726
@amroid
test_client now works with Spark backend, fixed mistake in test_sql i…
c96cb58 
…n test_client.py
@amroid
test_client now works with Spark backend, fixed mistake in test_sql i…
f934e40 
…n test_client.py
@amroid
test_column, test_generic now work with Spark backend (test_column re…
a8ecb0e 
…quired no changes)
@amroid
test_numeric now works with Spark backend, changed impala compiler im…
2ea6ac8 
…plementation of _number_literal_format
@amroid
test_param now works with Spark backend
45e80c6
@amroid
test_temporal now mostly works with Spark backend with some known iss…
2aa6d7a 
…ues, added floor option in convert_unit in util.py
@amroid
added string concat
c1b122f
@amroid
changed imports to not fail CI
87a2ece
@cpcloud cpcloud changed the title Spark tests FEAT: Spark tests Jun 20, 2019
cpcloud
cpcloud requested changes Jul 8, 2019
View reviewed changes
ibis/impala/compiler.py Show resolved Hide resolved
ibis/spark/api.py Show resolved Hide resolved
ibis/spark/client.py Show resolved Hide resolved
ibis/tests/all/test_temporal.py Show resolved Hide resolved
ibis/tests/all/test_temporal.py Show resolved Hide resolved
ibis/spark/compiler.py Show resolved Hide resolved
ibis/spark/compiler.py Outdated Show resolved Hide resolved
ibis/spark/compiler.py Outdated Show resolved Hide resolved
ibis/spark/compiler.py Outdated Show resolved Hide resolved
ibis/spark/compiler.py Outdated Show resolved Hide resolved
@cpcloud cpcloud added this to the Next Feature Release milestone Jul 9, 2019
@cpcloud cpcloud added the spark label Jul 9, 2019
@cpcloud cpcloud self-assigned this Jul 9, 2019
cpcloud
cpcloud requested changes Jul 9, 2019
View reviewed changes
ibis/impala/compiler.py Outdated Show resolved Hide resolved
ibis/spark/compiler.py Outdated Show resolved Hide resolved
ibis/spark/compiler.py Outdated Show resolved Hide resolved
@cpcloud
Copy link
Member

cpcloud commented Jul 10, 2019

ping @amroid what's the status here?

cpcloud
cpcloud requested changes Jul 16, 2019
View reviewed changes
ibis/spark/compiler.py Outdated Show resolved Hide resolved
ibis/spark/compiler.py Outdated Show resolved Hide resolved
ibis/spark/compiler.py Outdated Show resolved Hide resolved
ibis/tests/all/conftest.py Outdated Show resolved Hide resolved
cpcloud
cpcloud approved these changes Jul 17, 2019
View reviewed changes
Copy link
Member

@cpcloud cpcloud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet. Time to merge! Thanks @amroid.

@cpcloud cpcloud merged commit 923d975 into ibis-project:master Jul 17, 2019
@amroid amroid deleted the spark-tests branch July 17, 2019 18:24
@cpcloud cpcloud mentioned this pull request Jul 18, 2019
Closed
@cpcloud cpcloud modified the milestones: Next Feature Release, Next Major Release Aug 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cpcloud cpcloud cpcloud approved these changes

Assignees

@cpcloud cpcloud

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@amroid @cpcloud