Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature base wrangler #1

Merged
merged 23 commits into from Mar 1, 2019
Merged

Feature base wrangler #1

merged 23 commits into from Mar 1, 2019

Conversation

mansenfranzen
Copy link
Owner

This PR adds the BaseWrangler class and improves testing on TravisCI.

  1. The BaseWrangler class defines the basic interface for all wranglers. It is heavily inspired by the sklearn API (fit and transform).
  2. Testing python 3.7 is not possible on Ubuntu 14.04 with TravisCI. However, upgrading to Ubuntu 16.04 to test python 3.7, changed the Java version from 8 to 11 which in turn caused incompatibility with pyspark because spark does not work with Java 11. It was necessary to install Java 8 separately and setting JAVA_HOME (while passing it to tox test envs via passenv) in order to make it work. This solution is probably not stable but works for the moment until TravisCI either enables python 3.7 on Ubuntu 14.04 or Java 8 on Ubuntu 16.04 for python projects. See here for more.

@mansenfranzen
Add wranglers subpackage including initial description of `BaseWran…
6521c9a 
…gler`.
@mansenfranzen
Add _pprint helper module for printing.
7499d5c
@mansenfranzen
Add get_params, set_params and __repr__ and increase documentat…
ae0048d 
…ion detail.
@mansenfranzen
Enable tests for python 3.6 and 3.7 on TravisCI.
a303d18
@mansenfranzen
Add tests for BaseWrangler. Fix incorrect dictionary iteration.
a32132f
@mansenfranzen
Force upgrade of setuptools to be compatible with pyscaffold in Travi…
07460ce 
…sCI.
@mansenfranzen
Correct name of environment variable for test environment.
a982546
@mansenfranzen
Sort keys for enumeration print of dictionary.
310327c
@mansenfranzen
Add explicit distribution (xenial) to enable python 3.7 on TravisCI.
ebfb8aa
@mansenfranzen
Force jdk to be version 8 to prevent spark-java interaction errors.
58f73c8
@mansenfranzen
Retry with openjdk 8 for TravisCI.
14fd8d3
@mansenfranzen
Yet another TravisCI test to make python 3.7 work with java 8.
9e5f7da
@mansenfranzen
Yet another test using addons with oracle-java8-set-default.
f8e0eb0
@mansenfranzen
Add missing dist xenial.
8697e8f
@mansenfranzen
Add java 8 installer.
5a3765a
@mansenfranzen
Testing next workaround for Java 8.
d496382
@mansenfranzen
Get more information about installed java environments.
a7126b3
@mansenfranzen
Set JAVA_HOME as env variable.
e41b250
@mansenfranzen
Print JAVA_HOME before error.
d49af3a
@mansenfranzen
Fix wrong brackets :-(.
b082113
@mansenfranzen
Pass JAVA_HOME to tox test environments via passenv to allow tests …
49413df 
…to access JAVA_HOME to find correct Java version.
@mansenfranzen
Test against all python versions since JAVA 8 is now working on xenial.
53939e0
@mansenfranzen
Move Java 8 install into separate travis_java_install.sh script. On…
4bc5889 
…ly install Java 8 when spark tests are selected via `ENV_STRING`.
@mansenfranzen mansenfranzen added the new feature Add a new feature label Mar 1, 2019
@mansenfranzen mansenfranzen self-assigned this Mar 1, 2019
@mansenfranzen mansenfranzen merged commit 64a24de into master Mar 1, 2019
@mansenfranzen mansenfranzen deleted the feature_base_wrangler branch March 1, 2019 19:46
mansenfranzen pushed a commit that referenced this pull request Feb 4, 2020
@TobiasRasbold
Merge pull request #1 from mansenfranzen/master
550cf4e 
Update my fork from upstream repo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@mansenfranzen mansenfranzen

Labels
new feature Add a new feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@mansenfranzen