Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CsvPipe python api #401

Closed
sonalgoyal opened this issue Jul 16, 2022 · 3 comments
Closed

CsvPipe python api #401

sonalgoyal opened this issue Jul 16, 2022 · 3 comments
Assignees
@RavirajBaraiya
Milestone
0.3.4

Comments

@sonalgoyal
Copy link
Member

add a new constructor to CsvPipe which takes schema string and location and internally just sets the schema. make relevant changes everywhere in the examples etc. constructing a df to get the schema should happen internally in the constructor
@sonalgoyal sonalgoyal added this to the 0.3.4 milestone Jul 16, 2022
RavirajBaraiya added a commit to RavirajBaraiya/zingg that referenced this issue Jul 17, 2022
@RavirajBaraiya
CsvPipe python api issue zinggAI#401
bc4a969
RavirajBaraiya added a commit to RavirajBaraiya/zingg that referenced this issue Jul 17, 2022
@RavirajBaraiya
CsvPipe python api issue zinggAI#401
0bffcbc
@sonalgoyal
Copy link
Member Author

@raviraj baraiya

  • I made changes to csv pipe, please check latest commit of mine to 0.3.4 and change all examples. I have added overloaded contructor. Output pipe should use init(name, location). input pipe should use CsvPipe(name, location, schema string)

Please do a search for CsvPipe in the code base and make the changes

RavirajBaraiya added a commit to RavirajBaraiya/zingg that referenced this issue Jul 18, 2022
@RavirajBaraiya
changes in pipes module and csvPipe issue zinggAI#401
5a0b5be
@sonalgoyal
Copy link
Member Author

verify testrFebrl and then we can close.

RavirajBaraiya added a commit to RavirajBaraiya/zingg that referenced this issue Jul 19, 2022
@RavirajBaraiya
Pipes changes issue zinggAI#401
efe6c43
sonalgoyal added a commit that referenced this issue Jul 19, 2022
@sonalgoyal
Merge pull request #411 from RavirajBaraiya/Pipes
c71e32b 
Pipes changes issue #401
@sonalgoyal
Copy link
Member Author

2022-07-19 21:49:22,493 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Traceback (most recent call last):
File "/home/sgoyal/zingg/examples/febrl/FebrlExample.py", line 31, in
inputPipe = CsvPipe("testFebrl", "examples/febrl/test.csv", schema)
TypeError: init() takes 2 positional arguments but 4 were given

@RavirajBaraiya

sonalgoyal pushed a commit that referenced this issue Jul 20, 2022
@RavirajBaraiya @sonalgoyal
Pipes changes issue #401
87b9d5c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@RavirajBaraiya RavirajBaraiya

Labels
None yet
Projects
None yet
Milestone
0.3.4
Development

No branches or pull requests
2 participants
@sonalgoyal @RavirajBaraiya