Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added support for reading and writing avro files #490

Merged
merged 2 commits into from
Jun 11, 2020
Merged

Added support for reading and writing avro files #490

merged 2 commits into from
Jun 11, 2020

Conversation

juarezr
Copy link
Member

@juarezr juarezr commented Jun 11, 2020 •
edited
Loading

Added support for reading and writing avro files

Apache Avro is a data serialization framework with broad use in many business.

It is used in data serialization (especially in Hadoop ecosystem), for data exchange for databases (Redshift) and in RPC protocols (like in Kafka).

It has open source libraries to support many languages and generally is faster and safer than text formats like Json, XML or CSV.

Changes in pull request

Added functions petl.io.avro.fromavro, petl.io.avro.toavro, and petl.io.avro.appendavro for reading and writing to Avro files.

Checklist

  • Includes unit tests
  • New functions have docstrings with examples that can be run with doctest
  • New functions are included in API docs
  • Docstrings include notes for any changes to API or behaviour
  • Travis CI passes (unit tests run under Linux)
  • AppVeyor CI passes (unit tests run under Windows)
  • Unit test coverage has not decreased (see Coveralls)
  • All changes documented in docs/changes.rst

@coveralls
Copy link

coveralls commented Jun 11, 2020 •
edited
Loading

Coverage Status

Coverage decreased (-1.06%) to 90.995% when pulling 1491473 on juarezr:petlavro into 2f959d4 on petl-developers:master.

alimanfoo
alimanfoo reviewed Jun 11, 2020
View reviewed changes
Copy link
Collaborator

@alimanfoo alimanfoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @juarezr, this looks fantastic, very comprehensive. Just a couple of comments to bump the version number.

docs/changes.rst Outdated Show resolved Hide resolved
docs/changes.rst Outdated Show resolved Hide resolved
petl/io/avro.py Outdated Show resolved Hide resolved
petl/io/avro.py Outdated Show resolved Hide resolved
petl/io/avro.py Outdated Show resolved Hide resolved
@juarezr @alimanfoo
Apply suggestions from code review of avro support in pull petl-devel…
1491473 
…opers#490


Set version to 1.4.0 and issue to 490 as suggested in pull request petl-developers#490

Co-authored-by: Alistair Miles <alimanfoo@googlemail.com>
@alimanfoo alimanfoo added this to the v1.4.0 milestone Jun 11, 2020
alimanfoo
alimanfoo approved these changes Jun 11, 2020
View reviewed changes
Copy link
Collaborator

@alimanfoo alimanfoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the very nice PR.

@alimanfoo
Copy link
Collaborator

Ready to merge?

@juarezr
Copy link
Member Author

juarezr commented Jun 11, 2020 •
edited
Loading

I think this pull is ready to merge from my side.

Maybe there is something that I missed because this is my first python contribution ever, but we can handle further bugs and improvements in new pull requests.

So, if you don't have any considerations for now, please go ahead and merge .

Thanks for the support. (And for this nice code!)

@alimanfoo alimanfoo merged commit b1ef013 into petl-developers:master Jun 11, 2020
@alimanfoo
Copy link
Collaborator

Maybe there is something that I missed because this is my first python contribution ever, but we can handle further bugs and improvements in new pull requests.

Well I think you win the award for best first contribution I've ever seen :) Thanks again, I'll make a release so this functionality is available.

@juarezr
Copy link
Member Author

juarezr commented Jun 11, 2020

First contribution using python. 😄.
Also the work was most looking existing code and write a similar...

Thanks for the merge.

@alimanfoo
Copy link
Collaborator

Release 1.4.0 is on PyPI.

@juarezr juarezr deleted the petlavro branch September 17, 2020 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alimanfoo alimanfoo alimanfoo approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v1.4.0
Development

Successfully merging this pull request may close these issues.
3 participants
@juarezr @coveralls @alimanfoo