Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCFWriter.Append should return number of bytes written #183

Open
nmyk opened this issue Nov 5, 2019 · 0 comments
Open

OCFWriter.Append should return number of bytes written #183

nmyk opened this issue Nov 5, 2019 · 0 comments

Comments

@nmyk
Copy link

nmyk commented Nov 5, 2019

Hi! I'm trying to build a data pipeline that writes data to Avro files, and when an individual file reaches a certain size, I want to close it and start a new file. Currently I have to check the file size after every write (records are not fixed-size), but if OCFWriter.appendDataIntoBlock returned the number of bytes written through OCFWriter.Append, I could just add that number to an accumulator instead of making repeated calls to os.File.Stat().

The exact line where the number of bytes written is thrown away is https://github.com/linkedin/goavro/blob/master/ocf_writer.go#L227

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant