Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Added example reading Avro produced by Kafka #1151

Merged
merged 1 commit into from
Jul 11, 2022
Merged

Conversation

jorgecarleitao
Copy link
Owner

Kafka uses the single object representation of Avro. This PR adds an example showing how to read such stream.

This emerged from a question by scimas at rust users: https://users.rust-lang.org/t/status-of-avro-in-rust/78086/6

@jorgecarleitao jorgecarleitao added the documentation Improvements or additions to documentation label Jul 11, 2022
@codecov
Copy link

codecov bot commented Jul 11, 2022

Codecov Report

Merging #1151 (e539288) into main (ba45d55) will increase coverage by 0.06%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1151      +/-   ##
==========================================
+ Coverage   83.66%   83.72%   +0.06%     
==========================================
  Files         366      366              
  Lines       35912    35931      +19     
==========================================
+ Hits        30044    30082      +38     
+ Misses       5868     5849      -19     
Impacted Files Coverage Δ
src/array/utf8/mod.rs 85.62% <0.00%> (-0.96%) ⬇️
src/io/ipc/read/reader.rs 96.32% <0.00%> (+0.33%) ⬆️
src/io/parquet/read/deserialize/mod.rs 85.91% <0.00%> (+1.03%) ⬆️
src/io/parquet/read/deserialize/nested_utils.rs 86.37% <0.00%> (+3.48%) ⬆️
src/array/equal/struct_.rs 70.73% <0.00%> (+24.39%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba45d55...e539288. Read the comment docs.

@jorgecarleitao jorgecarleitao changed the title Added example reading Avro from a Kafka stream Added example reading Avro produced by Kafka Jul 11, 2022
@jorgecarleitao jorgecarleitao merged commit eba5f2e into main Jul 11, 2022
@jorgecarleitao jorgecarleitao deleted the avro_kafka branch July 11, 2022 14:02
"Avro requires the first byte to be a zero".to_string(),
));
}
Ok(i32::from_be_bytes(header[1..].try_into().unwrap()))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that this is already merged, but I would suggest changing this to u32 instead of i32.

I don't know where the specification for the schema_id is, but the API documentation for Confluent's schema registry specifies that version (schema_id) should be in the range [1, 2^31 - 1].

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants