Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression when serializing large json numbers #5038

Closed
Blajda opened this issue Nov 5, 2023 · 3 comments · Fixed by #5042
Closed

Regression when serializing large json numbers #5038

Blajda opened this issue Nov 5, 2023 · 3 comments · Fixed by #5042
Assignees
Labels
arrow Changes to the arrow crate bug

Comments

@Blajda
Copy link

Blajda commented Nov 5, 2023

Describe the bug
Serializing and writing a json Number Number(1699148028689) to parquet using arrow_json the value is not preserved.
When the value is read back from the parquet file, we obtain -1659020527.

To Reproduce

    #[test]
    fn test_serde_field() {
        let field = Field::new("int", DataType::Int64, true);
        let mut decoder = ReaderBuilder::new_with_field(field)
            .build_decoder()
            .unwrap();
        decoder.serialize(&[1699148028689_u64, 2, 3, 4]).unwrap();
        let b = decoder.flush().unwrap().unwrap();
        let values = b.column(0).as_primitive::<Int64Type>().values();
        assert_eq!(values, &[1699148028689, 2, 3, 4]);
    }
@tustvold
Copy link
Contributor

tustvold commented Nov 5, 2023

Possibly introduced by #4861. I can take a look

@Blajda
Copy link
Author

Blajda commented Nov 7, 2023

Thanks for the fix @tustvold

@alamb alamb mentioned this issue Nov 7, 2023
6 tasks
@tustvold tustvold added the arrow Changes to the arrow crate label Nov 7, 2023
@tustvold
Copy link
Contributor

tustvold commented Nov 7, 2023

label_issue.py automatically added labels {'arrow'} from #5042

alamb pushed a commit to alamb/arrow-rs that referenced this issue Nov 8, 2023
alamb added a commit that referenced this issue Nov 9, 2023
…0.0_maintenance` (#5059)

* Fix serialization of large integers (#5038) (#5042)

* fmt

---------

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants