Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates default inputSegmentSizeBytes in Compaction config #12534

Merged

Conversation

tejaswini-imply
Copy link
Member

@tejaswini-imply tejaswini-imply commented May 18, 2022

Fixes Cannot serialize BigInt value as JSON error while loading compaction config in console.

Description

The reason this error is occurring is that the current default value of inputSegmentSizeBytes is Long.MAX_VALUE. Numbers over Number.MAX_SAFE_INTEGER (9,007,199,254,740,991) may not be handled by few clients like JS.

Hence reducing the default inputSegmentSizeBytes from Long.MAX_VALUE to 100,000,000,000,000 (~100TB).


Key changed/added classes in this PR
  • DataSourceCompactionConfig

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

@cryptoe
Copy link
Contributor

cryptoe commented May 18, 2022

@tejaswini-imply This was very annoying. Thanks for the fix !!. Hope you have tested it locally !!

@abhishekagarwal87 abhishekagarwal87 added this to the 0.23.0 milestone May 18, 2022
@abhishekagarwal87
Copy link
Contributor

Marking it for 0.23.0 milestone as without the fix, auto-compaction dialog in console will throw an error.

@tejaswini-imply
Copy link
Member Author

@tejaswini-imply This was very annoying. Thanks for the fix !!. Hope you have tested it locally !!

Yes, @cryptoe. When I tested it locally the updated default value is reflected in the console after submitting the compaction config.

Copy link
Contributor

@kfaraz kfaraz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@abhishekagarwal87 abhishekagarwal87 merged commit c877d8a into apache:master May 19, 2022
@abhishekagarwal87
Copy link
Contributor

Thank you @tejaswini-imply. Can you also backport it to 0.23.0 branch?

tejaswini-imply added a commit to tejaswini-imply/druid that referenced this pull request May 20, 2022
)

Fixes Cannot serialize BigInt value as JSON error while loading compaction config in console.
abhishekagarwal87 pushed a commit that referenced this pull request May 20, 2022
…12548)

Fixes Cannot serialize BigInt value as JSON error while loading compaction config in console.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants