Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ConfigurationConverter.getProperties is unable to hundle Integer type values from Yaml #610

Closed
cerveada opened this issue Feb 22, 2023 · 4 comments · Fixed by #611
Closed

ConfigurationConverter.getProperties is unable to hundle Integer type values from Yaml #610

cerveada opened this issue Feb 22, 2023 · 4 comments · Fixed by #611
Assignees
@cerveada
Labels
bug Something isn't working
Milestone
1.0.5

Comments

@cerveada
Copy link
Contributor

For example:

spline.lineageDispatcher.kafka.producer.max.in.flight.requests.per.connection:1

This is fixed in commons-configuration:1.7 https://issues.apache.org/jira/browse/CONFIGURATION-432

Unfortunately, Spark 2.4 depends on version 1.6

There already were other issues with YAML as version 1.2 being interpreted as float and there may be more.
Other properties sources just treat everything as string and do the conversion once it is asked for. Maybe we could use Yaml the same way.
@cerveada cerveada added the bug Something isn't working label Feb 22, 2023
@cerveada cerveada added this to the 1.0.5 milestone Feb 22, 2023
@cerveada
Copy link
Contributor Author

Reported in #609

cerveada added a commit that referenced this issue Feb 22, 2023
@cerveada
spark agent #610 quickfix for kafka lineage dispatcher issue
92050be
@wajda wajda mentioned this issue Feb 24, 2023
Merged
@wajda wajda linked a pull request Feb 24, 2023 that will close this issue
spark agent #610 quickfix for kafka lineage dispatcher issue #611
Merged
cerveada added a commit that referenced this issue Feb 24, 2023
@cerveada
spark agent #610 quickfix for kafka lineage dispatcher issue
4897829
@cerveada
Copy link
Contributor Author

Should we keep this open for future complete fix? Since #611 is only fixing the default, not removing the issue completely.

@wajda
Copy link
Contributor

wajda commented Feb 24, 2023

I'd like to propose a proper fix. Give me a sec.

@wajda wajda self-assigned this Feb 24, 2023
@wajda
Copy link
Contributor

wajda commented Feb 24, 2023

Since the above PR is already merged, I'll move the rest part to a new issue.

@wajda wajda closed this as completed Feb 24, 2023
@wajda wajda mentioned this issue Feb 24, 2023
Closed
@wajda wajda assigned cerveada and unassigned wajda Feb 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cerveada cerveada

Labels
bug Something isn't working
Projects
Spline
Status: Done
Milestone
1.0.5
Development

Successfully merging a pull request may close this issue.

spark agent #610 quickfix for kafka lineage dispatcher issue AbsaOSS/spline-spark-agent
2 participants
@wajda @cerveada