Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The new log obfuscates the aggregation on Kafka topics #272

Closed
baixiac opened this issue Jul 30, 2021 · 4 comments
Closed

The new log obfuscates the aggregation on Kafka topics #272

baixiac opened this issue Jul 30, 2021 · 4 comments

Comments

@baixiac
Copy link
Member

baixiac commented Jul 30, 2021

Looks like the logging in the new release confuses radar-docker install when it tries to aggregate topics by getting the last line from stdout of list_aggregated.sh (using command radar-schemas-tools):

radar-schema-tools is initialized with root directory /schema/merged [org.radarbase.schema.tools.CommandLineApp:63] ,android_biovotion_vsm1_acceleration_10min,android_biovotion_vsm1_acceleration_10sec,android_biovotion_vsm1_acceleration_1day,android_biovotion_vsm1_acceleration_1hour,android_biovotion_vsm1_acceleration_1min,android_biovotion_vsm1_acceleration_1week...

rather than expected:

android_biovotion_vsm1_acceleration_10min,android_biovotion_vsm1_acceleration_10sec,android_biovotion_vsm1_acceleration_1day,android_biovotion_vsm1_acceleration_1hour,android_biovotion_vsm1_acceleration_1min,android_biovotion_vsm1_acceleration_1week...

Thus, ./etc/mongodb-connector/sink-mongo.properties becomes invalid:

...
topics=radar-schema-tools is initialized with root directory /schema/merged [org.radarbase.schema.tools.CommandLineApp:63] ,android_biovotion_vsm1_acceleration_10min,android_biovotion_vsm1_acceleration_10sec,android_biovotion_vsm1_acceleration_1day,android_biovotion_vsm1_acceleration_1hour,android_biovotion_vsm1_acceleration_1min,android_biovotion_vsm1_acceleration_1week...

Here is the call site of the logging:

logger.info("radar-schema-tools is initialized with root directory {}", this.root);

@baixiac
Copy link
Member Author

baixiac commented Aug 2, 2021

The maximum no. of retries on refreshing topics has been set to 10 and that's oh. However, the retry back-off seems not working/existing or the timeout is too short so they all happed within one sec:
https://github.com/RADAR-base/RADAR-Docker/runs/3203037035?check_suite_focus=true#step:6:185

More details in here:
https://github.com/RADAR-base/RADAR-Schemas/blob/master/java-sdk/radar-schemas-registration/src/main/java/org/radarbase/schema/registration/KafkaTopics.java#L193-L230

@yatharthranjan
Copy link
Member

The maximum no. of retries on refreshing topics has been set to 10 and that's oh. However, the retry back-off seems not working/existing or the timeout is too short so they all happed within one sec:
https://github.com/RADAR-base/RADAR-Docker/runs/3203037035?check_suite_focus=true#step:6:185
More details in here:
https://github.com/RADAR-base/RADAR-Schemas/blob/master/java-sdk/radar-schemas-registration/src/main/java/org/radarbase/schema/registration/KafkaTopics.java#L193-L230

Better to create a new issue for this as it is unrelated to this one

@yatharthranjan
Copy link
Member

Moved the second issue to #281

@blootsvoets
Copy link
Member

closed by #280.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants