Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cloud storage metrics #1848

Merged
merged 2 commits into from Apr 14, 2021
Merged

Add cloud storage metrics #1848

merged 2 commits into from Apr 14, 2021

Conversation

steven-sheehy
Copy link
Member

@steven-sheehy steven-sheehy commented Apr 13, 2021

Detailed description:

  • Add hedera.mirror.importer.cloud.latency metric to compare consensus time against cloud storage creation time
  • Add hedera.mirror.download.latency metric to all stream types
  • Add hedera.mirror.stream.close.latency metric to all stream types
  • Fix event file not persisting due to missing required consensus end field
  • Fix cloud storage response size and time metrics not counting _sig.gz files as signature files

Which issue(s) this PR fixes:

Special notes for your reviewer:

Checklist

  • Documentation added
  • Tests updated

Signed-off-by: Steven Sheehy <steven.sheehy@hedera.com>
@steven-sheehy steven-sheehy added enhancement Type: New feature P2 downloader Area: S3 downloader labels Apr 13, 2021
@steven-sheehy steven-sheehy added this to the Mirror 0.32.0 milestone Apr 13, 2021
@steven-sheehy steven-sheehy requested a review from a team April 13, 2021 21:38
@steven-sheehy steven-sheehy self-assigned this Apr 13, 2021
@codecov
Copy link

codecov bot commented Apr 13, 2021

Codecov Report

Merging #1848 (c304fc5) into master (8125fd6) will decrease coverage by 0.30%.
The diff coverage is 28.57%.

❗ Current head c304fc5 differs from pull request most recent head 4e6beb6. Consider uploading reports for the commit 4e6beb6 to get more accurate results
Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1848      +/-   ##
============================================
- Coverage     87.37%   87.07%   -0.31%     
- Complexity     1743     1748       +5     
============================================
  Files           315      315              
  Lines          7691     7736      +45     
  Branches        735      738       +3     
============================================
+ Hits           6720     6736      +16     
- Misses          744      773      +29     
  Partials        227      227              
Impacted Files Coverage Δ Complexity Δ
...porter/downloader/record/RecordFileDownloader.java 100.00% <ø> (ø) 2.00 <0.00> (-1.00)
...ra/mirror/monitor/config/MonitorConfiguration.java 0.00% <0.00%> (ø) 0.00 <0.00> (ø)
.../hedera/mirror/monitor/publish/PublishMetrics.java 0.00% <0.00%> (ø) 0.00 <0.00> (ø)
...a/mirror/monitor/publish/TransactionPublisher.java 1.17% <0.00%> (-0.39%) 1.00 <0.00> (ø)
...dera/mirror/monitor/publish/PublishProperties.java 78.57% <33.33%> (-4.77%) 4.00 <0.00> (ø)
...r/importer/config/MetricsExecutionInterceptor.java 82.35% <100.00%> (ø) 12.00 <0.00> (ø)
.../hedera/mirror/importer/downloader/Downloader.java 86.20% <100.00%> (+1.08%) 46.00 <2.00> (+1.00)
...ra/mirror/importer/downloader/PendingDownload.java 66.66% <100.00%> (+3.03%) 8.00 <1.00> (+1.00)
...irror/importer/reader/event/EventFileReaderV3.java 97.56% <100.00%> (+0.12%) 7.00 <0.00> (ø)
...or/monitor/expression/ExpressionConverterImpl.java 88.88% <100.00%> (+0.17%) 10.00 <0.00> (ø)
... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 896b905...4e6beb6. Read the comment docs.

Signed-off-by: Steven Sheehy <steven.sheehy@hedera.com>
@steven-sheehy steven-sheehy deleted the cloud-metrics branch April 14, 2021 03:43
@steven-sheehy steven-sheehy restored the cloud-metrics branch April 14, 2021 03:48
@steven-sheehy steven-sheehy reopened this Apr 14, 2021
@sonarcloud
Copy link

sonarcloud bot commented Apr 14, 2021

Copy link
Contributor

@ijungmann ijungmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@Nana-EC Nana-EC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor improvement suggestion

Copy link
Collaborator

@Nana-EC Nana-EC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@xin-hedera xin-hedera left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@steven-sheehy steven-sheehy merged commit 4c5949b into master Apr 14, 2021
@steven-sheehy steven-sheehy deleted the cloud-metrics branch April 14, 2021 20:08
ijungmann pushed a commit that referenced this pull request May 18, 2021
- Add `hedera.mirror.importer.cloud.latency` metric to compare consensus time against cloud storage creation time
- Add `hedera.mirror.download.latency` metric to all stream types
- Add `hedera.mirror.stream.close.latency` metric to all stream types
- Fix event file not persisting due to missing required consensus end field
- Fix cloud storage response size and time metrics not counting `_sig.gz` files as signature files

Signed-off-by: Steven Sheehy <steven.sheehy@hedera.com>
Signed-off-by: Ian Jungmann <ian.jungmann@hedera.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
downloader Area: S3 downloader enhancement Type: New feature P2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants