Skip to content

Conversation

@DomGarguilo
Copy link
Member

Fixes #4509

This PR adds two new metrics:

  1. accumulo.scan.reservation.writeout.timer which records the amount of time it takes for reservations to be added to the metadata table
  2. accumulo.scan.reservation.collision.count which adds a FunctionCounter that gets incremented if a scan is held up waiting for files that are in flux

I also renamed accumulo.scan.reservation.timer to accumulo.scan.reservation.total.timer.

supersedes #4541

@keith-turner
Copy link
Contributor

keith-turner commented May 21, 2024

@DomGarguilo sorry my writeup was awful for #4509. For the write time was looking to track how long it takes to call getContext().getAmple().putScanServerFileReferences(refs); here. This is the time it takes to write out scan server refs to the metadata table.

For collisions, my terminology was really bad. The event I wanted to track was the case where tablets files changed during the process of attempting to reserve files. That could be done by incrementing a metric immediately before returning null here. The log message a few lines up describes what is happening. Not sure colliosn is the best single word for the metric name that describes tablet files changed while attempting to reference files, but can not think of something more succinct.

# Conflicts:
#	server/tserver/src/main/java/org/apache/accumulo/tserver/ScanServerMetrics.java
@DomGarguilo
Copy link
Member Author

@keith-turner, I reimplemented things based on your feedback. Was able to see these values when metrics were set to be logged:


METRICS: 2024-05-21T16:23:46,854, accumulo.scan.reservation.total.timer{host=localhost,instance.name=uno,port=9996,process.name=sserver,resource.group=default} throughput=0.1/s mean=0.102909686s max=0.102909686s
METRICS: 2024-05-21T16:23:46,855, accumulo.scan.reservation.writeout.timer{host=localhost,instance.name=uno,port=9996,process.name=sserver,resource.group=default} throughput=0.1/s mean=0.025735159s max=0.025735159s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Add scan server metrics to track reservation write times and collisions

3 participants