Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Makes extensible metrics in remotecv #63

Merged
merged 1 commit into from
Dec 15, 2022
Merged

Makes extensible metrics in remotecv #63

merged 1 commit into from
Dec 15, 2022

Conversation

devppjr
Copy link
Contributor

@devppjr devppjr commented Nov 30, 2022

This PR aims to create metrics for remotecv and make it extensive for defining upstream metrics. So was added the metrics bellow:

For each detector:

  • latency (ms) - (worker.face.time, worker.feature.time, worker.glasses.time, worker.profile.time)
  • detected (if found points) - (worker.face.detected, worker.feature.detected, worker.glasses.detected, worker.profile.detected)

For original image, such as Thumbor does:

  • response_bytes - (worker.original_image.response_bytes)
  • status_code + host + latency - (worker.original_image.fetch.{code}.{netloc})
  • status_code + host + count - (worker.original_image.fetch.{code}.{netloc})
  • status_code + count - (worker.original_image.status.{code})

For each task (image to detect):

  • time (total time of detection) - (worker.pyres_task.time, worker.celery_task.time)
  • count - (worker.pyres_task.total, worker.celery_task.total)

closes #60

@coveralls
Copy link

coveralls commented Nov 30, 2022

Pull Request Test Coverage Report for Build 3704449885

  • 33 of 74 (44.59%) changed or added relevant lines in 10 files are covered.
  • 3 unchanged lines in 2 files lost coverage.
  • Overall coverage decreased (-2.0%) to 59.905%

Changes Missing Coverage Covered Lines Changed/Added Lines %
remotecv/importer.py 3 5 60.0%
remotecv/celery_tasks.py 0 5 0.0%
remotecv/worker.py 3 9 33.33%
remotecv/metrics/init.py 0 7 0.0%
remotecv/metrics/logger_metrics.py 0 7 0.0%
remotecv/http_loader.py 0 14 0.0%
Files with Coverage Reduction New Missed Lines %
remotecv/http_loader.py 1 0%
remotecv/celery_tasks.py 2 0%
Totals Coverage Status
Change from base Build 3677248203: -2.0%
Covered Lines: 305
Relevant Lines: 516

💛 - Coveralls

Copy link
Member

@heynemann heynemann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really good stuff, apart from how to measure time. Let's create a function in an utils.py file that returns current time for measurements (using time.perf_counter_ns). Then we can use it everywhere, instead of duplicating the code. If we decide to change it in the future it's easy. Also add a function to get the difference in seconds (get_interval_seconds(perf_counter_ns_start, perf_counter_ns_end)). This way if we want to change the function that returns the timings in the future, we can also change this one and the whole codebase just works.

remotecv/detectors/feature_detector/__init__.py Outdated Show resolved Hide resolved
@heynemann heynemann self-requested a review December 3, 2022 17:33
Copy link
Member

@heynemann heynemann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved by mistake. See comments in approval.

@devppjr devppjr force-pushed the metrics branch 7 times, most recently from 836c525 to e42d938 Compare December 8, 2022 14:41
Copy link
Member

@heynemann heynemann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! TYVM!

@heynemann
Copy link
Member

Will leave merge to after @guilhermef review :)

.github/workflows/codeql-analysis.yml Outdated Show resolved Hide resolved
remotecv/detectors/feature_detector/__init__.py Outdated Show resolved Hide resolved
@devppjr devppjr force-pushed the metrics branch 4 times, most recently from fa80b62 to 7703b07 Compare December 12, 2022 16:35
@guilhermef
Copy link
Member

It doesn't have to be done in this one, but I suggest a more modern approach like Prometheus metrics and taking advantage of the HTTP server we have running on the worker.
I'm also fine with the logger metrics.

@devppjr devppjr force-pushed the metrics branch 5 times, most recently from 87387ef to abc4ade Compare December 15, 2022 13:09
@codeclimate
Copy link

codeclimate bot commented Dec 15, 2022

Code Climate has analyzed commit 704a2e4 and detected 0 issues on this pull request.

View more on Code Climate.

@guilhermef
Copy link
Member

Awesome work @devppjr

@RaphaelVRossi
Copy link
Member

Great job @devppjr 🚀

@RaphaelVRossi RaphaelVRossi merged commit 48aef8a into master Dec 15, 2022
@devppjr devppjr deleted the metrics branch December 15, 2022 16:57
@RaphaelVRossi RaphaelVRossi mentioned this pull request Dec 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create metrics for Remotecv
5 participants