Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(#1132): Introduce async version of our API #1391

Merged
merged 22 commits into from Apr 25, 2022
Merged

Conversation

dcfidalgo
Copy link
Contributor

@dcfidalgo dcfidalgo commented Apr 7, 2022

Closes #1132

TODO:

  • tests
  • documentation
  • improve docstrings
  • guides
  • add examples
  • Log in batches. Server performance can be compromised otherwise (NOT IN THIS PR)
  • Add logging traces (success and failed logs)

@dcfidalgo dcfidalgo self-assigned this Apr 7, 2022
@dcfidalgo dcfidalgo added this to In progress in Release via automation Apr 7, 2022
@dcfidalgo
Copy link
Contributor Author

@frascuchon I did the refactoring discussed during the call. What's missing is updating the tests, documentation/docstrings, and maybe an improved behavior of the monitoring module (several retries, some automatic chunking, error messages), but maybe this can also be tackled in a separate PR.

@frascuchon frascuchon removed this from In progress in Release Apr 7, 2022
@frascuchon frascuchon marked this pull request as ready for review April 13, 2022 11:17
@codecov
Copy link

codecov bot commented Apr 13, 2022

Codecov Report

Merging #1391 (0bec916) into master (31c7939) will increase coverage by 0.14%.
The diff coverage is 88.70%.

@@            Coverage Diff             @@
##           master    #1391      +/-   ##
==========================================
+ Coverage   94.53%   94.67%   +0.14%     
==========================================
  Files         130      129       -1     
  Lines        5889     5898       +9     
==========================================
+ Hits         5567     5584      +17     
+ Misses        322      314       -8     
Flag Coverage Δ
pytest 94.67% <88.70%> (+0.14%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/rubrix/__init__.py 70.58% <ø> (ø)
src/rubrix/server/commons/es_wrapper.py 87.40% <ø> (-0.75%) ⬇️
src/rubrix/monitoring/base.py 71.42% <35.71%> (-2.11%) ⬇️
src/rubrix/client/api.py 96.26% <90.56%> (+1.09%) ⬆️
src/rubrix/client/rubrix_client.py 80.35% <100.00%> (-0.24%) ⬇️
src/rubrix/client/sdk/commons/api.py 89.13% <100.00%> (+4.75%) ⬆️
src/rubrix/client/sdk/text2text/api.py 100.00% <100.00%> (ø)
src/rubrix/client/sdk/text_classification/api.py 100.00% <100.00%> (ø)
src/rubrix/client/sdk/token_classification/api.py 100.00% <100.00%> (ø)
src/rubrix/monitoring/_flair.py 86.66% <100.00%> (-0.44%) ⬇️
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e4d7565...0bec916. Read the comment docs.

@frascuchon
Copy link
Member

frascuchon commented Apr 13, 2022

Since I didn't find a use case just using async log_async directly, I've included a background argument in the log function that could simplify the custom data monitoring.

future= rb.log(
    name="monitor-dataset",
    records=rb_records,
    background=True, 
)

# you can always wait for the results:
future.result()

Anyway, the async log version remains visible in API in order to discover some use case using directly async function

I've also included a minimal example in monitoring guides using bentoML + spaCy.

Please, feel free to add your comments

@dvsrepo @dcfidalgo

docs/guides/monitoring.ipynb Outdated Show resolved Hide resolved
docs/guides/monitoring.ipynb Outdated Show resolved Hide resolved
Co-authored-by: Daniel Vila Suero <daniel@recogn.ai>
@frascuchon frascuchon requested a review from dvsrepo April 22, 2022 13:41
dvsrepo
dvsrepo previously approved these changes Apr 22, 2022
@dcfidalgo
Copy link
Contributor Author

I removed the Api._log method since with the new implementation there is no need for this helper method. The example in the monitor guide looks good to me, I just added a small example in the docstring of the rb.log method to simply show the effect of setting background=True. Also improved a bit this docstring.

From my side, this is good to go.

Copy link
Member

@frascuchon frascuchon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

YEAH!

@frascuchon frascuchon merged commit aa67ed5 into master Apr 25, 2022
@frascuchon frascuchon deleted the feat/async_log branch April 25, 2022 16:07
frascuchon added a commit that referenced this pull request May 4, 2022
frascuchon added a commit that referenced this pull request May 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Monitoring] Introduce a more general async log
3 participants