Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add regression for chatbot #2218

Merged
merged 8 commits into from Mar 25, 2021
Merged

test: add regression for chatbot #2218

merged 8 commits into from Mar 25, 2021

Conversation

bwanglzu
Copy link
Member

@bwanglzu bwanglzu commented Mar 22, 2021

Given a system under test, test_chatbot depends on the index flow to be fired. While f.block blocked the process. I did some search and find the official plugin: pytest-xprocess

pytest plugin for managing processes across test runs.

This plugin:

  1. Bootstrap hello_world example in another process.
  2. Monitor system logs to check if it is ready to test using pattern matching.
  3. Run the test.
  4. Kill the process.

This PR completes regression tests for three examples.

@bwanglzu bwanglzu self-assigned this Mar 22, 2021
@jina-bot jina-bot added size/S area/testing This issue/PR affects testing labels Mar 22, 2021
@codecov
Copy link

codecov bot commented Mar 22, 2021

Codecov Report

Merging #2218 (0de3afa) into master (5c5cfef) will decrease coverage by 0.08%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2218      +/-   ##
==========================================
- Coverage   89.12%   89.03%   -0.09%     
==========================================
  Files         208      208              
  Lines       11226    11226              
==========================================
- Hits        10005     9995      -10     
- Misses       1221     1231      +10     
Flag Coverage Δ
daemon 50.26% <100.00%> (ø)
jina 89.38% <100.00%> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
jina/__init__.py 73.49% <100.00%> (ø)
jina/clients/base.py 78.51% <0.00%> (-2.48%) ⬇️
jina/peapods/zmq/__init__.py 80.80% <0.00%> (-2.17%) ⬇️
jina/peapods/runtimes/zmq/zed.py 87.94% <0.00%> (-1.42%) ⬇️
jina/peapods/peas/__init__.py 93.69% <0.00%> (+1.80%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba702b4...0de3afa. Read the comment docs.

@github-actions
Copy link

github-actions bot commented Mar 22, 2021

Latency summary

Current PR yields:

  • 😶 index QPS at 1000, delta to last 3 avg.: +0%
  • 😶 query QPS at 14, delta to last 3 avg.: -5%

Breakdown

Version Index QPS Query QPS
current 1000 14
1.0.13 994 14
1.0.12 1011 15

Backed by latency-tracking. Further commits will update this comment.

@jina-bot jina-bot added the area/core This issue/PR affects the core codebase label Mar 22, 2021
@jina-bot jina-bot added the area/setup This issue/PR affects setting up Jina label Mar 23, 2021
@bwanglzu bwanglzu linked an issue Mar 23, 2021 that may be closed by this pull request
4 tasks
@bwanglzu bwanglzu marked this pull request as ready for review March 23, 2021 07:30
@bwanglzu bwanglzu requested review from hanxiao and a team as code owners March 23, 2021 07:30
@bwanglzu
Copy link
Member Author

bwanglzu commented Mar 23, 2021

please DO NOT MERGE before CI fixed.

@bwanglzu bwanglzu requested review from deepankarm and nan-wang and removed request for Roshanjossey March 23, 2021 08:44
Copy link
Member

@nan-wang nan-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM👍

@nan-wang nan-wang merged commit 3d1b954 into master Mar 25, 2021
@nan-wang nan-wang deleted the test-regression-chatbot branch March 25, 2021 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/core This issue/PR affects the core codebase area/setup This issue/PR affects setting up Jina area/testing This issue/PR affects testing size/S
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Regression test for all three hello-world demos
4 participants