Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: moving jinad into daemon module of the core #1610

Merged
merged 18 commits into from
Jan 7, 2021
Merged

Conversation

hanxiao
Copy link
Member

@hanxiao hanxiao commented Jan 6, 2021

  • code
  • test
  • Dockerfile
  • setup
  • README (entrypoint changed, will rewrite in the next PR)
  • API docs generation
  • GH workflows

@jina-bot jina-bot added size/XL area/testing This issue/PR affects testing labels Jan 6, 2021
@github-actions
Copy link

github-actions bot commented Jan 6, 2021

Latency summary

Current PR yields:

  • 😶 index QPS at 2067, delta to last 3 avg.: +3%
  • 😶 query QPS at 37, delta to last 3 avg.: -2%

Breakdown

Version Index QPS Query QPS
current 2067 37
0.9.2 1976 37
0.9.1 1978 37
0.8.22 2038 37

Backed by latency-tracking. Further commits will update this comment.

@jina-bot jina-bot added the area/setup This issue/PR affects setting up Jina label Jan 6, 2021
@jina-bot jina-bot added the area/docker This issue/PR affects the docker functionality label Jan 6, 2021
@jina-bot jina-bot added area/cicd This issue/PR affects the cicd pipeline area/housekeeping This issue/PR is housekeeping labels Jan 7, 2021
def _start_fluentd():
daemon_logger.info('starting fluentd')
cfg = pkg_resources.resource_filename('jina', 'resources/fluent.conf')
subprocess.Popen(['fluentd', '-c', cfg])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fluentd also ships as a daemon (td-agent), which means we can have systemctl manage it along with jinad. We can avoid having fluentd invocation inside python code, as this makes jinad unusable without fluentd command.
https://github.com/jina-ai/jinad/blob/main/scripts/deb-systemd.sh#L49

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's handle this as another PR. right now it is too ahead to put systemctl together, the goal of this PR is to make the jinad usable in CICD.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw,

curl -L https://raw.githubusercontent.com/jina-ai/jinad/main/scripts/deb-systemd.sh | bash

this is good, but later will make it shorter, something like curl -L https://daemon-sh.jina.ai | bash

@jina-bot jina-bot added area/core This issue/PR affects the core codebase component/flow labels Jan 7, 2021
@jina-bot jina-bot added the area/cli This issue/PR affects the command line interface label Jan 7, 2021
@codecov
Copy link

codecov bot commented Jan 7, 2021

Codecov Report

Merging #1610 (8b5efe4) into master (7200c65) will increase coverage by 0.96%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1610      +/-   ##
==========================================
+ Coverage   83.98%   84.95%   +0.96%     
==========================================
  Files         127      126       -1     
  Lines        6652     6648       -4     
==========================================
+ Hits         5587     5648      +61     
+ Misses       1065     1000      -65     
Impacted Files Coverage Δ
jina/flow/base.py 86.80% <ø> (+0.32%) ⬆️
jina/parsers/peapods/runtimes/remote.py 100.00% <100.00%> (ø)
jina/types/document/multimodal.py 98.00% <100.00%> (+0.08%) ⬆️
jina/types/document/uid.py 84.61% <100.00%> (+3.36%) ⬆️
jina/logging/profile.py 69.84% <0.00%> (-0.56%) ⬇️
jina/drivers/craft.py 100.00% <0.00%> (ø)
jina/types/ndarray/generic.py 100.00% <0.00%> (ø)
jina/excepts.py
jina/drivers/encode.py 94.91% <0.00%> (+0.08%) ⬆️
jina/enums.py 96.59% <0.00%> (+0.09%) ⬆️
... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dfe3033...c33f975. Read the comment docs.

@hanxiao hanxiao marked this pull request as ready for review January 7, 2021 23:13
@hanxiao hanxiao requested a review from a team as a code owner January 7, 2021 23:13
@hanxiao hanxiao merged commit a338e51 into master Jan 7, 2021
@hanxiao hanxiao deleted the refactor-jinad branch January 7, 2021 23:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/cicd This issue/PR affects the cicd pipeline area/cli This issue/PR affects the command line interface area/core This issue/PR affects the core codebase area/docker This issue/PR affects the docker functionality area/housekeeping This issue/PR is housekeeping area/setup This issue/PR affects setting up Jina area/testing This issue/PR affects testing component/flow size/XL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants