New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dockerized pipeline #53
Conversation
docker/pipeline.yml
Outdated
context: .. | ||
dockerfile: docker/parsers/symbolscraper/Dockerfile | ||
|
||
# layoutparser: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Haven't added these yet. If we agree on this direction, I'll complete it.
If we do this, then we would move the current |
1e229be
to
23462d1
Compare
extras_require={"dev": ["pytest"]}, | ||
install_requires=["intervaltree", "tqdm", "pdf2image", "pdfplumber"], | ||
extras_require={ | ||
"dev": ["pytest"], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Split up the dependencies for the different predictors so you can install only what's needed.
23462d1
to
fa7d604
Compare
@rodneykinney as a part of this, do you mind adding a crude README that walks someone through usage of what youv'e set up here? i.e. assuming I've cloned the repo, what are the cmds I should execute until I can see a PDF processed? |
Added |
da308e0
to
f8e6ac7
Compare
Ready to merge from my point of view. Any other comments? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thx for readme; lgtm!
Pattern for wrapping services in a lighter-weight containers.
services
directory with aDockerfile
plus axxx_api.py
file for each service.pipeline.py
sample end-to-end pipeline script using the servicesrun-pipeline.sh
file to run pipeline@rauthur