Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker: initial Dockerfile support #2046

Merged
merged 10 commits into from
May 9, 2023

Conversation

QSmally
Copy link
Contributor

@QSmally QSmally commented Apr 20, 2023

Summary

This PR implements a basic Dockerfile in contrast to using Python virtual environment (as I personally had some dependency problems with it). It's basic, as in, it installs the CPU dependencies and leaves CUDA out of the mix. Maybe with build-args you can change some things but I don't have a Nvidia setup to test it.

How to test

$ cd training_extensions/
$ docker build -t trainer .
$ docker run -it --rm trainer
trainer% otx # ...

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added e2e tests for validation.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

@QSmally QSmally requested a review from a team as a code owner April 20, 2023 19:31
@ote-ci-bot
Copy link
Contributor

Can one of the admins verify this patch?

@github-actions github-actions bot added the DOC Improvements or additions to documentation label Apr 20, 2023
@samet-akcay
Copy link
Contributor

@ashwinvaidya17, can you review this PR please?

Copy link
Contributor

@ashwinvaidya17 ashwinvaidya17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. I have one comment regarding the placement of this file. The rest looks good.

Dockerfile Show resolved Hide resolved
Dockerfile Outdated Show resolved Hide resolved
ashwinvaidya17
ashwinvaidya17 previously approved these changes Apr 21, 2023
Copy link
Contributor

@goodsong81 goodsong81 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution @QSmally!
It looks generally OK but one only concern of mine is that the majority of use-cases of model training will run on GPUs. Current PR covers only Python3.9 + CPU.
@samet-akcay What do you think? Will we start from CPU and extend GPU support later?

@QSmally
Copy link
Contributor Author

QSmally commented Apr 24, 2023

Thank you for the contribution @QSmally!
It looks generally OK but one only concern of mine is that the majority of use-cases of model training will run on GPUs. Current PR covers only Python3.9 + CPU.

Ty @goodsong81, @ashwinvaidya17, @yunchu,

I implemented build-arg support for UBUNTU_VER, PYTHON_VER and SOURCE. There's an issue where python3.10-dev and python3.10-distutils don't exist but I did get it working with Python 3.8 (along with Ubuntu 18.04). The default values if no args were given are 20.04, 3.9 and /whl/cpu as given in the README.

However, I lack the installation knowledge and machine to get the GPU dependencies, so I'd like assistance with that

@samet-akcay
Copy link
Contributor

Thank you for the contribution @QSmally!
It looks generally OK but one only concern of mine is that the majority of use-cases of model training will run on GPUs. Current PR covers only Python3.9 + CPU.

Ty @goodsong81, @ashwinvaidya17, @yunchu,

I implemented build-arg support for UBUNTU_VER, PYTHON_VER and SOURCE. There's an issue where python3.10-dev and python3.10-distutils don't exist but I did get it working with Python 3.8 (along with Ubuntu 18.04). The default values if no args were given are 20.04, 3.9 and /whl/cpu as given in the README.

However, I lack the installation knowledge and machine to get the GPU dependencies, so I'd like assistance with that

Thanks for the contribution @QSmally. In this case, GPU support could be addressed by a follow-up PR by the team later.

ashwinvaidya17
ashwinvaidya17 previously approved these changes Apr 24, 2023
Copy link
Contributor

@ashwinvaidya17 ashwinvaidya17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

yunchu
yunchu previously approved these changes Apr 25, 2023
Copy link
Contributor

@yunchu yunchu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. @QSmally, Thank you for your contributions!

@goodsong81
Copy link
Contributor

@ashwinvaidya17 @yunchu Could you help pass the Dockerfile style check? I think the CI is regarding the warning as error.
image

@yunchu
Copy link
Contributor

yunchu commented Apr 25, 2023

@goodsong81, we can update linter setting for dockerfile but I think it's better to resolve all warnings.
@QSmally, Can you apply some changes to resolve warning issues from linter? I recommend to use pre-commit tool to your development environment to get the direct feedback from the linters while generating the commits.

@QSmally
Copy link
Contributor Author

QSmally commented Apr 25, 2023

I rarely encounter a Dockerfile with pinned apt versions, so I'm not sure how (and if) we should update that, @yunchu/@goodsong81

image

@yunchu
Copy link
Contributor

yunchu commented Apr 25, 2023

@QSmally
You can add the comment below to disable that warning on right before the apt-get install command in your Dockerfile.

# hadolint ignore=DL3008 

@yunchu
Copy link
Contributor

yunchu commented Apr 25, 2023

@QSmally I'm not sure but https://github.com/openvinotoolkit/training_extensions/blob/develop/.ci/Dockerfile this one could be the example for you.

Dockerfile Outdated Show resolved Hide resolved
Dockerfile Outdated Show resolved Hide resolved
Dockerfile Show resolved Hide resolved
Copy link
Contributor

@goodsong81 goodsong81 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution!

@goodsong81 goodsong81 merged commit d5da3c2 into openvinotoolkit:develop May 9, 2023
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DOC Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants