Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MAINTENANCE] Initial Dockerfile for running tests and associated README. #5541

Merged
merged 5 commits into from
Jul 20, 2022

Conversation

billdirks
Copy link
Contributor

@billdirks billdirks commented Jul 20, 2022

This creates a dockerfile that builds an image with Great Expectations and installs the dependencies. The plan is to use this as a reproducible environment for testing. Currently the tests do not pass when run on the container but that issue is with the tests vs the construction of the image. Follow up PRs will address these tests as well as integrating it into our CI pipeline.

This is for GREAT-1048

Please annotate your PR title to describe what the PR does, then give a brief bulleted description of your PR below. PR titles should begin with [BUGFIX], [FEATURE], [DOCS], or [MAINTENANCE]. If a new feature introduces breaking changes for the Great Expectations API or configuration files, please also add [BREAKING]. You can read about the tags in our contributor checklist.

Changes proposed in this pull request:

After submitting your PR, CI checks will run and @cla-bot will check for your CLA signature.

For a PR with nontrivial changes, we review with both design-centric and code-centric lenses.

In a design review, we aim to ensure that the PR is consistent with our relationship to the open source community, with our software architecture and abstractions, and with our users' needs and expectations. That review often starts well before a PR, for example in github issues or slack, so please link to relevant conversations in notes below to help reviewers understand and approve your PR more quickly (e.g. closes #123).

Previous Design Review notes:

Definition of Done

Please delete options that are not relevant.

  • My code follows the Great Expectations style guide
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added unit tests where applicable and made sure that new and existing tests are passing.
  • I have run any local integration tests and made sure that nothing is broken.

Thank you for submitting!

@netlify
Copy link

netlify bot commented Jul 20, 2022

Deploy Preview for niobium-lead-7998 failed.

Name Link
🔨 Latest commit c0542ba
🔍 Latest deploy log https://app.netlify.com/sites/niobium-lead-7998/deploys/62d7494c74fb04000cccd11d

@ghost
Copy link

ghost commented Jul 20, 2022

👇 Click on the image for a new way to code review
  • Make big changes easier — review code in small groups of related files

  • Know where to start — see the whole change at a glance

  • Take a code tour — explore the change with an interactive tour

  • Make comments and review — all fully sync’ed with github

    Try it now!

Review these changes using an interactive CodeSee Map

Legend

CodeSee Map Legend

@netlify
Copy link

netlify bot commented Jul 20, 2022

Deploy Preview for niobium-lead-7998 ready!

Name Link
🔨 Latest commit 2bfe1cd
🔍 Latest deploy log https://app.netlify.com/sites/niobium-lead-7998/deploys/62d87362d6637d00092164b2
😎 Deploy Preview https://deploy-preview-5541--niobium-lead-7998.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

Copy link
Contributor

@alexsherstinsky alexsherstinsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM -- will we have dockercompose for this? Thanks!

Comment on lines +7 to +8
These commands will hopefully be scripted but they are presented here for the time being. These commands **require**
they are invoked in the root directory of this repo.
Copy link
Member

@Kilo59 Kilo59 Jul 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might make a good invoke task.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed!

WORKDIR /great_expectations

FROM build_${SOURCE} AS dev
RUN pip install --requirement requirements.txt --requirement requirements-dev.txt \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should be able to just install the requirements-dev.txt file since that references all the other files that are explicitly listed

Copy link
Contributor Author

@billdirks billdirks Jul 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out! I was following what we currently do in the azure-pipelines.yml file but hadn't looked to closely at the contents of these requirement files. I'll give them a look and update. Thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@billdirks
Copy link
Contributor Author

@alexsherstinsky I'm going to write up a document of my plan and steps going forward this morning. The main motivation of the current work is to get a reproducible image that will run in CI. I don't believe we'll need to stand up other services for this work so I don't think I'll be creating a docker-compose file. I would like to get a fully functional system locally though so we'll see if that gets prioritized after this initial work is done.

@billdirks billdirks enabled auto-merge (squash) July 20, 2022 21:28
@billdirks billdirks merged commit b4c4c7a into develop Jul 20, 2022
@billdirks billdirks deleted the docker-test-image branch July 20, 2022 21:30
lccasagrande pushed a commit to lccasagrande/great_expectations that referenced this pull request Jul 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants