⚠️ CONFLICT! Lineage pull request for: skeleton #82

cisagovbot · 2021-07-19T17:33:50Z

Lineage Pull Request: CONFLICT

Lineage has created this pull request to incorporate new changes found in an
upstream repository:

Upstream repository: https://github.com/cisagov/skeleton-generic.git
Remote branch: HEAD

Check the changes in this pull request to ensure they won't cause issues with
your project.

The lineage/skeleton branch has one or more unresolved merge conflicts
that you must resolve before merging this pull request!

How to resolve the conflicts

Take ownership of this pull request by removing any other assignees.

Clone the repository locally, and reapply the merge:

git clone git@github.com:cisagov/skeleton-python-library.git skeleton-python-library
cd skeleton-python-library
git remote add skeleton https://github.com/cisagov/skeleton-generic.git
git remote set-url --push skeleton no_push
git switch develop
git checkout -b lineage/skeleton --track origin/develop
git pull skeleton HEAD
git status

Review the changes displayed by the status command. Fix any conflicts and
possibly incorrect auto-merges.
After resolving each of the conflicts, add your changes to the
branch, commit, and push your changes:
```
git add .github/workflows/build.yml 
git commit
git push --force --set-upstream origin lineage/skeleton
```
Note that you may append to the default merge commit message
that git creates for you, but please do not delete the existing
content. It provides useful information about the merge that is
being performed.
Wait for all the automated tests to pass.
Check the "Everything is cool" checkbox below:
- ✌️ The conflicts in this pull request have been resolved.
Mark this draft pull request "Ready for review".

Note: You are seeing this because one of this repository's maintainers has
configured Lineage to open pull requests.

For more information:

🛠 Lineage configurations for this project are stored in .github/lineage.yml

📚 Read more about Lineage

Add rules to enforce ATX-closed headers, dashes for unordered list elements, and `1.` for ordered list elements.

Add <h1> and <img> tags to the allowed list for MD033 (HTML elements) to support using an image as the first thing in a markdown file (header image).

This converts the existing `.mdl_config.json` file to an equivalent `.mdl_config.yaml` file. The reference in the markdownlint pre-commit hook configuration is updated to match. Co-authored-by: Shane Frasier <jeremy.frasier@trio.dhs.gov>

Now that this is a YAML file we can add comments explaining the rule modifications we use. This will make it easier to edit or expand in the future.

This hook will validate any pre-commit hook manifest files in the repository.

We should be doing this because the Packer and Terraform pre-commit hooks leverage the corresponding executables; therefore, it makes sense to go ahead and install the particular versions of those executables that we support. Also add support for optionally debugging via tmate. See also cisagov/skeleton-generic#74.

Co-authored-by: Nick M. <50747025+mcdonnnj@users.noreply.github.com>

There is no reason to create /usr/bin/terraform. This is a vestige of an earlier age. Co-authored-by: Nick M. <50747025+mcdonnnj@users.noreply.github.com>

The Terraform installation does not destroy the existing system Terraform installation, and neither should the Packer installation. Co-authored-by: Nick M. <50747025+mcdonnnj@users.noreply.github.com>

Note that this change is dependent on the merging of cisagov/setup-env-github-action#31. Co-authored-by: Nick M. <50747025+mcdonnnj@users.noreply.github.com>

Some variables defined in the go installation are used in the cache task, so the go installation must happen first. Co-authored-by: Nick M. <50747025+mcdonnnj@users.noreply.github.com>

Co-authored-by: Nick M. <50747025+mcdonnnj@users.noreply.github.com>

As of [Go 1.16](https://tip.golang.org/doc/go1.16#go-command) the `GO111MODULE` environment variable defaults to `on` and `go get` has been deprecated for module installation. Co-authored-by: Nick M. <50747025+mcdonnnj@users.noreply.github.com>

The `shfmt` tool does not ship on the GitHub Actions runners so we must install it manually.

We have had a difficult time with how beautysh parses some shellscripts. I went in pursuit of an alternative and I believe shfmt to be a good alternative. Co-authored-by: Shane Frasier <jeremy.frasier@trio.dhs.gov>

This is performed by running `pre-commit autoupdate`, but with the `ansible-lint` hook held back manually.

Update pre-commit Hooks

Add the validate_manifest Hook to pre-commit Configuration

…ting

Update markdownlint Configuration

…ting

…-for-linting Install terraform and packer for the linting job

…_shfmt Replace beautysh hook with cisagov/pre-commit-shfmt

With the release of 0.900 mypy moved to a modular stub model. This means that only stdlib stubs are "baked in" and everything else must be installed separately. Since the mypy hook runs in a venv the stubs it recommends need to be installed as additional dependencies.

jsf9k

One item I thought of that we should resolve before merging this.

jsf9k · 2021-07-20T18:14:24Z

.pre-commit-config.yaml

+        additional_dependencies:
+          - types-setuptools


Should we also add these dependencies to requirements-test.txt or requirements-dev.txt so that mypy also works from the developer's local venv?

If so, then is there any way to add a requirements-mypy.txt, say, that can be pulled in by both requirements-test.txt (or requirements-dev.txt - I'm not certain where it should go) and our pre-commit configuration file? I'd prefer to add these dependencies in only one place.

I think it'd be cleaner to leverage mypy --install-types --non-interactive src/ in setup-env to automatically install them to the local environment as part of setup.

Since that also goes through type checking, we could eschew the --non-interactive part (which will also force a user to interact with the setup-env script during setup).

I think it'd be cleaner to leverage mypy --install-types --non-interactive src/ in setup-env to automatically install them to the local environment as part of setup.

OK, that makes sense. That's a change we'd want to make upstream, in skeleton-generic right?

Since that also goes through type checking, we could eschew the --non-interactive part (which will also force a user to interact with the setup-env script during setup).

I don't understand what you're saying here.

Sorry the --non-interactive step will also proceed with mypy's type checking. Without that flag you have to do the --install-types step and then run mypy to do type checking. People wanted to do it all at once, but for our purposes it's ok to keep it broken up I believe.

I think it'd be cleaner to leverage mypy --install-types --non-interactive src/ in setup-env to automatically install them to the local environment as part of setup.

OK, that makes sense. That's a change we'd want to make upstream, in skeleton-generic right?

Currently mypy is only a development requirement for projects descending from this skeleton. We can either add it here or add it to cisagov/skeleton-generic and add mypy as a requirement as well.

I think it'd be cleaner to leverage mypy --install-types --non-interactive src/ in setup-env to automatically install them to the local environment as part of setup.

OK, that makes sense. That's a change we'd want to make upstream, in skeleton-generic right?

Never mind, I see now that mypy isn't even installed in skeleton-generic.

See commit 3452b9d.

We already do this as part of the mypy pre-commit hook, but this way mypy is also ready to run manually in the developer's local Python virtual env.

dav3r

Love it! 🏆

mcdonnnj and others added 27 commits June 18, 2021 03:33

Add style enforcement rules

1e8f822

Add rules to enforce ATX-closed headers, dashes for unordered list elements, and `1.` for ordered list elements.

Add rule for image headers

afc6bd6

Add <h1> and <img> tags to the allowed list for MD033 (HTML elements) to support using an image as the first thing in a markdown file (header image).

Switch to a YAML markdownlint configuration file

ce173f4

This converts the existing `.mdl_config.json` file to an equivalent `.mdl_config.yaml` file. The reference in the markdownlint pre-commit hook configuration is updated to match. Co-authored-by: Shane Frasier <jeremy.frasier@trio.dhs.gov>

Add comments to markdownlint configuration

f2a4230

Now that this is a YAML file we can add comments explaining the rule modifications we use. This will make it easier to edit or expand in the future.

Add the validate_manifest hook from pre-commit

d4781ee

This hook will validate any pre-commit hook manifest files in the repository.

Break out the curl cache creation into its own step

c481043

Co-authored-by: Nick M. <50747025+mcdonnnj@users.noreply.github.com>

Remove unnecessary line in tasks

70414cf

There is no reason to create /usr/bin/terraform. This is a vestige of an earlier age. Co-authored-by: Nick M. <50747025+mcdonnnj@users.noreply.github.com>

Modify the Packer installation to model that of Terraform

b629f7f

The Terraform installation does not destroy the existing system Terraform installation, and neither should the Packer installation. Co-authored-by: Nick M. <50747025+mcdonnnj@users.noreply.github.com>

Install a specific version of terraform-docs

181d1b2

Note that this change is dependent on the merging of cisagov/setup-env-github-action#31. Co-authored-by: Nick M. <50747025+mcdonnnj@users.noreply.github.com>

Move go installation so that it takes place before the cache task

bb6e566

Some variables defined in the go installation are used in the cache task, so the go installation must happen first. Co-authored-by: Nick M. <50747025+mcdonnnj@users.noreply.github.com>

Capitalize Go for consistency

337d1ef

Co-authored-by: Nick M. <50747025+mcdonnnj@users.noreply.github.com>

Prefer the newer "go install" syntax

8ee2116

As of [Go 1.16](https://tip.golang.org/doc/go1.16#go-command) the `GO111MODULE` environment variable defaults to `on` and `go get` has been deprecated for module installation. Co-authored-by: Nick M. <50747025+mcdonnnj@users.noreply.github.com>

Install the shfmt tool for GHA

e2a729d

The `shfmt` tool does not ship on the GitHub Actions runners so we must install it manually.

Replace the beautysh hook with pre-commit-shfmt

406b688

We have had a difficult time with how beautysh parses some shellscripts. I went in pursuit of an alternative and I believe shfmt to be a good alternative. Co-authored-by: Shane Frasier <jeremy.frasier@trio.dhs.gov>

Apply changes from the shfmt pre-commit hook

2b48e75

Update pre-commit hooks

1708b5c

This is performed by running `pre-commit autoupdate`, but with the `ansible-lint` hook held back manually.

Merge pull request #83 from cisagov/maintenance/update_pre-commit_hooks

d7dcfee

Update pre-commit Hooks

Merge branch 'develop' into improvement/add_pre-commit_hook

c17800a

Merge pull request #80 from cisagov/improvement/add_pre-commit_hook

41a5286

Add the validate_manifest Hook to pre-commit Configuration

Merge branch 'develop' into improvement/update_mdl_configuration

e607360

Merge branch 'develop' into improvement/install-tf-and-packer-for-lin…

f6ad0e9

…ting

Merge pull request #79 from cisagov/improvement/update_mdl_configuration

382c39c

Update markdownlint Configuration

Merge branch 'develop' into improvement/install-tf-and-packer-for-lin…

3e3b918

…ting

Merge pull request #82 from cisagov/improvement/install-tf-and-packer…

0e4fc41

…-for-linting Install terraform and packer for the linting job

Merge branch 'develop' into improvement/replace_beautysh_with_shfmt

3e83a80

Merge pull request #84 from cisagov/improvement/replace_beautysh_with…

81cdb4d

…_shfmt Replace beautysh hook with cisagov/pre-commit-shfmt

cisagovbot assigned dav3r, felddy and jsf9k Jul 19, 2021

cisagovbot assigned mcdonnnj Jul 19, 2021

cisagovbot added the upstream update This issue or pull request pulls in upstream updates label Jul 19, 2021

mcdonnnj unassigned felddy, jsf9k and dav3r Jul 19, 2021

mcdonnnj added 3 commits July 20, 2021 12:54

Merge github.com:cisagov/skeleton-generic into lineage/skeleton

e45aec3

Changes from the pre-commit-shfmt hook

63cf76f

mcdonnnj force-pushed the lineage/skeleton branch from 5e32a17 to 77c19cb Compare July 20, 2021 17:07

mcdonnnj marked this pull request as ready for review July 20, 2021 17:20

mcdonnnj requested review from dav3r, felddy, jsf9k and mcdonnnj as code owners July 20, 2021 17:20

jsf9k approved these changes Jul 20, 2021

View reviewed changes

jsf9k requested changes Jul 20, 2021

View reviewed changes

Install mypy type stubs as part of setup-env

3452b9d

We already do this as part of the mypy pre-commit hook, but this way mypy is also ready to run manually in the developer's local Python virtual env.

jsf9k approved these changes Jul 20, 2021

View reviewed changes

dav3r approved these changes Jul 21, 2021

View reviewed changes

mcdonnnj merged commit 0efaf16 into develop Jul 22, 2021

mcdonnnj deleted the lineage/skeleton branch July 22, 2021 19:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

⚠️ CONFLICT! Lineage pull request for: skeleton #82

⚠️ CONFLICT! Lineage pull request for: skeleton #82

Uh oh!

cisagovbot commented Jul 19, 2021 •

edited by mcdonnnj

Loading

Uh oh!

jsf9k left a comment

Uh oh!

jsf9k Jul 20, 2021 •

edited

Loading

Uh oh!

mcdonnnj Jul 20, 2021

Uh oh!

mcdonnnj Jul 20, 2021

Uh oh!

jsf9k Jul 20, 2021

Uh oh!

jsf9k Jul 20, 2021

Uh oh!

mcdonnnj Jul 20, 2021

Uh oh!

mcdonnnj Jul 20, 2021

Uh oh!

jsf9k Jul 20, 2021

Uh oh!

jsf9k Jul 20, 2021

Uh oh!

dav3r left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

⚠️ CONFLICT! Lineage pull request for: skeleton #82

⚠️ CONFLICT! Lineage pull request for: skeleton #82

Uh oh!

Conversation

cisagovbot commented Jul 19, 2021 • edited by mcdonnnj Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Lineage Pull Request: CONFLICT

How to resolve the conflicts

Uh oh!

jsf9k left a comment

Choose a reason for hiding this comment

Uh oh!

jsf9k Jul 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dav3r left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

cisagovbot commented Jul 19, 2021 •

edited by mcdonnnj

Loading

jsf9k Jul 20, 2021 •

edited

Loading