Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some missing backends to generated docs #14290

Merged

Conversation

Eric-Arellano
Copy link
Contributor

I discovered we're missing some backends like google_cloud_function.python.

Our current approach of matching all backends available (init/BUILD) against what we activate internally (pants.toml) against generate_docs.py has proven too error-prone. So, I'm simplifying by ignoring what we have internally and simply listing in generate_docs.py everything that is publishable.

Copy link
Member

@kaos kaos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I'm simplifying by ignoring what we have internally

I don't see that we are.

As the backends are still using --backend-packages=-[..] and ..=+[...] to add/remove entries to what is already configured, right?

IIUC this simply explicitly lists all backends that we do want to have docs generated for, in addition to what is not explicitly excluded from those enabled in pants.toml..?

@Eric-Arellano
Copy link
Contributor Author

As the backends are still using --backend-packages=-[..] and ..=+[...] to add/remove entries to what is already configured, right?

Oh wait that's why simpler to stop using +. Good idea!

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
f"--backend-packages=-[{', '.join(map(repr, deactivated_backends))}]",
f"--backend-packages=+[{', '.join(map(repr, activated_backends))}]",
"--plugins=[]",
f"--backend-packages={repr(backends)}",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

idkay, but I like ..={backends!r}.. being more succinct ;)

"pants.backend.python.typecheck.mypy",
"pants.backend.shell",
"pants.backend.shell.lint.shellcheck",
"pants.backend.shell.lint.shfmt",
]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI this is the list of backends. Any you think I should add? I left off these:

  • src/python/pants/backend/experimental/codegen/avro/java
  • src/python/pants/backend/experimental/codegen/protobuf/{java,scala}
  • src/python/pants/backend/experimental/codegen/thrift/apache/java
  • src/python/pants/backend/experimental/codegen/thrift/scrooge/{java,scala}
  • src/python/pants/backend/experimental/debian
  • src/python/pants/backend/experimental/go/lint/vet
  • src/python/pants/backend/experimental/{java,scala}/debug_goals
  • src/python/pants/backend/experimental/python
  • src/python/pants/backend/experimental/terraform

"src/python/pants/backend/awslambda/python",
"src/python/pants/backend/codegen/protobuf/python",
"src/python/pants/backend/experimental/codegen/avro/java",
"src/python/pants/backend/experimental/codegen/protobuf/java",
"src/python/pants/backend/experimental/codegen/protobuf/scala",
"src/python/pants/backend/experimental/codegen/thrift/apache/java",
"src/python/pants/backend/experimental/codegen/thrift/apache/python",
"src/python/pants/backend/experimental/codegen/thrift/scrooge/java",
"src/python/pants/backend/experimental/codegen/thrift/scrooge/scala",
"src/python/pants/backend/experimental/debian",
"src/python/pants/backend/experimental/docker",
"src/python/pants/backend/experimental/docker/lint/hadolint",
"src/python/pants/backend/experimental/go",
"src/python/pants/backend/experimental/go/lint/vet",
"src/python/pants/backend/experimental/java",
"src/python/pants/backend/experimental/java/debug_goals",
"src/python/pants/backend/experimental/java/lint/google_java_format",
"src/python/pants/backend/experimental/python",
"src/python/pants/backend/experimental/python/lint/autoflake",
"src/python/pants/backend/experimental/python/lint/pyupgrade",
"src/python/pants/backend/experimental/scala",
"src/python/pants/backend/experimental/scala/debug_goals",
"src/python/pants/backend/experimental/scala/lint/scalafmt",
"src/python/pants/backend/experimental/terraform",
"src/python/pants/backend/google_cloud_function/python",
"src/python/pants/backend/plugin_development",
"src/python/pants/backend/project_info",
"src/python/pants/backend/python",
"src/python/pants/backend/python/lint/bandit",
"src/python/pants/backend/python/lint/black",
"src/python/pants/backend/python/lint/docformatter",
"src/python/pants/backend/python/lint/flake8",
"src/python/pants/backend/python/lint/isort",
"src/python/pants/backend/python/lint/pylint",
"src/python/pants/backend/python/lint/yapf",
"src/python/pants/backend/python/mixed_interpreter_constraints",
"src/python/pants/backend/python/typecheck/mypy",
"src/python/pants/backend/shell",
"src/python/pants/backend/shell/lint/shellcheck",
"src/python/pants/backend/shell/lint/shfmt",
"src/python/pants/core",

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps some (if not all) stuff in experimental python would be nice to include.. ?
Unless they're in such a state that not documenting them is preferable.. ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like that backend is necessary for Twine and publish to have automated references, which would be good. I don't love that user-generate-lockfiles will also be picked up, but I just finished deprecating that goal, so that's fine with me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @tdyas as the author of several of these backends

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd leave out the non-Python codegen backends for now since there is additional work needed to get them actually working.

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
@Eric-Arellano Eric-Arellano merged commit 3401220 into pantsbuild:main Jan 28, 2022
@Eric-Arellano Eric-Arellano deleted the google-cloud-function-docs branch January 28, 2022 15:52
Eric-Arellano added a commit to Eric-Arellano/pants that referenced this pull request Jan 28, 2022
I discovered we're missing some backends like `google_cloud_function.python`.

Our current approach of matching all backends available (`init/BUILD`) against what we activate internally (`pants.toml`) against `generate_docs.py` has proven too error-prone. So, I'm simplifying by ignoring what we have internally and simply listing in `generate_docs.py` everything that is publishable.

[ci skip-rust]
Eric-Arellano added a commit that referenced this pull request Jan 28, 2022
…14292)

I discovered we're missing some backends like `google_cloud_function.python`.

Our current approach of matching all backends available (`init/BUILD`) against what we activate internally (`pants.toml`) against `generate_docs.py` has proven too error-prone. So, I'm simplifying by ignoring what we have internally and simply listing in `generate_docs.py` everything that is publishable.

[ci skip-rust]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants