Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initialize this repo #1

Closed
fnesveda opened this issue Jul 7, 2023 · 0 comments · Fixed by #2
Closed

Initialize this repo #1

fnesveda opened this issue Jul 7, 2023 · 0 comments · Fixed by #2
Assignees
Labels
enhancement New feature or request. t-tooling Issues with this label are in the ownership of the tooling team.

Comments

@fnesveda
Copy link
Member

fnesveda commented Jul 7, 2023

Right now, we have two major Python projects - the Apify SDK and the Apify API client. They're both using some of the same constants and utilities, and right now they're just copied between the two repositories, because it was the easiest way to get the projects out quickly.

In the long term, we're planning to have more Python libraries and use more shared constants and utilities, so it will be beneficial to have those defined here, in one place, and import them from here in other projects (similar to how we do it with https://github.com/apify/apify-shared-js and JavaScript libraries).

To do that, we need to:

  • create a new project on PyPI called apify-shared
  • copy over the project setup from e.g. https://github.com/apify/apify-client-python to here, and change the library name from apify-client to apify-shared
  • copy the constants and the utilities used in the API client and SDK to here (only the relevant ones, some of them make sense only in the context of one project, so they can stay there)
    • they're typically in the consts.py and _utils.py files in the project repositories
  • publish this on PyPI (the CI copied from the API client should work for this)
  • replace the constants in the API client and SDK with those from the shared package (and reexport them for convenience)

There are some things to keep in mind when working on this:

  • documentation
    • this project won't have its own documentation, so we can remove all the docs workflows
    • we want to have the constants (not necessarily the utilities yet, because those are all internal) documented in the SDK and API client docs, so we should somehow set up the documentation building there to merge the documentation from the main project to include them
      • it will be a bit ugly, but I think the easiest would be to git clone the right tag of this repo when building the project docs, build the documentation for this project, and then merge the resulting API reference JSON in the project's API reference JSON.
  • sharing the repo setup across Python projects
    • if we add more and more Python projects, we should somehow share the linter & type checker configs between the repos, so if we e.g. add a linter rule, we don't have to add it to every repo separately
    • this is not necessary for this repo's initialization, just something to start thinking about
  • sharing constants between JS and Python
    • there will be a lot of constants that are shared between JS and Python (probably almost all of them)
    • we should start thinking about how to make sure they're the same
      • either we should single-source them and automatically generate the JS and Python packages
      • but probably it would be enough to just write a CI job which checks that they're the same regularly, there won't be that many changes to make single-sourcing worth the effort
    • again, this is not necessary for this repo's initialization, just something to start thinking about
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request. t-tooling Issues with this label are in the ownership of the tooling team.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants