-
Notifications
You must be signed in to change notification settings - Fork 399
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor artifact publishing #3132
Labels
infrastructure
Everything related to builds tools, CI configurations and project tooling
tech-debt
A technical issue that is not observable by the users, but improves maintainers quality of life
Milestone
Comments
This was referenced Aug 23, 2023
IgnatBeresnev
added a commit
that referenced
this issue
Nov 10, 2023
* Refactor and simplify artifact publishing * Update Gradle to 8.4 * Refactor and simplify convention plugins and build scripts Fixes #3132 --------- Co-authored-by: Adam <897017+aSemy@users.noreply.github.com> Co-authored-by: Oleg Yukhnevich <whyoleg@gmail.com>
github-merge-queue bot
pushed a commit
to slackhq/slack-gradle-plugin
that referenced
this issue
Mar 5, 2024
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [org.jetbrains.dokka](https://togithub.com/Kotlin/dokka) | plugin | patch | `1.9.10` -> `1.9.20` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>Kotlin/dokka (org.jetbrains.dokka)</summary> ### [`v1.9.20`](https://togithub.com/Kotlin/dokka/releases/tag/v1.9.20): 1.9.20 #### General bugfixes - Fixed sealed interfaces not having the `sealed` keyword in signatures ([Kotlin/dokka#2994) - Fixed incorrect links in multi-module projects with non-unique package names ([Kotlin/dokka#2272). Huge thanks to [@​EddieRingle](https://togithub.com/EddieRingle)! - Fixed member extensions not being shown on index pages in certain scenarios ([Kotlin/dokka#3187) - Fixed Java's inner classes not having the `inner` keyword in Kotlin signatures ([Kotlin/dokka#2793) - Fixed Java's `@param` tag not working with type parameters ([Kotlin/dokka#3199) - Fixed Dokka failing in KMP projects when the JVM source set is suppressed ([Kotlin/dokka#3209) #### HTML format - Provide an ability to add a custom homepage link to the header, more details in [Kotlin/dokka#2948 (comment) - Fixed tab selection resetting after navigating to a different page ([Kotlin/dokka#2899) - Fixed inline code not always being aligned with the surrounding text ([Kotlin/dokka#3228) - Fixed the "No options found" text in search being barely visible ([Kotlin/dokka#3281) - Fixed empty HTML tags being rendered for no reason ([Kotlin/dokka#3343, [Kotlin/dokka#3095) #### Runners ##### Gradle Plugin - Mark tasks as not compatible with Gradle configuration cache, second try ([Kotlin/dokka#3438). Thanks to [@​3flex](https://togithub.com/3flex) for noticing and fixing the problem! ##### Maven Plugin - Fixed `dokka:help` being absent ([Kotlin/dokka#3035). Thanks to [@​aSemy](https://togithub.com/aSemy)! - Fixed the source links configuration not working ([Kotlin/dokka#3046). Thanks to [@​freya022](https://togithub.com/freya022) for fixing this one! ##### CLI runner - Allow using relative paths in the `sourceRoots` configuration option ([Kotlin/dokka#2571) #### Plugin API - Provide an extension point to customize the rendering of code blocks in HTML format ([Kotlin/dokka#3244) #### Other: - Make sure `wasm-js` and `wasm-wasi` targets introduced in Kotlin 1.9.20 are supported ([Kotlin/dokka#3310) - Avoid concurrent invocations of Kotlin compiler's API due to the compiler API itself not always being thread safe ([Kotlin/dokka#3151). No noticeable performance loss is expected. - Bump dependencies to the latest versions ([Kotlin/dokka#3231, [Kotlin/dokka#3206, [Kotlin/dokka#3204) - Fix a documentation link ([Kotlin/dokka#3213). Thanks to [@​SubhrajyotiSen](https://togithub.com/SubhrajyotiSen) for noticing and fixing it! - Various build and project structure improvements ([Kotlin/dokka#3174, [Kotlin/dokka#3132). Enormous thanks to [@​aSemy](https://togithub.com/aSemy) for the help! See [Dokka 1.9.20](https://togithub.com/Kotlin/dokka/milestone/30?closed=1) milestone for the list of all changes. </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yMjcuMSIsInVwZGF0ZWRJblZlciI6IjM3LjIyNy4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
github-merge-queue bot
pushed a commit
to slackhq/circuit
that referenced
this issue
Mar 5, 2024
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [org.jetbrains.dokka](https://togithub.com/Kotlin/dokka) | plugin | patch | `1.9.10` -> `1.9.20` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>Kotlin/dokka (org.jetbrains.dokka)</summary> ### [`v1.9.20`](https://togithub.com/Kotlin/dokka/releases/tag/v1.9.20): 1.9.20 #### General bugfixes - Fixed sealed interfaces not having the `sealed` keyword in signatures ([Kotlin/dokka#2994) - Fixed incorrect links in multi-module projects with non-unique package names ([Kotlin/dokka#2272). Huge thanks to [@​EddieRingle](https://togithub.com/EddieRingle)! - Fixed member extensions not being shown on index pages in certain scenarios ([Kotlin/dokka#3187) - Fixed Java's inner classes not having the `inner` keyword in Kotlin signatures ([Kotlin/dokka#2793) - Fixed Java's `@param` tag not working with type parameters ([Kotlin/dokka#3199) - Fixed Dokka failing in KMP projects when the JVM source set is suppressed ([Kotlin/dokka#3209) #### HTML format - Provide an ability to add a custom homepage link to the header, more details in [Kotlin/dokka#2948 (comment) - Fixed tab selection resetting after navigating to a different page ([Kotlin/dokka#2899) - Fixed inline code not always being aligned with the surrounding text ([Kotlin/dokka#3228) - Fixed the "No options found" text in search being barely visible ([Kotlin/dokka#3281) - Fixed empty HTML tags being rendered for no reason ([Kotlin/dokka#3343, [Kotlin/dokka#3095) #### Runners ##### Gradle Plugin - Mark tasks as not compatible with Gradle configuration cache, second try ([Kotlin/dokka#3438). Thanks to [@​3flex](https://togithub.com/3flex) for noticing and fixing the problem! ##### Maven Plugin - Fixed `dokka:help` being absent ([Kotlin/dokka#3035). Thanks to [@​aSemy](https://togithub.com/aSemy)! - Fixed the source links configuration not working ([Kotlin/dokka#3046). Thanks to [@​freya022](https://togithub.com/freya022) for fixing this one! ##### CLI runner - Allow using relative paths in the `sourceRoots` configuration option ([Kotlin/dokka#2571) #### Plugin API - Provide an extension point to customize the rendering of code blocks in HTML format ([Kotlin/dokka#3244) #### Other: - Make sure `wasm-js` and `wasm-wasi` targets introduced in Kotlin 1.9.20 are supported ([Kotlin/dokka#3310) - Avoid concurrent invocations of Kotlin compiler's API due to the compiler API itself not always being thread safe ([Kotlin/dokka#3151). No noticeable performance loss is expected. - Bump dependencies to the latest versions ([Kotlin/dokka#3231, [Kotlin/dokka#3206, [Kotlin/dokka#3204) - Fix a documentation link ([Kotlin/dokka#3213). Thanks to [@​SubhrajyotiSen](https://togithub.com/SubhrajyotiSen) for noticing and fixing it! - Various build and project structure improvements ([Kotlin/dokka#3174, [Kotlin/dokka#3132). Enormous thanks to [@​aSemy](https://togithub.com/aSemy) for the help! See [Dokka 1.9.20](https://togithub.com/Kotlin/dokka/milestone/30?closed=1) milestone for the list of all changes. </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yMjcuMSIsInVwZGF0ZWRJblZlciI6IjM3LjIyNy4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
infrastructure
Everything related to builds tools, CI configurations and project tooling
tech-debt
A technical issue that is not observable by the users, but improves maintainers quality of life
Background
The current solution for publishing artifacts is quite restrictive (adding a different repository requires hardcoding it), and the TeamCity configuration we use is neither convenient nor safe to use (it's a single build configuration with checkboxes and input fields that does everything), especially if used by people outside of the Dokka team.
There's an ongoing effort by @aSemy to refactor publishing in #3030, but it tries to keep compatibility with the current solution and TC build configuration, which can get in the way of some improvements. Now might be a good time to re-evaluate our approach to publishing, and to refactor not only the build scripts, but all of publishing in general (including TC configurations), which will allow us to implement long-awaited improvements.
Publishing requirements
There are 3 types of repositories Dokka needs to publish to:
Gradle plugin portal
No specific requirements or complaints about the current solution, so this bit can most likely be reused.
Would be nice to have:
Maven Central
Currently, publishing to Maven Central happens in a single run, even though it really consists of 3 steps:
It might be convenient in some cases, but I'd would like to separate these two steps. So publishing to Maven Central should initialize the staging repository and upload the artifacts there, and that's it. Then we can test the artifacts manually, make sure everything is correct, and close the staging repository ourselves. Otherwise (and how it works now), if something happens during publishing and one of the modules is built incorrectly, we can't do anything about it, because it's all one giant step.
Additionally, for some reason, the staging repository is initialized even if Dokka is being published to Space only - this is a bug (most likely in configuration), it shouldn't happen.
If publishing to staging and closing it are two different actions, we don't need to support the publishing of
-SNAPSHOT
versions to Maven Central (DokkaPublicationChannel.MAVEN_CENTRAL_SNAPSHOT
), as we only use it before the release itself, mostly as a sanity check.CI-wise, I will likely create a separate build configuration with restricted access, so there's no need to put it under the same task like now.
Space
Dokka currently has a single Space repository for
-dev
artifacts (maven.pkg.jetbrains.space/kotlin/p/dokka/dev) that we publish to manually. The artifacts include both custom branches that are behindmaster
, and latestmaster
builds, and it is confusing that1.9.0-dev-130
might actually be behind1.9.0-dev-120
. For downstream users (Google, Stdlib), it is virtually impossible to guess which versions to use if they want latest changes.For this reason, I'd like to create a second space repository. One will be used for
-dev
artifacts and it will contain incremental builds of themaster
branch (likely once a week or upon request), so that the latest-dev
artifact is always of the latestmaster
commits. The other will be used fortest
artifacts, which will be very short-lived and could be built from custom branches.This means that Dokka must have the ability to publish to two different Space repositories (the dev one is hardcoded now, should it be an environment/build property?).
Would be nice to have:
-dev
artifacts are used by Google, and they've asked us to start signing them. It'd be great if signing could be added as part of this refactoring.Backward compatibility
The only expectation is: from the outside, the "before" and "after" artifacts should look the same, meaning they must have the same
.pom
,.module
and other contents. The rest can be changed as much as required :)DokkaPublicationChannel
abstraction can be dropped.The text was updated successfully, but these errors were encountered: