Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop integration tests that compare the output of different Dokka versions #3142

Open
IgnatBeresnev opened this issue Aug 24, 2023 · 0 comments
Labels
tech-debt A technical issue that is not observable by the users, but improves maintainers quality of life topic: K2 Issues / PRs that are related to the K2 migration. See #2888

Comments

@IgnatBeresnev
Copy link
Member

IgnatBeresnev commented Aug 24, 2023

Background

Dokka has good unit test coverage (> 80%) and OK-ish integration tests, but sometimes dependency updates or internal refactorings can introduce bugs or barely noticeable changes in behaviour or in the output. Most of the time, it's insignificant and not even bugs, but it would not hurt to know about it in advance.

Proposal

To make sure such subtle changes do not go unnoticed, it should be possible to compare the output of the "old" version of Dokka (before the update/refactoring) and the "new" (after). If there's no regression, the diffs should be the same. If even the smallest detail changes, the diff should show it.

This will help immensely with testing Dokka's migration to the K2 compiler (#2888) and with testing the new Gradle runner (#3131).

What exactly should be compared (the HTML outputs or the serialized documentable model), and how, is to be discussed.

Notes

There are projects that already have similar tests and that we can learn from:

  • Dackka (Google's Dokka plugin for Android docs) compares the diffs of their own HTML format.
  • Dokkatoo compares the HTML output that Dokka's native Gradle plugin produces with its own, but only for example projects. Ideally, it would be nice to compare big user projects used in integration tests.

Related issues: #2925

@IgnatBeresnev IgnatBeresnev added the tech-debt A technical issue that is not observable by the users, but improves maintainers quality of life label Aug 24, 2023
@IgnatBeresnev IgnatBeresnev added runner: Gradle plugin An issue/PR related to Dokka's Gradle plugin topic: K2 Issues / PRs that are related to the K2 migration. See #2888 and removed runner: Gradle plugin An issue/PR related to Dokka's Gradle plugin labels Oct 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tech-debt A technical issue that is not observable by the users, but improves maintainers quality of life topic: K2 Issues / PRs that are related to the K2 migration. See #2888
Projects
None yet
Development

No branches or pull requests

1 participant