Issue using rules_kotlin as local_repository

[jeffzoch]

Been a few months since i used the rules locally but im running into two strange errors:

1. Unable to find package for @rkt_1_4//jvm:opts.bzl: The repository '@rkt_1_4' could not be resolved
2. If i specify a kotlin compiler release I get an error Error: 'dict' value has no field or method 'url_templates'

here is my workspace:

local_repository(
    name = "io_bazel_rules_kotlin",
    path = "../rules_kotlin",
)

load("@io_bazel_rules_kotlin//kotlin:dependencies.bzl", "kt_download_local_dev_dependencies")
kt_download_local_dev_dependencies()
# KOTLIN
load("@io_bazel_rules_kotlin//kotlin:repositories.bzl", "kotlin_repositories")

KOTLINC_RELEASE = {
    "version" : "1.5.20",
    "url_templates" : [
        "https://github.com/JetBrains/kotlin/releases/download/v{version}/kotlin-compiler-{version}.zip",
    ],
    "sha256" : "edf34263ddaabd48f7ec59661e4c0d1dc868462fd3a1ea323083d0e3e83a8a8b",
}

kotlin_repositories(compiler_release = KOTLINC_RELEASE)

register_toolchains("//tools/kotlin:kotlin_toolchain")

and here is my toolchain:

load(
    "@io_bazel_rules_kotlin//kotlin:core.bzl",
    "define_kt_toolchain",
    "kt_compiler_plugin",
    "kt_kotlinc_options",
)
load("@io_bazel_rules_kotlin//kotlin:jvm.bzl", "kt_jvm_library", "kt_javac_options")

KOTLIN_LANGUAGE_LEVEL = "1.4"

kt_kotlinc_options(
    name = "kotlinc_options",
    x_allow_result_return_type = True,
    x_inline_classes = True,
)

kt_javac_options(
    name = "default_javac_options",
)

define_kt_toolchain(
    name = "kotlin_toolchain",
    api_version = KOTLIN_LANGUAGE_LEVEL,  # "1.1", "1.2", or "1.3"
    javac_options = "//:default_javac_options",
    jvm_target = "1.8",  # "1.6", "1.8", "9", "10", "11", or "12",
    kotlinc_options = "//:kotlinc_options",
    language_version = KOTLIN_LANGUAGE_LEVEL,  # "1.1", "1.2", or "1.3"
    experimental_strict_kotlin_deps = "warn"
)

I am on the latest commit a73e838
Both seem related to the large versioning change added by @restingbull

[jeffzoch]

the 2nd issue seems to be due to the "version" provider not being exported.

[restingbull]

Apologies, did a bit of moving the deck chairs.

#571 should fix it, though the requirements for working from head have changed. This was due to --override_repository not completely removing the local_repository files.

[jeffzoch]

@restingbull i just tried out the latest changes with #547 and am still getting strange errors like:

ERROR: Error fetching repository: java.io.IOException: No WORKSPACE file found in /private/var/tmp/_bazel_j.zoch/8bdf7aad5539a314ee98a49e42674806/external/io_bazel_rules_kotlin_head
ERROR: error loading package '': Encountered error while reading extension file 'kotlin/repositories.bzl': no such package '@io_bazel_rules_kotlin//kotlin': no such package '@io_bazel_rules_kotlin_head//': No WORKSPACE file found in /private/var/tmp/_bazel_j.zoch/8bdf7aad5539a314ee98a49e42674806/external/io_bazel_rules_kotlin_head

despite never defining io_bazel_rules_kotlin_head anywhere.
Edit: I see that the _head is defined in release_archive but am not sure why the workspace cannot be found.

I followed the latest instructions on the readme and ran bazel build //:rules_kotlin_release

local_repository(
    name = "release_archive",
    path = "../rules_kotlin/src/main/starlark/release_archive",
)

load("@release_archive//:repository.bzl", "archive_repository")
archive_repository(
    name = "io_bazel_rules_kotlin",
)
...
load("@io_bazel_rules_kotlin//kotlin:repositories.bzl", "kotlin_repositories", "kotlinc_version")

kotlin_repositories(
    compiler_release = kotlinc_version(
        release = "1.5.20", # just the numeric version
        sha256 = "edf34263ddaabd48f7ec59661e4c0d1dc868462fd3a1ea323083d0e3e83a8a8b"
    )
)
register_toolchains("//tools/kotlin:kotlin_toolchain")

edit: tried not specifying custom toolchain or compiler distribution and same error occurs.

[Nikolas-LFDesigns]

Uh, it does really take some cunning but still the sequence i did doesn't seem to even try to execute a repository rule needed to build the rules.
Though my attempts were about trying to download an archive with http_archive (you know, sandboxing and stuff) like this:

http_archive(
    name = "io_bazel_rules_kotlin",
    strip_prefix = "rules_kotlin-fa0ff96cc78d3cdc4420fe8251c5b13b4d6a4923",
    type = "zip",
    urls = [
        "https://github.com/bazelbuild/rules_kotlin/archive/fa0ff96cc78d3cdc4420fe8251c5b13b4d6a4923.zip",
    ],
)

Then define a release archive with bzl:

load("@io_bazel_rules_kotlin//src/main/starlark/release_archive:repository.bzl", "archive_repository")

def kotlin_archive_repository(name):
    archive_repository(
        name = name,
        release_archive_target = Label("@io_bazel_rules_kotlin//:rules_kotlin_release"), # that bit is confusing, but is a key to override default `//:rules_kotlin_release` leading to my repo root
    )

and in WORKSPACE:

kotlin_archive_repository(name = "rules_kotlin"):

Leads to no error except banal "couldn't fetch, uh, IOException, go away" stuff:

ERROR: Error fetching repository: java.io.IOException: No WORKSPACE file found in [...]/external/rules_kotlin_head
ERROR: error loading package '': Encountered error while reading extension file 'kotlin/core.bzl': no such package '@rules_kotlin//kotlin': no such package '@rules_kotlin_head//': No WORKSPACE file found in [...]/external/rules_kotlin_head

Maybe I'm just missing something obvious?
My remaining toolchain configuration is pure like it was on old library versions:

load("@rules_kotlin//kotlin:core.bzl", "kt_register_toolchains")

kt_register_toolchains()

load("@rules_kotlin//kotlin:repositories.bzl", "kotlin_repositories")

kotlin_repositories()

(Yes, I removed "kt_download_local_dependencies" and built its dependencies into repository and it worked until now)

[restingbull]

I believe I made a bit of a mistake in setting the local-from-head -- Since the archive_repostory operates in a very "probably shouldn't be doing that space" Bazel doesn't know to update the external/<from head> repo.

I'm testing a fix that should handle that (track the local_repository as input to archive_repository).

Means while, try deleting (yes, I just recommended modifying files in bazel-. I'm going to a very warm place for saying that) the external/<from head> and external/< release_archive> directories.

[restingbull]

Try pulling #577 to see if it works.

[jeffzoch]

@restingbull with no modifications (ie deleting stuff from bazel-*) #577 is reporting the same error

ERROR: Error fetching repository: java.io.IOException: No WORKSPACE file found in /private/var/tmp/_bazel_j.zoch/8bdf7aad5539a314ee98a49e42674806/external/io_bazel_rules_kotlin_head
ERROR: error loading package '': Encountered error while reading extension file 'kotlin/repositories.bzl': no such package '@io_bazel_rules_kotlin//kotlin': no such package '@io_bazel_rules_kotlin_head//': No WORKSPACE file found in /private/var/tmp/_bazel_j.zoch/8bdf7aad5539a314ee98a49e42674806/external/io_bazel_rules_kotlin_head

I tried deleting bazel-bin/external/**/* and that didnt help (there was a few different dirs starting with "release_archive" but none of those worked either) @restingbull

[Nikolas-LFDesigns]

Any updates here?
Still not building as of d174c98

[oliviernotteghem]

Getting exact same error too, even after bazel clean --expunge

ERROR: error loading package '': Encountered error while reading extension file 'kotlin/repositories.bzl': no such package '@io_bazel_rules_kotlin//kotlin': no such package '@io_bazel_rules_kotlin_head//': No WORKSPACE file found in /private/var/tmp/_bazel_oliviern/91a99194e71852b0e089ca5cc1c48073/external/io_bazel_rules_kotlin_head

[restingbull]

I'm an idiot, and I apologize for this one. I've been snowed under a new annual planning process, so the rules have suffered.

I'll make it cleaner momentarily, but here is the workaround:

local_repository(
    name = "release_archive",
    path = "../rules_kotlin/src/main/starlark/release_archive",
)

load("@release_archive//:repository.bzl", "archive_repository")
archive_repository(
    name = "io_bazel_rules_kotlin",
    local_path = "../rules_kotlin" # needs to know where the head repo is. Should infer it from the release_archive repo.
)

[oliviernotteghem]

thanks @restingbull for fixing! Did the trick.

[Nikolas-LFDesigns]

@restingbull ok, that seem to work.. any way to force these building with http_archive instead of local_repository as a source? We are using latest stuff from the rules and hence have to build from source, we are using sanboxing as well and very prefer to not include some stray repository as a local_repository.

[restingbull]

@Nikolas-LFDesigns -- the rules really aren't prod quality at head. Being able to build them that way is intended for active development on the rules... Which isn't particularly hermetic.

That said, what is the usecase ya'll need that prevents working off the latest release?

[Nikolas-LFDesigns]

There were many updates that were not released, like plugin support, important bug fixes, kt_android_local_test, worker support and so on which we constantly use and have to be sure that support is continuous, nevermind the current release.

[restingbull]

Well, the simplest solution, while I try to figure out how to balance the release archive (necessary for multiversion builds) with the correct repository layout (the reason the label is weird -- archive is meant to be an independent repo), would be to do this

http_archive(
    name = "io_bazel_rules_kotlin",
    strip_prefix = "rules_kotlin-fa0ff96cc78d3cdc4420fe8251c5b13b4d6a4923",
    type = "zip",
    urls = [
        "https://github.com/bazelbuild/rules_kotlin/archive/fa0ff96cc78d3cdc4420fe8251c5b13b4d6a4923.zip",
    ],
)

load("@io_bazel_rules_kotlin//src/main/starlark/release_archive:repository.bzl", "archive_repository")

http_archive(
    name = "rules_kotlin_head",
    strip_prefix = "rules_kotlin-fa0ff96cc78d3cdc4420fe8251c5b13b4d6a4923",
    type = "zip",
    urls = [
        "https://github.com/bazelbuild/rules_kotlin/archive/fa0ff96cc78d3cdc4420fe8251c5b13b4d6a4923.zip",
    ],
)

archive_repository(
        name = "rules_kotlin",
        release_archive_target = Label("@io_bazel_rules_kotlin//:rules_kotlin_release"), # that bit is confusing, but is a key to override default `//:rules_kotlin_release` leading to my repo root
 )

[Nikolas-LFDesigns]

@restingbull
Thanks for update, it got me a little further:

ERROR: Skipping '//:rules_kotlin_release': error loading package '': in [bazel path]/external/rules_kotlin_head/src/main/starlark/release/packager.bzl: Unable to find package for @rules_pkg//:pkg.bzl: The repository '@rules_pkg' could not be resolved.
WARNING: Target pattern parsing failed.
ERROR: error loading package '': in [bazel path]/external/rules_kotlin_head/src/main/starlark/release/packager.bzl: Unable to find package for @rules_pkg//:pkg.bzl: The repository '@rules_pkg' could not be resolved.

Sounds like rules_pkg is part of the Bazel and should be provided to any workspace, but it's not. Adding it to my WORKSPACE explicitly doesn't work either (possibly because it should be present on archive WORKSPACE).
Also, bootstrapping like that is not gonna work without Bazel binary in the PATH cause it is explicitly called in an archive repository.

Sounds like we'll have to fork and custom-release a project in the event of some useful change being added to the master on current assembly system..

[jeffzoch]

Yeah I have resorted to just uploading release tars to my fork instead of trying to consume the rules locally. It makes the test-debug lifecycle significantly longer but it may take time to sort this out

[cgruber]

I also have it on my list to arrange a nightly (if different) release. Just have to figure out how to make that work with buildkite. That should at least shorten the cycle.

[jeffzoch]

@cgruber not sure if youve considered a release per-commit? Might be easier to configure if thats the issue with buildkite. Kotlin seems to do something similar based on the number of releases in their repo

[cgruber]

@cgruber not sure if youve considered a release per-commit? Might be easier to configure if thats the issue with buildkite. Kotlin seems to do something similar based on the number of releases in their repo

Per-commit or per-nightly - regardless I'll need to get a pipeline executing. The main issue is that we're using bazelbuild's custom org config of buildKite, and I'm not authorized to admin that, so it's a bit trickier. Nevertheless, let's shelve this because it's orthogonal to this issue.

[restingbull]

Finally got a minute and sorted out the build from master -- #612

Of special note: http_archive automatically replaces the WORKSPACE file with workspace(name = "*repo name*")

Which is why building from head wouldn't work when downloaded.

This is the way to make it work after #612 is merged.

# Download master or specific revisions
http_archive(
    name = "io_bazel_rules_kotlin_master",
    urls = [ "https://github.com/bazelbuild/rules_kotlin/archive/refs/heads/master.zip", ],
    strip_prefix = "rules_kotlin-master",
)

load("@io_bazel_rules_kotlin_master//src/main/starklark/release_archive", "archive_repository")

archive_repository(
    name = "io_bazel_rules_kotlin",
    source_repository_name = "io_bazel_rules_kotlin_master"
)

load("@io_bazel_rules_kotlin//kotlin:repositories.bzl", "kotlin_repositories", "versions")

kotlin_repositories()

load("@io_bazel_rules_kotlin//kotlin:core.bzl", "kt_register_toolchains")

kt_register_toolchains()

[mgenov]

Tried the example above but I'm encountering the following error:

WORKSPACE:104:6: in load statement: The label must reference a file with extension '.bzl'

Any idea what could be the cause?

[restingbull]

Typo, (sorry). Try:

load("@io_bazel_rules_kotlin_master//src/main/starklark/release_archive.bzl", "archive_repository")

[mgenov]

Got the following error:

ERROR: error loading package '': Label '@io_bazel_rules_kotlin_master//src/main/starklark/release_archive.bzl:release_archive.bzl' is invalid because 'src/main/starklark/release_archive.bzl'  is not a package; perhaps you meant to put the colon here: '@io_bazel_rules_kotlin_master//:src/main/starklark/release_archive.bzl/release_archive.bzl'?

As I can see, the src/main/starlark/release_archive contains a WORKSPACE file, so it doesn't look like it's a bzl file.

[eikemeier]

I don't know if this is helpful, but when I use

http_archive(
    name = "io_bazel_rules_kotlin_head",
    sha256 = "e18c230aa7f817850c11b3fcbfc9b688a891d40bd5cf2c976224832dea3e5eb4",
    strip_prefix = "rules_kotlin-1f61f73ac5fc78cb08807bf629e8626e64646abe",
    url = "https://github.com/bazelbuild/rules_kotlin/archive/1f61f73ac5fc78cb08807bf629e8626e64646abe.tar.gz",
)

load("@io_bazel_rules_kotlin_head//src/main/starlark/release_archive:repository.bzl", "archive_repository")

archive_repository(
    name = "io_bazel_rules_kotlin",
    source_repository_name = "io_bazel_rules_kotlin_head"
)

I get

ERROR: Analysis of target '//:rules_kotlin_release' failed; build aborted: Either the path attribute of android_ndk_repository or the ANDROID_NDK_HOME environment variable must be set.

Bazel 5.0rc2, macOS 10.15, no Android SDK installed.

[eikemeier]

Works when using Bazel 4.2.1. So use

load("@io_bazel_rules_kotlin_head//src/main/starlark/release_archive:repository.bzl", "archive_repository")

[mgenov]

Seems something on my side or something related to bazel version check:

bazel --version

bazel 4.2.1-homebrew

bazel build //kotlin/common/cli:lib shows the following error:

ERROR: The project you're trying to build requires Bazel 4.2.0 (specified in /private/var/tmp/_bazel_mgenov/8cac0968a0c17cc631135ceb621314f4/external/io_bazel_rules_kotlin_head/.bazelversion), but it wasn't found in /usr/local/Cellar/bazel/4.2.1_1/libexec/bin.

Here is information regarding version of bazel and it's location:

 ls -alh ls /usr/local/Cellar/bazel/4.2.1_1/bin
ls: ls: No such file or directory
/usr/local/Cellar/bazel/4.2.1_1/bin:
total 16
drwxr-xr-x  4 mgenov  staff   128B Nov 21 12:32 .
drwxr-xr-x  8 mgenov  staff   256B Nov 21 12:33 ..
-r-xr-xr-x  1 mgenov  staff   147B Nov 21 12:32 bazel
-r-xr-xr-x  1 mgenov  staff   153B Nov 21 12:32 bazel-4.2.1

bazel --version from /usr/localCellar/bazel/

/usr/local/Cellar/bazel/4.2.1_1/bin/bazel --version
bazel 4.2.1-homebrew

[eikemeier]

ERROR: The project you're trying to build requires Bazel 4.2.0 (specified in /private/var/tmp/_bazel_mgenov/8cac0968a0c17cc631135ceb621314f4/external/io_bazel_rules_kotlin_head/.bazelversion), but it wasn't found in /usr/local/Cellar/bazel/4.2.1_1/libexec/bin.

I would recommend to deinstall bazel und install bazelisk, it makes versioning much easier.

[mgenov]

It works with bazelisk as expected. Thanks for your help @eikemeier