proposal: embed: remove import restriction

[rsc]

@mdempsky points out in #41191 (comment) that the late addition of string and []byte to the embedding proposal has some unfortunate side effects that we might want to make sure we are happy with.

There are three details of string and []byte embedding that are at least surprising. This issue is about two of them. See #43216 for the third. The two for this issue are as follows.

First, to make sure source files containing embedding directives are clearly identified (by an import), we require an import _ "embed" to write

//go:embed hello.txt
var msg string

even though there is no use of the embed package. This requires a special case in goimports. (This is noted in the proposal as accepted.)

Second, the late addition says “a plain string or []byte variable”, and I really did mean those words. Only those specific types are allowed, not []uint8 and not other user-defined aliases for those types.

I see two options here. One is to do nothing and leave things as they are. That's possible and is the "this issue gets declined" outcome. The other option I can see is to introduce new type aliases embed.String and embed.Bytes and require those to be used as the types, which is what this proposal issue is meant to discuss and decide whether to accept.

Specifically:

package embed

// A String is a string that can be initialized using a //go:embed directive.
// It is an alias so that embed.String can be used as a string without
// any explicit conversion. The only expected use of this type is in
// var declarations accompanied by //go:embed directives. Elsewhere,
// code should simply use string.
type String = string

// A Bytes is a []byte that can be initialized using a //go:embed directive.
// It is an alias so that embed.Bytes can be used as a []byte without
// any explicit conversion. The only expected use of this type is in
// var declarations accompanied by //go:embed directives. Elsewhere,
// code should simply use []byte.
type Bytes = []byte

Then instead of writing:

//go:embed hello.txt
var msg string

You have to write:

//go:embed hello.txt
var msg embed.String

That solves both of the issues: the import is now required without any special case in goimports, and there is no question about whether []uint8 or other user-defined alias is OK. Only embed.String and embed.Bytes are OK.

What do people think? (Thumbs up / thumbs down is fine.)

[One option that's not on the table is removing string and []byte embedding entirely. Many people asked for that during the pre-proposal discussions, and I think it is important to respect that process. We can fix problems, but I think it would be improper to drop the functionality completely.]

Thank you!

[SamWhited]

Using a magic import import _ "embed" just feels confusing and makes me wonder what magic the Go compiler is hiding behind the scenes. I believe this sort of import was a mistake in the original design of the language and that its use in database/sql and the image libraries and the like was a mistake that we shouldn't add more of. I have run into any number of unexpected panics and other confusion due to import side effects. This specific magic import may not have any side effects that could cause problems, but I can't know that from looking at it. Please don't hide behavior behind magic imports. I'd almost rather the compiler implicitly magically import embed without that line being present just so that I don't have to see it since this would just be part of the go:embed directive behavior which is already magic that I sort of hate anyways but, per your email saying this wasn't going to change, we have to live with already.

I'm more okay with embed.{Bytes,String} because it at least gives me some reason to be importing embed without magic. Without knowing anything about the internals of the package (which will be the case for most Go users) I still immediately think "this is weird, it's just an alias, so why can't I just use a string?". This isn't the end of the world, Go has plenty of other more glaring quirks, but it's where my mind goes.

[hherman1]

Personally I like that its just a plain string and a []byte. The type wrappers aren't the end of the world, but I don't find the "_" import offensive enough to justify the expanded api personally, esp. considering there aren't actually any side effects of this import.

[earthboundkid]

FWIW, I started writing a blog post about //go:embed and had a little section about remembering to use import _ "embed" and example showing how it wouldn't work if you left it out. This simplifies the blog post because now I don't have to explain about a possible pitfall. I think being able to simplifying teaching materials is a good sign.

[hherman1]

I’m sure there’s already a great explanation of why this is a bad idea that I missed, but what if you just had no import at all, and //go:embed worked on its own? I take it that’s off the table?

[mdempsky]

I’m sure there’s already a great explanation of why this is a bad idea that I missed, but what if you just had no import at all, and //go:embed worked on its own? I take it that’s off the table?

That would slow down go/build (e.g., cmd/go). Currently to compute package dependencies, it only parses Go source files up through the import declarations.

With //go:embed directives, it's necessary for go/build to parse these too, to figure out which files are embedded. By requiring that package embed is imported to use them, go/build can continue parsing only the import declarations for most files; and only continue to parse the rest of the file when it saw an import for package embed.

[marwan-at-work]

One potential expansion on this proposal:

The declaration of a var <name> embed.{FS,String,Bytes} can be a hint to the Go compiler to validate the comment directive that are defined on top of the declaration.

For example,

//go;embed: file.txt 
var file embed.String

would cause a compiler error which will alleviate programmer confusion that they misspelled go:embed .

Similarly, the lack of a go directive comment could also be a compiler error, because why would a programmer declare an embed.{Type} without actually needing to embed a file?

This would address both @mdempsky and @rsc 's comments here by keeping the embedding syntax as is while providing better developer experience.

Hopefully my skimming of the comments didn't miss a similar suggestion.

[mdempsky]

Similarly, the lack of a go directive comment could also be a compiler error, because why would a programmer declare an embed.{Type} without actually needing to embed a file?

The draft proposal specifically states: "It is not an error to declare an embed.Files without a //go:embed directive. That variable simply contains no embedded files."

Note also that a single variable declaration can have multiple //go:embed directives. (Unlike normal Go function call syntax, which allow spreading across multiple lines, //go: directives are line oriented.) Reporting an error about no //go:embed directives wouldn't catch the case where the user typed some //go:embed directives correctly, but not all.

(That said, because all of the //go:embed lines would be clustered together, it would probably be rather visually obvious if there were any mistakes on some lines but not all.)

[hherman1]

I've been playing with go:embed on the beta a bit, and I noticed something I'd like to add to the mix. If I start working on a file with an embedded variable, and then I decide I want to move this embedded variable to a different file or module, its very easy to forget to remove the _ "embed" import, and because the import is explicitly _ my IDE/go itself don't complain about its presence or auto-remove it.

For that reason, I think it would be valuable at the very least for go to complain about unused _ "embed" imports, but I also think this is a point for the embed.Bytes proposal.

I really like the idea of embedding simple []byte/string. I don't like needing to learn about new types or the obscurity that comes with it, although slight. That being said, I think the case for these special types is stronger than I initially gave it credit.

[hherman1]

While I'm here, I'll add on that I found it surprising that ../../resources/file.zip was an invalid pattern. This is clearly not the right place for this comment, but I'm not aware of another place to put embed feedback.