Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It is now possible to create globals in a different module #54607

Closed
LilithHafner opened this issue May 28, 2024 · 12 comments · Fixed by #54678
Closed

It is now possible to create globals in a different module #54607

LilithHafner opened this issue May 28, 2024 · 12 comments · Fixed by #54678
Labels
kind:minor change Marginal behavior change acceptable for a minor release

Comments

@LilithHafner
Copy link
Member

x@fedora:~$ julia +1.8 -q
julia> module M; end
Main.M

julia> M.x = 6
ERROR: cannot assign variables in other modules
Stacktrace:
 [1] setproperty!(x::Module, f::Symbol, v::Int64)
   @ Base ./Base.jl:32
 [2] top-level scope
   @ REPL[2]:1

julia> 
x@fedora:~$ julia +1.9 -q
julia> module M; end
Main.M

julia> M.x = 6
6

From the discussion at #44231, this seems unintentional.

@cstjean
Copy link
Contributor

cstjean commented May 29, 2024

Personally, I love the change. Especially now that globals can be typed properly, being able to just MyPackage.some_global_setting = true is just a very nice and convenient interface for packages.

The former behaviour had that "You are a bad person for using globals" vibe that just felt out of sync with the general permissiveness of a language that allows me to redefine addition if I feel like it.

@vtjnash
Copy link
Sponsor Member

vtjnash commented May 29, 2024

This is a little different, since it not only sets the global, it also creates it if it didn't yet exist and assigns its type. I have entertained the idea of disallowing setting globals that don't exist (it is a 2 line patch roughly), but it breaks a few minor things (Serialization specifically) so it isn't entirely obvious that is beneficial just to help catch typos.

@KristofferC
Copy link
Sponsor Member

KristofferC commented May 29, 2024

From the discussion at #44231, this seems unintentional.

I'm reading through that PR but can't really find that discussion. Could you link to it explicitly?

Anyway, this seems fine to me, having to go via eval always felt silly to me.

@LilithHafner
Copy link
Member Author

I suspect that it happened in a triage discussion referenced by #44231 (comment)

@bvdmitri
Copy link
Contributor

@vtjnash could Serialization use a somewhat private API for it instead of relying on the = operator?

@vtjnash
Copy link
Sponsor Member

vtjnash commented May 29, 2024

Sure, it could use Core.eval(m, :(global $var)) before setglobal!(m, var, val). The patch looks something like this, though apparently some code has refactored so it takes a bit more than 2 lines right now to add it:

diff --git a/src/module.c b/src/module.c
index 7a12552415..470bf3df98 100644
--- a/src/module.c
+++ b/src/module.c
@@ -219,13 +219,16 @@ static void check_safe_newbinding(jl_module_t *m, jl_sym_t *var)
 static jl_module_t *jl_binding_dbgmodule(jl_binding_t *b, jl_module_t *m, jl_sym_t *var) JL_GLOBALLY_ROOTED;
 
 // get binding for assignment
-JL_DLLEXPORT jl_binding_t *jl_get_binding_wr(jl_module_t *m JL_PROPAGATES_ROOT, jl_sym_t *var)
+JL_DLLEXPORT jl_binding_t *jl_get_binding_wr(jl_module_t *m JL_PROPAGATES_ROOT, jl_sym_t *var, int alloc)
 {
     jl_binding_t *b = jl_get_module_binding(m, var, 1);
     jl_binding_t *b2 = jl_atomic_load_relaxed(&b->owner);
     if (b2 != b) {
-        if (b2 == NULL)
+        if (b2 == NULL) {
             check_safe_newbinding(m, var);
+            if (!alloc)
+                jl_errorf("Global %s.%s cannot be set since it does not exist.", jl_symbol_name(m->name), jl_symbol_name(var));
+        }
         if (b2 != NULL || (!jl_atomic_cmpswap(&b->owner, &b2, b) && b2 != b)) {
             jl_module_t *from = jl_binding_dbgmodule(b, m, var);
             if (from == m)

@KristofferC KristofferC added the kind:breaking This change will break code label May 30, 2024
@KristofferC
Copy link
Sponsor Member

I suspect that it happened in a triage discussion referenced by #44231 (comment)

But there is nothing in that comment saying that setting globals in different modules should not be allowed? In fact, it explicitly removes the test that checks that this is not allowed https://github.com/JuliaLang/julia/pull/44231/files#diff-829029ba7e34234a5360ccd30854edbcff7fb932c1d2f9a52f1027e881ae9a7a.

Removing this now would be breaking anyway so maybe just best to let this be.

@LilithHafner
Copy link
Member Author

Perhaps I misunderstood the "don't do setfield!" sentiment as "don't do property access syntax". In any event, this issue has accomplished its objective of double checking that we removed that intentional error intentionally.

@LilithHafner LilithHafner closed this as not planned Won't fix, can't repro, duplicate, stale May 30, 2024
@KristofferC
Copy link
Sponsor Member

KristofferC commented May 30, 2024

I think that the statement there is intended to be read that settings and reading globals from modules shouldn't piggy back on the setfield! / getfield intrinsics but have their own. Then setproperty!/getproperty just uses these new intrinsics.

@topolarity
Copy link
Member

This is a little different, since it not only sets the global, it also creates it if it didn't yet exist and assigns its type.

I think that means that #53750 is incomplete and also needs to consider that binding types can revert to undef, unless we prohibit this behavior

@Keno Keno reopened this Jun 4, 2024
@Keno
Copy link
Member

Keno commented Jun 4, 2024

Discussed in the context of the meeting we had for #54654 and decided that this is a serious enough oversight that we should attempt to correct this as quickly as possible, keeping in mind that it was in the system for two releases, so we need to at the very least pkgeval and may do a soft deprecation.

Keno added a commit to JuliaLang/Distributed.jl that referenced this issue Jun 5, 2024
As discussed in [1], the implicit creation of bindings through the
setglobal! intrinsic was accidentally added in 1.9 unintentionally
and will be removed (ideally) or at the very least deprecated in 1.11.

The recommended replacement syntax is `Core.eval(mod, Expr(:global, sym))` to
introduce the binding and `invokelatest(setglobal!, mod, sym, val)` to set
it. The invokelatest is not presently required, but may be required
for JuliaLang/julia#54654, so it's included
in the recommendation.

[1] JuliaLang/julia#54607
Keno added a commit that referenced this issue Jun 5, 2024
PR #44231 (part of Julia 1.9) introduced the ability to modify globals with `Mod.sym = val` syntax.
However, the intention of this syntax was always to modify *existing* globals in other modules.
Unfortunately, as implemented, it also implicitly creates new bindings in the other module, even
if the binding was not previously declared. This was not intended, but it's a bit of a syntax corner
case, so nobody caught it at the time.

After some extensive discussions and taking into account the near future direction we want to go
with bindings (#54654 for both), the consensus was reached that we should try to undo the implicit
creation of bindings (but not the ability to assign the *value* of globals in other modules).
Note that this was always an error until Julia 1.9, so hopefully it hasn't crept into too many
packages yet. We'll see what pkgeval says. If use is extensive, we may want to consider a softer
removal strategy.

Across base and stdlib, there's two cases affected by this change:
1. A left over debug statement in `precompile` that wanted to assign a new variable in Base for debugging. Removed in this PR.
2. Distributed wanting to create new bindings. This is a legimitate use case for wanting to
   create bindings in other modules. This is fixed in JuliaLang/Distributed.jl#102.

As noted in that PR, the recommended replacement where implicit binding creation is desired is:
```
Core.eval(mod, Expr(:global, sym))
invokelatest(setglobal!, mod, sym, val)
```

The `invokelatest` is not presently required, but may be needed by #54654, so it's included
in the recommendation now.

Fixes #54607
Keno added a commit to JuliaLang/Distributed.jl that referenced this issue Jun 5, 2024
As discussed in [1], the implicit creation of bindings through the
setglobal! intrinsic was accidentally added in 1.9 unintentionally
and will be removed (ideally) or at the very least deprecated in 1.11.

The recommended replacement syntax is `Core.eval(mod, Expr(:global, sym))` to
introduce the binding and `invokelatest(setglobal!, mod, sym, val)` to set
it. The invokelatest is not presently required, but may be required
for JuliaLang/julia#54654, so it's included
in the recommendation.

[1] JuliaLang/julia#54607
Keno added a commit that referenced this issue Jun 5, 2024
PR #44231 (part of Julia 1.9) introduced the ability to modify globals with `Mod.sym = val` syntax.
However, the intention of this syntax was always to modify *existing* globals in other modules.
Unfortunately, as implemented, it also implicitly creates new bindings in the other module, even
if the binding was not previously declared. This was not intended, but it's a bit of a syntax corner
case, so nobody caught it at the time.

After some extensive discussions and taking into account the near future direction we want to go
with bindings (#54654 for both), the consensus was reached that we should try to undo the implicit
creation of bindings (but not the ability to assign the *value* of globals in other modules).
Note that this was always an error until Julia 1.9, so hopefully it hasn't crept into too many
packages yet. We'll see what pkgeval says. If use is extensive, we may want to consider a softer
removal strategy.

Across base and stdlib, there's two cases affected by this change:
1. A left over debug statement in `precompile` that wanted to assign a new variable in Base for debugging. Removed in this PR.
2. Distributed wanting to create new bindings. This is a legimitate use case for wanting to
   create bindings in other modules. This is fixed in JuliaLang/Distributed.jl#102.

As noted in that PR, the recommended replacement where implicit binding creation is desired is:
```
Core.eval(mod, Expr(:global, sym))
invokelatest(setglobal!, mod, sym, val)
```

The `invokelatest` is not presently required, but may be needed by #54654, so it's included
in the recommendation now.

Fixes #54607
Keno added a commit that referenced this issue Jun 5, 2024
PR #44231 (part of Julia 1.9) introduced the ability to modify globals with `Mod.sym = val` syntax.
However, the intention of this syntax was always to modify *existing* globals in other modules.
Unfortunately, as implemented, it also implicitly creates new bindings in the other module, even
if the binding was not previously declared. This was not intended, but it's a bit of a syntax corner
case, so nobody caught it at the time.

After some extensive discussions and taking into account the near future direction we want to go
with bindings (#54654 for both), the consensus was reached that we should try to undo the implicit
creation of bindings (but not the ability to assign the *value* of globals in other modules).
Note that this was always an error until Julia 1.9, so hopefully it hasn't crept into too many
packages yet. We'll see what pkgeval says. If use is extensive, we may want to consider a softer
removal strategy.

Across base and stdlib, there's two cases affected by this change:
1. A left over debug statement in `precompile` that wanted to assign a new variable in Base for debugging. Removed in this PR.
2. Distributed wanting to create new bindings. This is a legimitate use case for wanting to
   create bindings in other modules. This is fixed in JuliaLang/Distributed.jl#102.

As noted in that PR, the recommended replacement where implicit binding creation is desired is:
```
Core.eval(mod, Expr(:global, sym))
invokelatest(setglobal!, mod, sym, val)
```

The `invokelatest` is not presently required, but may be needed by #54654, so it's included
in the recommendation now.

Fixes #54607
@LilithHafner LilithHafner added the status:triage This should be discussed on a triage call label Jun 6, 2024
@LilithHafner
Copy link
Member Author

Triage is are okay with keeping changing values of globals in other modules with Mod.x = 6 syntax and setglobal!.

We want to change setglobal! to no longer create new globals. Mod.x = 6 should keep calling setglobal!.
x = 6 within a module should be special in that it may create a new global (and increment world age)

@LilithHafner LilithHafner changed the title It is now possible to update globals in a different module It is now possible to create globals in a different module Jun 6, 2024
@LilithHafner LilithHafner added needs more info Clarification or a reproducible example is required kind:minor change Marginal behavior change acceptable for a minor release and removed kind:breaking This change will break code status:triage This should be discussed on a triage call labels Jun 6, 2024
@LilithHafner LilithHafner removed the needs more info Clarification or a reproducible example is required label Jun 6, 2024
Keno added a commit that referenced this issue Jun 8, 2024
PR #44231 (part of Julia 1.9) introduced the ability to modify globals with `Mod.sym = val` syntax.
However, the intention of this syntax was always to modify *existing* globals in other modules.
Unfortunately, as implemented, it also implicitly creates new bindings in the other module, even
if the binding was not previously declared. This was not intended, but it's a bit of a syntax corner
case, so nobody caught it at the time.

After some extensive discussions and taking into account the near future direction we want to go
with bindings (#54654 for both), the consensus was reached that we should try to undo the implicit
creation of bindings (but not the ability to assign the *value* of globals in other modules).
Note that this was always an error until Julia 1.9, so hopefully it hasn't crept into too many
packages yet. We'll see what pkgeval says. If use is extensive, we may want to consider a softer
removal strategy.

Across base and stdlib, there's two cases affected by this change:
1. A left over debug statement in `precompile` that wanted to assign a new variable in Base for debugging. Removed in this PR.
2. Distributed wanting to create new bindings. This is a legimitate use case for wanting to
   create bindings in other modules. This is fixed in JuliaLang/Distributed.jl#102.

As noted in that PR, the recommended replacement where implicit binding creation is desired is:
```
Core.eval(mod, Expr(:global, sym))
invokelatest(setglobal!, mod, sym, val)
```

The `invokelatest` is not presently required, but may be needed by #54654, so it's included
in the recommendation now.

Fixes #54607
Keno added a commit that referenced this issue Jun 8, 2024
PR #44231 (part of Julia 1.9) introduced the ability to modify globals
with `Mod.sym = val` syntax. However, the intention of this syntax was
always to modify *existing* globals in other modules. Unfortunately, as
implemented, it also implicitly creates new bindings in the other
module, even if the binding was not previously declared. This was not
intended, but it's a bit of a syntax corner case, so nobody caught it at
the time.

After some extensive discussions and taking into account the near future
direction we want to go with bindings (#54654 for both), the consensus
was reached that we should try to undo the implicit creation of bindings
(but not the ability to assign the *value* of globals in other modules).
Note that this was always an error until Julia 1.9, so hopefully it
hasn't crept into too many packages yet. We'll see what pkgeval says. If
use is extensive, we may want to consider a softer removal strategy.

Across base and stdlib, there's two cases affected by this change:
1. A left over debug statement in `precompile` that wanted to assign a
new variable in Base for debugging. Removed in this PR.
2. Distributed wanting to create new bindings. This is a legimitate use
case for wanting to create bindings in other modules. This is fixed in
JuliaLang/Distributed.jl#102.

As noted in that PR, the recommended replacement where implicit binding
creation is desired is:
```
Core.eval(mod, Expr(:global, sym))
invokelatest(setglobal!, mod, sym, val)
```

The `invokelatest` is not presently required, but may be needed by
#54654, so it's included in the recommendation now.

Fixes #54607
KristofferC pushed a commit that referenced this issue Jun 13, 2024
PR #44231 (part of Julia 1.9) introduced the ability to modify globals
with `Mod.sym = val` syntax. However, the intention of this syntax was
always to modify *existing* globals in other modules. Unfortunately, as
implemented, it also implicitly creates new bindings in the other
module, even if the binding was not previously declared. This was not
intended, but it's a bit of a syntax corner case, so nobody caught it at
the time.

After some extensive discussions and taking into account the near future
direction we want to go with bindings (#54654 for both), the consensus
was reached that we should try to undo the implicit creation of bindings
(but not the ability to assign the *value* of globals in other modules).
Note that this was always an error until Julia 1.9, so hopefully it
hasn't crept into too many packages yet. We'll see what pkgeval says. If
use is extensive, we may want to consider a softer removal strategy.

Across base and stdlib, there's two cases affected by this change:
1. A left over debug statement in `precompile` that wanted to assign a
new variable in Base for debugging. Removed in this PR.
2. Distributed wanting to create new bindings. This is a legimitate use
case for wanting to create bindings in other modules. This is fixed in
JuliaLang/Distributed.jl#102.

As noted in that PR, the recommended replacement where implicit binding
creation is desired is:
```
Core.eval(mod, Expr(:global, sym))
invokelatest(setglobal!, mod, sym, val)
```

The `invokelatest` is not presently required, but may be needed by
#54654, so it's included in the recommendation now.

Fixes #54607

(cherry picked from commit b7e7232)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:minor change Marginal behavior change acceptable for a minor release
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants