RFC: Export Keyword for Values

[bradsharp]

Allow users to export methods and values from their libraries directly through the use of the export keyword.

Rendered View

[aatxe]

Perhaps worth mentioning that the future change that would enable
type _EXT.Point = Point -- note: this doesn't actually work today but one day it should would also likely necessitate this restriction on explicit returning as well, since the way we'd likely be able to make that possible is by giving tables the ability to have (phantom, i.e. non-existent at runtime) types associated with them as a value.

[bradsharp]

To me that implies we need to make this functional (even if it's bad). We can't silently drop the second return, and we can't let it override the returned table. If modules supported multiple returns we might be able to get around this by making sure the export table is always the last value returned (e.g. return <user-defined>, _EXT), not sure how others would feel about this.

[aatxe]

Sorry, maybe I wasn't clear. I think the correct behavior here, as you're suggesting in the RFC, is that we should throw an error on any explicit return in a module with export function or export id, and I also think that even though this may seem a bit odd right now since the restriction is specific to exporting values, it's also a restriction we will have for types if we do the work to enable assigning a type into a "field" (not one that exists at runtime) as you had written in the earlier example.

[alexmccord]

One thing I noticed not mentioned here is how this would work with two mutually dependent exported functions. If you replace export with local in this snippet, it doesn't do what you want it to do.

export function f()
  g()
end

export function g()
  f()
end

[aatxe]

I think my personal expectation here is that this is going to behave the same way as locals, and we can perhaps provide a plain export f export g (though I suspect syntactic ambiguity will be an issue here) to let you export a global.

I would be open to hearing reasons why it should go the other way instead and behave like globals, but I think regardless of the choice, we don't want to introduce a new third semantics here, export should behave like one of them.

[bradsharp]

Agree with @aatxe on this one.
If we want to support the described behavior I think we should look into hoisting but that would be a separate RFC.

[alexmccord]

Given that export probably only makes sense at module level, it stands to reason they should work like globals.

[bradsharp]

I've adopted the local semantics since they seem like the best place to start, with the option to forward-declare using const or local. In the future we may consider hoisting functions on the form <keyword> function <ident>()

[nezuo]

I think this is a great UX improvement! I do have one issue with it:

When you require a module with no return in Roblox, it gives the error: "Module code did not return exactly one value."

Having to specify a return by default and then remove it once you add something you want to export would be a bit annoying. I'm not sure what the right solution would be. Is it possible to change the behavior of modules that don't return anything?

[aatxe]

Having to specify a return by default and then remove it once you add something you want to export would be a bit annoying. I'm not sure what the right solution would be. Is it possible to change the behavior of modules that don't return anything?

I think it would be reasonable to have an implicit return {} if no return statement is present since I think that's more-or-less what this RFC is proposing as the meaning of a module (i.e. it returns a table almost always).

[bradsharp]

Having to specify a return by default and then remove it once you add something you want to export would be a bit annoying. I'm not sure what the right solution would be. Is it possible to change the behavior of modules that don't return anything?

I think it would be reasonable to have an implicit return {} if no return statement is present since I think that's more-or-less what this RFC is proposing as the meaning of a module (i.e. it returns a table almost always).

Not sure I agree with return {} being the default. I think it should be return; since you didn't return or export anything.

[ccuser44]

I very much would like this feature in Luau for many reasons (expecially easier AST analysis and standardization), but because it's a syntax change I don't think it meets the bar to be added as a syntax feature.

[ccuser44]

With further introspection I'm leaning fully on the side of not accepting this RFC. While nice in theory the practical gains of the feature are marginal and (in my opinion) don't justifu the increased lexer/parser complexity.

[jackdotink]

As someone who has written a luau parser in luau, and contributed to other luau parsers, the added parsing complexity is just about 0.

[aatxe]

Having to specify a return by default and then remove it once you add something you want to export would be a bit annoying. I'm not sure what the right solution would be. Is it possible to change the behavior of modules that don't return anything?

I think it would be reasonable to have an implicit return {} if no return statement is present since I think that's more-or-less what this RFC is proposing as the meaning of a module (i.e. it returns a table almost always).

Not sure I agree with return {} being the default. I think it should be return; since you didn't return or export anything.

This is already what the behavior is (leaving nothing on the stack) though, and what's why you get an error saying it has to return exactly one value. Returning nothing is returning zero values! There's a question of whether ModuleScript could potentially just be okay with returning zero values as well, but that is technically out of scope for the RFC itself.

[alexmccord]

There's an opportunity to talk about the possibility of adding re-exports here that I think this misses:

export M = require("./foo")

Which can re-export both types and values.

[ccuser44]

As someone who has written a luau parser in luau, and contributed to other luau parsers, the added parsing complexity is just about 0.

Yeah I don't know why I just said that. Adding the export keyword would only require adding a new lex token type and having it generate a SETTABLE instruction that adds the var to the export table.

My point was is that I don't think I don't think this RFC meets the high bar of usefulness that's required for new syntax features. I was originally for approving this RFC but now due to further introsoection I'm againts the idea.

[dphfox]

Here's a question; right now all imports are namespaced, e.g. you access them via ModuleName.foo. However for files that export single functions etc. this doesn't make a lot of sense.

Would we be open to an anonymous version of the syntax for exporting a single function or type for symmetry, or would we remain using return? If so, would we ever be able to return a type that is not namespaced alongside a function that is not namespaced?

[alexmccord]

For those, you can still use return the_thing, but it does mean an XOR: either you use return or you use export, not both. It's not too hard to change between them, though. Also seems a bit of a semver hazard if you have to export more than one thing.

[SPY]

I would like to double down on this proposal.
Statically known and immutable exports can unblock a whole class of cross-module optimizations in VM(first of all inlining with constant folding and dead code elimination, but also some checks can be skipped in table access code).
It will play especially great together with static require/imports, but even without it a dependency graph can be recovered with bytecode analysis.

[alexmccord]

Can I please see some of the feedback posted on this issue be folded into the actual document?

[bradsharp]

I've updated this RFC to clarify mutual dependence and also const-ness of values (to avoid the case where you can mutate a value within the module, but not outside).

[alexmccord]

Alternative framing: this is not necessarily a const binding.

export foo = 7
print(foo)
foo = 8

After elaboration, all occurrences of foo are a field projection of the export table that you cannot utter in the surface syntax:

local mod = {}
mod.foo = 7
table.freeze(mod)

print(mod.foo)
mod.foo = 8

And since a field projection foo inherits the permissions of the projected-from lvalue mod which is frozen, foo cannot be reassigned. This avoids talking about const bindings and everything is still framed around the pre-existing mechanism of frozen tables.

[SPY]

It doesn't work if we try to desugar several declarations this way:

export a = 2 -- 1
a = 3        -- 2
export b = 4 -- 3

If table.freeze happens before 2, then 3 should rise an error.
If table.freeze happens at the end 2 shouldn't be an error.
It is easer to use const semantic here

[alexmccord]

Elaboration is not desugaring and is allowed to fail. Consider:

export a = 1
print(a)
export a = 2
print(a)

If we desugar to its const form, the above snippet is valid because const is allowed to shadow. This is effectively the same as your example. You'd need a rule in this case that rejects any export whose name have already been exported. This makes exports inexpressible in terms of const bindings.

[SPY]

I would consider that code snipped valid. It outputs 1 and 2 and exports a as 2.
I understand it can be confusing for corner cases, but I think this approach makes the language more consistent in design choices.
We could prevent this kind of behavior to disallow constant shadowing on the same level(as JS does), allowing it only in nested scopes. I think it is a good idea in general, but it contradicts Lua(u)'s behavior of shadowing rules for locals. So I would like not to do it to stay consistent.
Also, I consider idiomatic to use "local" intuition for explaining Lua(u)'s syntactic sugar.
For example, local function f is traditionally explained as local f + f = function() formula. Desugaring export with const we keep it 'local' in reasoning without bringing global file-level semantic.

[alexmccord]

Not sure the RFC should show the desugared representation with any constructs that doesn't exist.

[SPY]

I think it is acceptable as soon as dependency on other RFC explicitly stated and const RFC is not rejected yet.
Of course export proposal cannot be land in this form until const proposal is.

[alexmccord]

Exporting before a binding is initialized is an error.

Agreed.

Supporting hoisted exports or forward declarations is out of scope for this proposal and may be explored in a future RFC.

Disagree. This is not backward compatible to retrofit and as such it is not a decision we can defer until later, and mutually dependent functions are fundamental in nontrivial programs. Pre-existing idiom allows mutual dependencies already, so this RFC is more or less a downgrade in its current form. In short, this is a design choice we have to make now or never, and deferring a backward-incompatible design choice is the same as choosing to never do it.

If we could resolve names in function bodies after parsing the body itself rather than immediately, then we already support mutual dependencies. This is sort of easy from a conceptual point of view: after parsing the module, all occurrences of AstExprGlobal implies a free variable that are yet to be bound. Walk each occurrence and check if an export binder name exists and then rebind that occurrence to the export binder.

export type t<a> =
  | { type: "leaf", value: a }
  | { type: "node", children: {t<a>} }

export function walk_node<a, b>(
  node: t<a>,
  leaf_fn: (a) -> b,
  node_fn: ({b}) -> b
): b
  return if node.type == "leaf"
    then leaf_fn(node.value)
    else node_fn(walk_children(node.children, leaf_fn, node_fn))
end

export function walk_children<a, b>(
  children: {t<a>},
  leaf_fn: (a) -> b,
  node_fn: ({b}) -> b
): {b}
  local result = table.create(#children)
  for _, node in children do
    table.insert(result, walk_node(node, leaf_fn, node_fn))
  end
  return result
end

As you can see, it is not possible to reorder either walk_node or walk_children. Initially, when the AST is produced, the walk_children at line 12 is an AstExprGlobal which is analogous to a free variable in lambda calculus, and during name resolution we find out that we could bind that walk_children to the binder export function walk_children turning it into a bound variable. Voila, you have mutual dependencies.

This approach will require changes in everything else though, since if export function walk_children defines a local named walk_children, then the reference at line 12 must be an AstExprLocal, which lexically occurs before its declaration! There's a bunch of ICEs that will happen and needs to be taken care of. Presumably the bytecode compiler requires more fixes as well if this was added (@vegorov-rbx?).

With that said, I will be contrarian for a minute because there are problems on the other side of this beyond the existing implementation assumptions that needs to be solved if we decided to support mutual dependencies out of the box, because there's a critical caveat to note: we don't want to rebind any AstExprGlobal to any exports before it was declared except those inside of a function body, and the above approach must continue to maintain this invariant, otherwise you'll end up with nasty bugs in the program like this:

print(nasty()) -- nil

local x = 5

export function nasty()
  return x
end

Which is precisely equivalent to JavaScript's hoisting bug:

console.log(nasty()); // undefined

var x = 5; // change to `let` or `const` for TDZ semantics

function nasty() {
  return x;
}

ES6 added let/const as well as TDZ (short for "temporal dead zone") semantics. Note that we can't make this a compile error because of the below snippet which is "technically well-defined" as per the rule I gave earlier, but is still ill-formed because local x = 5 has yet to be executed:

export function hella_nasty()
  return nasty()
end

print(hella_nasty())

local x = 5

export function nasty()
  return x
end

Unfortunately it is not possible to turn this class of bugs into a compile error because the analysis is nontrivial, and, per Rice's theorem, is undecidable in the general case (hence why TDZ was punted onto the runtime), so if we did support mutual dependencies, then we have these choices:

1. TDZ semantics (please, for the love of god, no)
2. Don't allow module-level code to call exported functions
   • Honestly, module-level code should seldom have side-effects anyway, sans entry points. They're a bit like impl blocks in Rust.
   • That would mean removing this: https://github.com/luau-lang/rfcs/pull/42/changes#diff-444355a7a572290ed4e1533ed7a9de5b36d25acde77f1c7e4d3893558bf2801bR116-R126
3. OCaml-style and for explicit grouping of mutual dependencies.

   export function f()
     ... -- g is available here
   end
   and g()
     ... -- f is available here
   end

[SPY]

I don't think we need to introduce hoisting for this purpose.
It brings more harm than convenience.
I would prefer the code snippet above can be expressed with usage of temporary locals.

export type t<a> =
  | { type: "leaf", value: a }
  | { type: "node", children: {t<a>} }

local walk_children: <a, b> (
  children: {t<a>},
  leaf_fn: (a) -> b,
  node_fn: ({b}) -> b
) -> {b}

export function walk_node<a, b>(
  node: t<a>,
  leaf_fn: (a) -> b,
  node_fn: ({b}) -> b
): b
  return if node.type == "leaf"
    then leaf_fn(node.value)
    else node_fn(walk_children(node.children, leaf_fn, node_fn))
end

walk_children = function<a, b>(
  children: {t<a>},
  leaf_fn: (a) -> b,
  node_fn: ({b}) -> b
): {b}
  local result = table.create(#children)
  for _, node in children do
    table.insert(result, walk_node(node, leaf_fn, node_fn))
  end
  return result
end

export walk_children

I also like OCaml style approach. In this case my snippet will be the desugaring of it.

[alexmccord]

This is quite awkward. What is the value of mod.foo after you call mod.increment()?

local foo = 5

export function increment()
	foo += 1
end

export foo -- ???

Currently, mod.foo after calling mod.increment() would always be 5 because it copies the value of foo into the field foo of the exported table. Instinctively, I want it to be 6 because, to me, export foo looks like it exports an alias to the inner foo which is distinct from export foo = foo which copies the value of foo into the read-only symbol foo.

I know that this RFC says export foo is sugar for export foo = foo, but I'm not so sure about that now, it enables a footgun:

local foo = 5

export function increment()
  foo += 1
end

export foo

export function get()
  return foo -- this will always be 5
end

And we can't even have a linter fire a warning on this (doesn't make sense).

A fix is to be isomorphic to something akin to this, using __index to provide aliasing.

local foo = 5

local function increment()
  foo += 1
end

-- this is an empty table because of `for k, v in mod do` which conflicts with aliasing
return table.freeze(setmetatable({}, {
  __index = function(_, k)
    return if k == "foo" then foo
      else if k == "increment" then increment
      else nil
  end,
}))

[SPY]

I agree this approach would be more flexible and beneficial for some code patterns, but in this case we will loose ability to statically reason about exported values during loading. It will impede some potential optimizations for cross module inlining and constant folding. As soon as Luau doesn't allow to control readonlyness with property level granularity we will need to make some compromises here.
User intention can be expressed using getter function or intermediate object.

[alexmccord]

I mean, an optimization you can do is literally just move some of the exported symbols into the table part for all exported bindings that have not been mutated (or rather, not mutated through functions). From that you have a guarantee that increment always points to the same function object, whereas foo still goes through __index.

The reason why I didn't do that was because of for k, v in mod do would not iterate over foo. But here, let's just do that. The downside is that now pairs(mod)/rawget(mod, "increment") is weird.

local order = {"foo", "increment"} -- sorted by hash, just like hashmaps
return table.freeze(setmetatable({ increment = increment }, {
  __index = function(_, k)
    return if k == "foo" then foo
      else nil
  end,
  __iter = function(self)
    local i = 0
    return function()
      i += 1
      local k = order[i]
      return k, self[k]
    end
  end,
}))

Problem solved, the compiler or VM can still see that increment field exists in the table and did not require __index.

Module systems are complicated for a reason. You can't hand-wave all the issues away in an RFC with 165 lines that barely even talks about edge cases.

[MagmaBurnsV]

I say if you plan on reintroducing assignability through shorthand exports, don't even bother making exported values unassignable in the first place. I'm actually leaning more in the camp of not even supporting shorthand exports because of this issue.

In my personal opinion, exporting should just be pure syntax sugar that maps exactly to returning a plain frozen table. There should be no special metatable mechanics like the proposed aliasing rules in the desugared form.

[alexmccord]

Yeah, I think just remove the shorthand export. This RFC is proposing CommonJS-esque modules, not anything fancy like ES modules which has live bindings.

[MagmaBurnsV]

I believe you should still be able to export values from within do end blocks, as this could allow for neat namespacing with lexical scoping:

do
	local count = 0
	
	export function increment(): number
		count += 1
		return count
	end
end

-- count and increment not visible here

[alexmccord]

This is what local modules does, if we had some sort of module keyword.

module M is
  local count = 0

  export function increment(): number
    count += 1
    return count
  end
end

export increment = M.increment

Which is equivalent to a closure, modulo (heh) namespacing/qualified paths/etc.

local function makeM(): () -> number
  local count = 0

  return function() -- ignoring the _actual_ exporting machinery as red herring
    count += 1
    return count
  end
end

export increment = makeM()

You could even instantiate it multiple times. Module functors.

module Make(n: number)
  local count = 0
  
  export function next(): number
    count += n
    return count
  end
end

module Counter = Make(1)

export increment = Counter.next

export incrementX = makeM() -- has its own independent `count` state
export incrementY = makeM() -- and its own independent `count` state

[MagmaBurnsV]

I've been thinking about possible alternative semantics, and this is what I've come up with: what if we treat exporting values as like assigning keys to the export table and allow exported values to be reassigned until the end of the module when the export table is frozen and returned.

This mirrors the way people currently write module exports, and it solves all the problems with mutually dependent functions, aliased shorthand exports (you don't need shorthand syntax anymore), and still allows for lexical scoping rules:

export a = 1
a = 2

-- becomes

local _EXP = {}
_EXP.a = 1
_EXP.a = 2

return table.freeze(_EXP)

-- like export f, g = nil, nil
export f, g

function f()
	-- calling g() before it is assigned
	-- throws an "attempt to call a nil value" error
	g()
end

function g()
	f()
end

-- becomes

local _EXP = {}
_EXP.f = nil
_EXP.g = nil

function _EXP.f()
	_EXP.g()
end

function _EXP.g()
	_EXP.f()
end

return table.freeze(_EXP)

export foo = 5

export function increment()
	-- this works until the module ends and freezes the export table
	-- which subsequently throws an "attempt to modify a readonly table" error
	foo += 1
end

export function get()
  return foo
end

-- becomes

local _EXP = {}
_EXP.foo = 5

function _EXP.increment()
	_EXP.foo += 1
end

function _EXP.get()
	return _EXP.foo
end

return table.freeze(_EXP)

do
	local count = 0
	
	export function increment(): number
		count += 1
		return count
	end
	
	-- count and increment only visible in here
end

-- becomes

local _EXP = {}

do
	local count = 0
	
	function _EXP.increment(): number
		count += 1
		return count
	end
end

return table.freeze(_EXP)

There are no extra features like const required to make this work, and I would imagine it would also still be possible to unlock cross-module optimization opportunities this way by inlining and folding exports that are never reassigned.

[SPY]

what if we treat exporting values as like assigning keys to the export table and allow exported values to be reassigned until the end of the module when the export table is frozen and returned.

One of the primary motivation for this proposal is to make import/export pairs to be statically resolvable without executing top-level stat. Seal at the last moment semantic brings in Turing completeness and makes it impossible:

-- utils.luau
export f = function(n: number): string return "42" end

if globalVar > 67 then
   f = function(s: string): string return 42 end
end

-- main.luau
import f from "./utils.lual"

f(22) -- cannot be inlined during compilation phase
-- what type of f?

[MagmaBurnsV]

As I alluded to at the end, you can still do static optimizations by determining if the exported value is ever reassigned to. In your example, f can't be inlined because it obviously gets reassigned to, but if it didn't, it would be safe to optimize away. This is how current local inlining and constant folding work.

Typing also works if you apply the same rules that local variables currently have with flow-sensitive type inference:

local f = function(n: number): string return "42" end

if globalVar > 67 then
	f = function(s: string): string return 42 end
end

-- f: ((n: number) -> string) | ((s: string) -> number)