Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

copying a single external to a new location #8748

Open
rickyvetter opened this issue Jun 18, 2019 · 10 comments

Comments

@rickyvetter
Copy link

commented Jun 18, 2019

Currently there is no way to copy a single external into a new module/location without reifying into a value. This makes writing code that can manipulate modules with external values tricky (functors, first-class modules, ppx). This is especially true for environments where accessing an external too early in execution can cause unexpected changes to code size. I have seen this specifically when targeting JS with BuckleScript and modeling dynamic code loading that is common in the JS ecosystem, but I would imagine you could hit these types of issues while targeting any platform with the capability of asynchronously loading code as a way of deferring work.

The functionality for copying an external value does exist in the form of an include for the module, but I'd like to request a feature in the form of new syntax that allows for copying a single external value without casting such a wide net.

module Target = struct
  type t
  external ext : unit -> t = "foo"
end

module Reifies_too_early = struct let ext = Target.ext end

module Copies_too_much = struct include Target end

Seemingly available syntax:

external ext = Target.ext

Is there any wider interest in this functionality?

@nojb

This comment has been minimized.

Copy link
Contributor

commented Jun 18, 2019

Could you provide a concrete example where this functionality would be useful ? Thanks!

@bobzhang

This comment has been minimized.

Copy link
Member

commented Jun 19, 2019

@nojb Currently when aliasing an external function using let let f = external_f, the compiler would create a function, the proposal want to keep/propagate the external property so that no code generated. It helped most in JS backend where function creation has a performance cost and code size matters

@lpw25

This comment has been minimized.

Copy link
Contributor

commented Jun 19, 2019

Why is:

module Foo = struct external ext : unit -> t = "foo" end

not sufficient?

You have to restate the primitive and the type, but if I understand your proposal correctly you'd have to do that in the .mli file anyway.

@rickyvetter

This comment has been minimized.

Copy link
Author

commented Jun 19, 2019

You have to restate the primitive and the type

This is exactly why. If we are copying an external in a functor or via ppx we don't have the ability to restate the type.

you'd have to do that in the .mli file anyway

I'm not really sure why a similar syntax wouldn't be viable. The proposal points to an external that carries both value and type information. external ext: Target.ext seems like a pretty reasonable syntax - although it may cause collisions with existing external syntax, in which case = might work. Very similar to val foo: Target.foo.

Even if we had to restate the type in the .mli, this opens up the ability to flexibly work with externals in implementations which seems valuable to me.

@lpw25

This comment has been minimized.

Copy link
Contributor

commented Jun 20, 2019

I'm not really sure why a similar syntax wouldn't be viable.

I guess it depends on how we think about externals and what their notion of equality is. At first it seemed strange to me because you could have:

external foo : int -> int = "foo_ext"
module M : sig
  external bar = foo
end = struct
  external bar : int -> int = "foo_ext"
end

where the interface says the externals are equal but the implementations are unrelated. This seemed odd because I was thinking of the externals as being generative/nominative in some sense. However, they are probably better thought of as being structural. You can clearly see them as aliases for some pre-existing (but not directly referable in expressions) external -- named "foo_ext" in the above example. From that perspective it seems fine to have aliases like the ones you propose appear in signatures.

@gasche

This comment has been minimized.

Copy link
Member

commented Jun 20, 2019

I agree that external aliases of the form external foo = <identifier> would make sense (in both structures and signatures). This is not a very elegant feature (I think that generally we've been trying to move away from having externals in signatures and expose val there, which is more user-facing) but it could in theory make sense for advanced users (and maybe API-generation tools, I suppose?).

I would still like to see, as @nojb suggested, a concrete example of program that has a problematic behavior. It is not obvious to me when it is that users want to rebind externals or otherwise "manipulate modules with externals in them" (in a way that assumes explicit knowledge that those externals have special compilation properties).

@rickyvetter

This comment has been minimized.

Copy link
Author

commented Jun 20, 2019

So here's a snippet from a ppx that won't work as we would like with some comments explaining the issue. Please excuse the code - it's a little messy.

The reason we are operating on modules with known externals here is because in ReasonReact a component is actually a pair of functions, one of which is usually an external.

I'm not sharing an entire project with potential output because it's very JS-target specific, but if it would help to have a working project let me know and I can try and set one up (currently working in a closed-source project so it's not super easy to share). I stand by my statement that this kind of issue could happen in functor/first-class module work as well and could happen in other languages that OCaml interfaces with via external, but it'd take me much longer to construct such an example.

@rickyvetter

This comment has been minimized.

Copy link
Author

commented Jun 20, 2019

This is not a very elegant feature (I think that generally we've been trying to move away from having externals in signatures and expose val there, which is more user-facing)

I think it's only "not very elegant" if your goal is to live in a pure OCaml world. If your goal is rich interop with a target language, it feels quite elegant. As much as I would enjoy living in a pure OCaml world these days, it's not the reality for many of us. Having rich and meaningful interop is a huge feature of OCaml that I would love to see celebrated. The fact that we can use idiomatic OCaml to compile to idiomatic and runtime-free JS for a library as complex as React is absolutely awesome, and could not happen without externals being exposed.

@gasche

This comment has been minimized.

Copy link
Member

commented Jun 20, 2019

I certainly didn't want to give the impression of criticizing your use-case with my remark on externals. My problem with externals is that they don't, to my knowledge, have a clear specification in terms of some precise model of the language. They are used in a certain way because their typical compilation scheme happens to allow it, but what they mean is not clearly specified.

@rickyvetter

This comment has been minimized.

Copy link
Author

commented Jun 20, 2019

No, I don't feel criticized and hope I'm not taken as critical of your statement either. I think we likely have very different use-cases and I just wanted to express how this might feel very different for folks coming JS and moving toward writing OCaml.

I agree that the hand-wavy specification of externals is a major issue with this proposal and that if we were to formalize externals it would be much easier to talk about. That said, I think this is a worthwhile goal. The OCaml documentation mentions externals almost exclusively in the context of C interop with the native compiler. This doesn't capture the actual broad uses by every language-target-compiler I'm aware of - specifically both js_of_ocaml and BuckleScript make heavy use of externals for JS interop. Specifying externals and documenting in a way that provides a consistent way for these compilers to work would be pretty awesome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.