Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extending `open` to accept arbitrary module expression #1506

Closed
wants to merge 46 commits into from

Conversation

Projects
None yet
@objmagic
Copy link
Contributor

commented Dec 2, 2017

This pull request allows open to accept an arbitrary module expression.

Full paper accepted at OCaml Workshop 2017 (online playground is not up-to-date with this pull request)

Summary

OCaml provides two operations for introducing names exported from one module into another module: open and include. Both operations introduce M's bindings into the current scope.
include also re-exports the bindings from the current scope.

A second difference between open and include concerns the form of the argument. The argument to open can only be a module path: open A.B.C. The argument to include, however, can be any module expression:

 include F(X)  include (M:S)  include struct ... end

This PR proposes extending open to eliminate that second difference, so that both open and include accept an arbitrary module expression as argument:

type include open
path include A.B.C open A.B.C
with module type constraint include (M: S) open (M:S)
structure include struct ... end open struct ... end
functor application include F(X) open F(X)

This extension is not added into local opens.

Examples

The extended open has many applications. Please see examples in paper above.

Implementation

Most of the work was done in typing/typemod.ml. For an extended open statement like open struct ... end, we go over the following steps:

  1. Type-check the module expression
  2. Generate a module identifier, say, M#1, then insert module M#1 = struct ... end before open statement
  3. Call Env.open_signature on M#1 to export bindings into environment
  4. Check if the generated identifier can be eliminated from rest of the structure (see section 4 "Restrictions and design considerations" in the paper above for details).

Different from paper, this patch also introduced open extension for let-expression. Scope checking of generated module identifier comes for free in let-expression.

I would appreciate reviews and corner cases that I fail to cover.testsuite/tests/typing-modules/open-struct.ml contains some test cases that one may find useful when reviewing this patch. If you want to try out this patch, follow the tips here.

$ opam repo add ocaml-pr https://github.com/ocaml/ocaml-pr-repository.git
$ opam switch 4.06.0+pr1506
$ eval `opam config env`

I am grateful to @yallop and @lpw25 for their help and review.

objmagic added some commits Apr 9, 2017

Extend `open`.
Signed-off-by: Runhang Li <marklrh@gmail.com>
Merge branch 'trunk' into open-struct-final
Signed-off-by: Runhang Li <marklrh@gmail.com>
Correct `open` for module sig.
Check if we can eliminate identifier in module sig
no matter if we are in nested open.

@objmagic objmagic referenced this pull request Dec 2, 2017

Open

ppx_export mark 2 #1251

7 of 7 tasks complete
@objmagic

This comment has been minimized.

Copy link
Contributor Author

commented Dec 28, 2017

cc @alainfrisch

May I have a round of review?

@Octachron
Copy link
Contributor

left a comment

After a quick look, I had a few questions in term of user-facing messages and API, and code readability:

let loc = Location.in_file "command line" in
let lid = {loc; txt = Longident.parse m } in
let me = Parsetree.({pmod_desc=Pmod_ident lid; pmod_loc=loc; pmod_attributes=[]}) in
snd (Typemod.type_open_ ?toplevel:None Override env lid.loc me)

This comment has been minimized.

Copy link
@Octachron

Octachron Dec 28, 2017

Contributor

This pattern of first creating a module expression from a Longident.t then applying type_open_ appears quite frequently. Would it make sense to have a type_open_id function fot this common case?

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Jan 8, 2018

Contributor

Alternatively, one could introduce a helper function such as:

let modexpr_of_lid lid = {Parsetree.pmod_desc=Pmod_ident lid; pmod_loc=lid.loc; pmod_attributes=[]}

(or adapt Ast_helper.ident to use the lid location when ~loc is not passed explicitly)

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Jan 8, 2018

Contributor

Or just use Ast_helper.ident as is, since type_open would use the location of the lid, not of the module_expr in that case anyway.

@@ -322,7 +322,7 @@ module Mb:
module Opn:
sig
val mk: ?loc: loc -> ?attrs:attrs -> ?docs:docs ->
?override:override_flag -> lid -> open_description
?override:override_flag -> module_expr -> open_description

This comment has been minimized.

Copy link
@Octachron

Octachron Dec 28, 2017

Contributor

Similarly, I think it would be useful to have specialized function for the lid case. Moreover, since mk is exposed in compiler-libs, I am wondering if it would not be more pratical for users to add a mk_expr for the generic case and let mk as the specialized version.

let gen_mod_ident () =
let n = !mod_ident_counter in
incr mod_ident_counter;
let ident = Ident.create (Printf.sprintf "M#%d" n) in

This comment has been minimized.

Copy link
@Octachron

Octachron Dec 28, 2017

Contributor

I find this naming scheme M#%d quite lacking since both the name M and the identifier %d are arbitrary and not really informative for users. Would it be possible to use open#%location#optional_disambiguation_id or a more expressive naming scheme?

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Mar 26, 2018

Contributor

Also, cosmetic: let ident = ... in ident => ....

let remove =
List.filter (
function
| Sig_module({Ident.name}, _, _) when String.contains name '#' -> false

This comment has been minimized.

Copy link
@Octachron

Octachron Dec 28, 2017

Contributor

This feels a bit brittle and not really informative, would it be possible to define a is_open_generated (or a better name) function rather than use String.contains name '#' ?

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Jan 8, 2018

Contributor

I'd also suggest inlining remove where it is used (in the case for Mty_signature), and drop the aux inner function (just making remove_inserted_modtype recursive). (Btw, it's not yet obvious to me why we need to recurse under mty_arg -- I haven't reviewed all the PR yet.)

This comment has been minimized.

Copy link
@objmagic

objmagic Jan 21, 2018

Author Contributor

@alainfrisch

One can openin functor argument signature

module M (S: sig open struct ... end ... end) = struct ... end

if StringSet.mem name val_names then k
else (component :: sg, StringSet.add name val_names)
| Sig_module({Ident.name}, _, _) :: sg
when String.contains name '#' -> aux sg

This comment has been minimized.

Copy link
@Octachron

Octachron Dec 28, 2017

Contributor

Same remark as above: having a specific function rather String.contains name '#' would be nice.

| exception Not_found ->
raise(Error(sod.popen_loc, env,
Cannot_eliminate_anon_module(id, rem)))
| _ -> assert false

This comment has been minimized.

Copy link
@Octachron

Octachron Dec 28, 2017

Contributor

Why is this case impossible? A comment would be nice.

@@ -2020,6 +2182,11 @@ let report_error ppf = function
fprintf ppf
"This is an alias for module %a, which is missing"
path p
| Invalid_open _me ->
fprintf ppf "Invalid open"

This comment has been minimized.

Copy link
@Octachron

Octachron Dec 28, 2017

Contributor

This error message is a bit dry. Why is the open invalid? Why is the module expression not used?

This comment has been minimized.

Copy link
@objmagic

objmagic Apr 8, 2018

Author Contributor

now Invalid_open is no longer used so I removed it.

fprintf ppf "Invalid open"
| Cannot_eliminate_anon_module (id, sg) ->
fprintf ppf "The module identifier %a cannot be \
eliminated from %a" ident id signature sg

This comment has been minimized.

Copy link
@Octachron

Octachron Dec 28, 2017

Contributor

I would add an enclosing box @[…@] and at least a break hint before the signature (from@ %a).

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Jan 8, 2018

Contributor

The message should be improved to mention that the module identifier has been synthesized and correspond to the opened module expression. Even better of course would be to never leak the identifier in the error message and pin-point the actual signature item whose type still refer to that identifier, but it's more difficult.

| None ->
let md = Env.find_module path env in
ignore (extract_sig_open env lid.loc md.md_type);
assert false

This comment has been minimized.

Copy link
@Octachron

Octachron Dec 28, 2017

Contributor

I know the code was here originally, but it may be nice to add a comment explaining that extract_sig_open raises an structure expected since the opened module is a functior in this branch.

This comment has been minimized.

Copy link
@objmagic

objmagic Jan 21, 2018

Author Contributor

I have refactored so that extraction is no longer needed

push_current_mid (ident, md, newenv);
let root = Pident ident in
match Env.open_signature ~loc ?used_slot ?toplevel ovf root newenv with
| None -> assert false

This comment has been minimized.

Copy link
@Octachron

Octachron Dec 28, 2017

Contributor

I still think that a comment by assert false sounds nice.

@ubsan

This comment has been minimized.

Copy link

commented Jan 5, 2018

This would be awesome for me! I'm implementing a sort of typeclass thing, and it'd be great to not have to call the typeclass implementation functors, except in open.

(right now, I have something like

(* in Result *)
module Monad(E: Type) = struct
  type error = E.t;
  ...
end

(* in cafec/parse/error.re *)
module Monad_result = Result.Monad(sig type nonrec t = t end);

(* in cafec/parse/parse.re *)
open Error.Monad_result;

I'd rather write

(* Result is the same *)

(* in cafec/parse/parse.re *)
open Result.Monad(sig type t = Error.t end)

)

@ubsan

This comment has been minimized.

Copy link

commented Jan 6, 2018

There's a bug in this patch. I've installed this and switched to it, and attempted to install jbuilder:

$ opam install jbuilder
The following actions will be performed:
  ∗  install conf-m4   1                      [required by ocamlfind]
  ∗  install ocamlfind 1.7.3                  [required by jbuilder]
  ∗  install jbuilder  1.0+beta16
===== ∗  3 =====
Do you want to continue ? [Y/n] y

=-=- Gathering sources =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[default] https://opam.ocaml.org/archives/ocamlfind.1.7.3+opam.tar.gz downloaded
[default] https://opam.ocaml.org/archives/jbuilder.1.0+beta16+opam.tar.gz downloaded

=-=- Processing actions -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
∗  installed conf-m4.1
∗  installed ocamlfind.1.7.3
[ERROR] The compilation of jbuilder failed at "ocaml bootstrap.ml".

#=== ERROR while installing jbuilder.1.0+beta16 ===============================#
# opam-version 1.2.2
# os           linux
# command      ocaml bootstrap.ml
# path         /home/nicole/.opam/4.06.0+pr1506/build/jbuilder.1.0+beta16
# compiler     4.06.0+pr1506
# exit-code    2
# env-file     /home/nicole/.opam/4.06.0+pr1506/build/jbuilder.1.0+beta16/jbuilder-6972-8b8a2d.env
# stdout-file  /home/nicole/.opam/4.06.0+pr1506/build/jbuilder.1.0+beta16/jbuilder-6972-8b8a2d.out
# stderr-file  /home/nicole/.opam/4.06.0+pr1506/build/jbuilder.1.0+beta16/jbuilder-6972-8b8a2d.err
### stdout ###
# /home/nicole/.opam/4.06.0+pr1506/bin/ocamllex.opt -q src/sexp_lexer.mll
# /home/nicole/.opam/4.06.0+pr1506/bin/ocamllex.opt -q src/meta_lexer.mll
# /home/nicole/.opam/4.06.0+pr1506/bin/ocamldep.opt -modules src/action.ml src/action_intf.ml src/alias.ml src/ansi_color.ml src/arg_spec.ml src/artifacts.ml src/bin.ml src/build.ml src/build_interpret.ml src/build_system.ml src/clflags.ml src/cm_kind.ml src/config.ml src/context.ml src/file_tree.ml src/findlib.ml src/future.ml src/gen_meta.ml src/gen_rules.ml src/glob_lexer.boot.ml src/import.ml src/install.ml src/io.ml src/jbuild.ml src/jbuild_load.ml vendor/boot/jbuilder_opam_file_format.ml vendor/boot/jbuilder_re.ml src/js_of_ocaml_rules.ml src/lib.ml src/lib_db.ml src/loc.ml src/log.ml src/main.ml src/merlin.ml src/meta.ml src/meta_lexer.ml src/ml_kind.ml src/mode.ml src/module.ml src/module_compilation.ml src/ocaml_flags.ml src/ocamldep.ml src/odoc.ml src/opam_file.ml src/ordered_set_lang.ml src/package.ml src/path.ml src/sexp.ml src/sexp_lexer.ml src/string_with_vars.ml src/super_context.ml src/top_closure.ml src/utils.ml src/utop.ml src/vfile_kind.ml src/watermarks.ml src/workspace.ml > boot-depends.txt
# /home/nicole/.opam/4.06.0+pr1506/bin/ocamlopt.opt -w -40 -o boot.exe unix.cmxa boot.ml
### stderr ###
# Fatal error: exception Invalid_argument("index out of bounds")



=-=- Error report -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
The following actions failed
  ∗  install jbuilder 1.0+beta16
The following changes have been performed
  ∗  install conf-m4   1
  ∗  install ocamlfind 1.7.3
@gasche

This comment has been minimized.

Copy link
Member

commented Jan 6, 2018

@ubsan: I believe this is not caused by the patch itself. During a few weeks, trunk broke jbuilder (see #1493). This is now fixed, but I believe the present PR was submitted from a broken-jbuilder trunk version, and it needs to be rebased on the current trunk (which correctly builds jbuilder).

@ubsan

This comment has been minimized.

Copy link

commented Jan 6, 2018

@gasche ah, okay :) thanks!

on another thing - the paper doesn't support local opens, apparently, but I'd really love local opens to support this as well, for purposes similar to typeclasses.

module type Print = sig
  type t
  val print: t -> unit
end
module Print_int: Print with type t = int = struct
  ...
end
module Print_list(P: Print): Print with type t = P.t list = struct
  ...
end

let print_list_of_int lst = Print_list(Print_int).(print lst)
@objmagic

This comment has been minimized.

Copy link
Contributor Author

commented Jan 8, 2018

@ubsan Thanks for your comment!

Extension on local open introduces grammar ambiguity. I didn't look closely or try to remove ambiguities because at first I thought such use case is rare. However, one reviewer and you mentioned it would be good to add support on local open as well. I can definitely look into this again.

@ubsan

This comment has been minimized.

Copy link

commented Jan 8, 2018

@objmagic thanks <3

@@ -117,7 +117,7 @@ let iterator =
in
let open_description self opn =
super.open_description self opn;
simple_longident opn.popen_lid
super.module_expr self opn.popen_expr

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Jan 8, 2018

Contributor

Is this call to super.module_expr really necessary? It should be done already by super.open_description, no?

This comment has been minimized.

Copy link
@objmagic

objmagic Jan 21, 2018

Author Contributor

Good catch. Fixed.

@@ -69,7 +69,7 @@ and expression =
and exp_extra =
| Texp_constraint of core_type
| Texp_coerce of core_type option * core_type
| Texp_open of override_flag * Path.t * Longident.t loc * Env.t
| Texp_open of override_flag * Longident.t loc * Env.t

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Jan 8, 2018

Contributor

Cannot we arrange to keep the Path.t here? (This concerns goes away if you manage to support local opens, in which case one would get the full module_expr.)

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Jan 8, 2018

Contributor

Also, if the Tpat_open, Texp_open, Tcl_open, Tcty_open constructors change, perhaps it would be a good opportunity to have them refer to open_description for uniformity (adding the Env.t to open_description).

This comment has been minimized.

Copy link
@objmagic

objmagic Jan 21, 2018

Author Contributor

I am not sure about supporting local opens. @yallop and I agree that such extension seems not very useful. However, one paper reviewer, you, and ubsan (in this PR) believe it is nice to have.

Let me see if it is possible to resolve grammar ambiguity first.

This comment has been minimized.

Copy link
@objmagic

objmagic Mar 21, 2018

Author Contributor

I have managed to support local open and refactored all of those variants to use open_description. I only extended Pexp_open this time but not opens in class.

let type_open_ ?used_slot ?toplevel ovf env loc me =
match me.pmod_desc with
| Pmod_functor _ | Pmod_unpack _ | Pmod_extension _ ->
raise(Error(me.pmod_loc, env, Invalid_open me))

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Jan 8, 2018

Contributor

Why is Pmod_unpack always rejected here? Should we plumb funct_body through type_open_ to type_module?

This comment has been minimized.

Copy link
@objmagic

objmagic Jan 21, 2018

Author Contributor

Fixed. Pmod_unpack is now supported.

Should we plumb funct_body through type_open_ to type_module

I am not following. Can you explain a bit more here?

ignore (extract_sig_open env lid.loc md.md_type);
assert false
end
| _ -> begin

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Jan 8, 2018

Contributor

It seems more idiomatic/common to put the begin...end around the tail match...with, not around the whole case.

This comment has been minimized.

Copy link
@objmagic

objmagic Apr 8, 2018

Author Contributor

done

let ident = gen_mod_ident () in
let tme = !type_module_fwd env me in
leave_struct ();
begin

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Jan 8, 2018

Contributor

begin match on the same line is more idiomatic.

This comment has been minimized.

Copy link
@objmagic

objmagic Apr 8, 2018

Author Contributor

done

begin
match tme.mod_type with
| Mty_signature _ -> ()
| _ -> raise(Error(me.pmod_loc, env, Invalid_open me));

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Jan 8, 2018

Contributor

It would be better to list explicitly all cases, and assert false for cases that cannot happen (Mty_alias, I believe).

This comment has been minimized.

Copy link
@objmagic

objmagic Apr 8, 2018

Author Contributor

done

mb_attributes=od.open_expr.mod_attributes;
mb_loc=od.open_expr.mod_loc} in
Some (tm, md, id, env)
| _ -> assert false

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Jan 8, 2018

Contributor

Please list remaining constructors explicitly, it helps reasoning about the code.

This comment has been minimized.

Copy link
@objmagic

objmagic Apr 8, 2018

Author Contributor

done

let (trem, rem, final_env) = transl_sig newenv srem in
let remr = ref rem in

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Jan 8, 2018

Contributor

Please avoid this reference here.

This comment has been minimized.

Copy link
@objmagic

objmagic Jan 21, 2018

Author Contributor

fixed. I forgot to clean this up.

mksig (Tsig_open od) env loc :: trem,
rem, final_env
!remr, final_env

This comment has been minimized.

Copy link
@alainfrisch

alainfrisch Jan 8, 2018

Contributor

Can trem contains reference to the generated identifier? It could confuse external tools that would see references to unbound module identifiers. What about keeping the generated identifier in the typedtree?

@yallop

This comment has been minimized.

Copy link
Member

commented Apr 20, 2018

I think it's not really an ideal time to restart this whole discussion, after @objmagic was told that the feature was discussed and approved at the developers' meeting, and subsequently spent significant effort on implementation. While it's always fine to raise new technical issues, the objections seem to be old ones, and rather "soft" than technical, which makes it difficult to reach a definite conclusion.

Currently, since open only operates on values, it is compatible with more than one mental model (e.g. as a non-exporting version of include, or as a static operation on environments). The fact that extending it means committing to one of those models (and not the one you prefer) isn't a strong argument in itself. And while people could conceivably be confused by pretty much any change, I haven't encountered anyone who's at all confused by what's proposed here.

@lpw25

This comment has been minimized.

Copy link
Contributor

commented Apr 20, 2018

after @objmagic was told that the feature was discussed and approved at the developers' meeting

That only happened a couple of weeks ago, and I don't think Gabriel accurately reflected the discussion at the meeting, where I believe I already raised these points. I have also raised them with @objmagic in person in the past.

If everyone else is in favour, then as I said, I'm not going to try and stop it, but I thought that the public discussion on this feature should include my objections. (This was at least partly motivated by a conscious attempt to try to move more of these discussions into the public domain).

Currently, since open only operates on values, it is compatible with more than one mental model

The problem with signatures indicates that it is not compatible with both mental models. If open were a non-exporting version of include then it would take a signature when used in signatures. This is the inconsistency that I think will be confusing, it also means that this feature is only a partial solution to the problem of local definitions.

@yallop

This comment has been minimized.

Copy link
Member

commented Apr 20, 2018

after @objmagic was told that the feature was discussed and approved at the developers' meeting

That only happened a couple of weeks ago

No, I told him the day of the meeting (November last year).

I thought that the public discussion on this feature should include my objections. (This was at least partly motivated by a conscious attempt to try to move more of these discussions into the public domain).

I agree that having the objections & discussion publicly available is a good idea. My only real objection is to the timing.

@lpw25

This comment has been minimized.

Copy link
Contributor

commented Apr 20, 2018

My only real objection is to the timing.

Understandable, but to be fair it's not like I'm making new design objections at the last minute, I'm just reiterating publicly design objections that I've already made privately and consistently to all parties involved.

@yallop

This comment has been minimized.

Copy link
Member

commented Apr 20, 2018

I'm just reiterating publicly design objections that I've already made privately and consistently to all parties involved.

Nobody objects to that, of course. But you're also planning to implement and propose a competing design, which is a reasonable thing to do before consensus is reached, but not so much afterwards, unless new issues come to light.

@lpw25

This comment has been minimized.

Copy link
Contributor

commented Apr 20, 2018

But you're also planning to implement and propose a competing design, which is a reasonable thing to do before consensus is reached, but not so much afterwards, unless new issues come to light.

Well, as I said, I'm going to do that even if this feature is already released because it addresses important use cases not handled by this feature. I don't think the overlap is likely to cause users any confusion because, whilst they can be used to solve the same problems, the mechanisms they use are very different from a user's perspective.

@bluddy

This comment has been minimized.

Copy link

commented Apr 20, 2018

Thank you for posting your view, @lpw25. I believe at least some of the support this feature has received (including my own) stems from the fact that we had no idea you had reservations about it, which only stresses how important it is to have a transparent process.

@lpw25 do you think this can get us into similar issues as the recent backwards-incompatible module type change?

And in terms of a meta-discussion:
a. I do appreciate all the effort @objmagic put into this specific PR.
b. I really think we should have either a recording or the notes of developer meetings posted online.
c. A PR such as this should probably be posted as an RFC before implementation begins, so that this discussion could take place publicly without the bias of sunk cost. Only some features seem to be suggested this way, probably because there's no explicit procedure specifying it.

Should we start a discussion on Discourse about procedures for these things? Other languages seem to have very specific procedures in place for RFCs and such.

@lpw25

This comment has been minimized.

Copy link
Contributor

commented Apr 20, 2018

do you think this can get us into similar issues as the recent backwards-incompatible module type change?

My objections are purely on the design side. There are no technical issues with this change, nor any backwards compatibility concerns.

And in terms of a meta-discussion:

PRs are not the place for meta-discussion. Please use one of the other forums (e.g. discuss, caml-list) for such discussions.

@@ -107,11 +107,15 @@ let type_module =
ref ((fun _env _md -> assert false) :
Env.t -> Parsetree.module_expr -> Typedtree.module_expr)

let gen_mod_ident : (Parsetree.module_expr -> Ident.t option) ref = ref (fun _ -> assert false)

This comment has been minimized.

Copy link
@gasche

gasche Apr 21, 2018

Member

Could we have a comment on top of this forward declaration, with the declaration site, as for the others above and below?

@garrigue

This comment has been minimized.

Copy link
Contributor

commented Apr 23, 2018

I don't think the eventually of another way to do something similar should stop us from merging this PR. What it does is clean from a semantical point of view.

A note on private annotations:

  • This was part of the object system a long time ago, but disappeared in 2.00.
  • At the time, @xavierleroy was strongly opposed to this kind of "ad hoc" scoping, but he may have changed since.
  • Since 2.00, private in classes has a different meaning.

For these reasons, I'm not stopping you from making a proposal in that direction (actually at least then I thought it was handy), but there is no guarantee that it would be accepted.

@alainfrisch

This comment has been minimized.

Copy link
Contributor

commented Apr 23, 2018

Just in case somebody wants to work on the "private" approach: I did some quick experiment with them on #682 and #683.

@alainfrisch

This comment has been minimized.

Copy link
Contributor

commented Oct 24, 2018

Notes from today's caml-dev meeting:

  • Let's restrict to open as structure items and local opens (but not as signature items).
  • @trefis will have a look at the implementation
@yallop

This comment has been minimized.

Copy link
Member

commented Oct 27, 2018

Are there any notes of the caml-dev discussion?

I think it would be better to keep the signature version, too, both for symmetry/consistency, and because it makes it possible to give better signatures in some cases. For example, the following module

type t = T
open Set.Make(String)
let union_all : t list -> t = List.fold_left union empty

can be given a signature that matches the implementation very directly:

type t = T
open Set.Make(String)
val union_all : t list -> t

Without extended open in signatures you have two choices:

  1. make up a name for the type:

    type t = T
    type made_up := Set.Make(String).t
    val union_all : made_up list -> made_up
  2. expand all aliases:

    type t = T
    val union_all : Set.Make(String).t list -> Set.Make(String).t

Both of these seem worse than the version with open.

@garrigue

This comment has been minimized.

Copy link
Contributor

commented Oct 27, 2018

@yallop The comment was a bit short indeed.
Actually what was decided was that the signature case would benefit from a clearer syntax, as the use cases are more restricted, and this would be better reached by an independent PR.
See #2122 for this specific feature.
You are of course welcome to comment on that.

@yallop

This comment has been minimized.

Copy link
Member

commented Oct 27, 2018

@garrigue: yes, I've seen #2122, and my comment above makes use of the := syntax from that PR. As the comment also says, := is a worse alternative in some situations, since it requires making up a fresh name where extended open does not.

So, even if #2122 (which I quite like) is merged, I still think it would be better to stick with the original proposal from this PR, and extend open uniformly in both structures and signatures.

@garrigue

This comment has been minimized.

Copy link
Contributor

commented Oct 27, 2018

Sorry, seems I didn't read your post to the end.
There were 3 criticisms for extending open in signatures:

  • there is no symmetry with include, which has a different meaning in signatures
  • allowing to write possible side-effecting structures in signatures is confusing (module type of being seen as a special case, that we would rather remove)
  • #2122 carefully avoids the avoidance problem, which could be particularly confusing in signatures

I'm not sure about your example of of open on paths containing applications.
It is already ok in types, so I'm not sure the possible side-effect would be a problem.
We would still have to restrict it to non-generative functors to avoid the avoidance problem.

@lpw25

This comment has been minimized.

Copy link
Contributor

commented Oct 27, 2018

I'm personally not keen on open with arbitrary module expressions in signatures. Essentially, I don't think that side-effecting expressions should appear in positions where the side-effects won't be executed.

However, this position is complicated by a number of issues with OCaml's current module system. OCaml's notion of a pure module expression is a path, but this is simultaneously too restrictive -- I would like to also include structures whose elements are all defined by aliases and functors whose result is a pure module expression -- and too permissive -- in some positions paths are allowed to include applications of side-effecting functors. I have plans, which were discussed at the meeting, to address the first of these issues by extending paths to a more general notion of pure module expression. The second issue is hard to address backwards-compatibly, especially without tracking side-effects in types, so those will probably remain in an awkward middle ground.

So I think that open in signatures should be restricted to paths -- including applicative functor applications. Once the notion of path is extended this will include structures containing only aliases. I think that will cover all the useful cases for open in signature. It also obeys the basic idea that side-effect expressions should not be allowed in signatures -- modulo the case of functor applications where OCaml is already a bit off.

This is more restrictive than the current patch because it doesn't allow structures that define values, module coercions or generative functor applications, but I think those are both quite dubious in signatures and not really useful for anything.

@yallop

This comment has been minimized.

Copy link
Member

commented Oct 27, 2018

I mostly agree with @lpw25 on this: it's not especially useful for open in signatures to allow completely arbitrary module expressions, but it is useful to support applicative functor application and structures built from type aliases and other supported elements. (Aside: I think it'd be good to choose a name other than "path" for this restricted set of module expressions.)

values, module coercions or generative functor applications, but I think those are both quite dubious in signatures and not really useful for anything.

Transparent ascription (i.e. the operation M <: S that retains type equalities from M) could be useful as the argument of open.

@lpw25

This comment has been minimized.

Copy link
Contributor

commented Oct 27, 2018

Aside: I think it'd be good to choose a name other than "path" for this restricted set of module expressions.

Yes, I've been calling them something like "pure module expressions" when working on the modular implicits specification.

Transparent ascription (i.e. the operation M <: S that retains type equalities from M) could be useful as the argument of open.

Yes, I didn't mention it, but adding transparent ascription to the pure module expressions was also discussed at the meeting. You can see the specification that I was using here, although the transparent ascription is somewhat hidden because it is included as part of module aliases -- other uses of transparent ascription can be simplified away so they are not in the formal spec.

@objmagic

This comment has been minimized.

Copy link
Contributor Author

commented Nov 4, 2018

Thanks for all the feedback and I am sorry for being absent. I was going through some personal life event and have just moved to a new place. I will go back working on this PR as soon as possible

@lpw25

This comment has been minimized.

Copy link
Contributor

commented Nov 4, 2018

@trefis has been looking at updating this PR so that it shares its implementation with the code that was added #1892 and is also used by #2122. I think his branch also takes account of the recent feedback (i.e. it restricts open in signatures to paths -- including functor applications), so you two should probably talk before you start updating your code.

@trefis

This comment has been minimized.

Copy link
Contributor

commented Nov 5, 2018

Indeed, I had started to look at this PR last week and to rebase it on top of mine.
I decided to do that work myself because I assumed there would be a lot of conflicts with my code (and indeed there were), but also some sharing opportunities.
While I was at it, I also included some changes suggested by @alainfrisch during his review, as well as the new restriction that was decided upon during the last meeting two weeks ago.

I think I'm close to being done and I will open a new PR in the next few days (while I squashed many commits together, I make sure to keep you as the author as well as keeping your name in the changelog).

@trefis trefis referenced this pull request Nov 13, 2018

Merged

Extending open #2147

@objmagic

This comment has been minimized.

Copy link
Contributor Author

commented Nov 27, 2018

superseded by #2147

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.