New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add toplevel helpers #10

Closed
dbuenzli opened this Issue Oct 4, 2016 · 5 comments

Comments

Projects
None yet
1 participant
@dbuenzli
Owner

dbuenzli commented Oct 4, 2016

Related to #9.

@dbuenzli dbuenzli added the enhancement label Oct 4, 2016

@dbuenzli

This comment has been minimized.

Show comment
Hide comment
@dbuenzli

dbuenzli Oct 30, 2016

Owner

Quite a few things were added by 7999680 and cccc541. Documented here.

A few things that need to be solved.

  1. For now ambiguity resolution of the stdlib simply select the non-vmthread version of the libraries. We should try to detect if vmhtreads are being used by the toplevel.
  2. In general we need a way to be able to refine manually the load order via an opam x-field but this should be defined in the more general context of #9, since it's simply the ability to redefine the link order.
  3. Whenever packages are looked up recursively we should limit searches to deps: and depopts: field of packages, rather than all the global install base as is done now. Global lookup is unstable w.r.t. switch state (e.g. if a new package is installed that has a toplevel name clash, even though packages are not used toghether; EDIT that would only really be a problem if they implement the same interface though).
  4. For now unfound cmos are simply ignored. We should try to resolve their cmis and if found add their directory to the include paths. This could be a cmi only interface.
  5. Related to 4. we could always try to find cmis for cmos and if existing add their directory to the include path. For now we add the directory where the cmo (or its including cma) to the include path.
  6. We need to check for freshness on each Odig.* toplevel command. The user might have opam installed something new meanwhile and it is a curse to make her restart the toplevel to get an up-to-date view of the package install.

A few things that need to be improved:

  1. Interactive mode to resolve ambiguities for Odig.load (in which search will remain unrestricted).
Owner

dbuenzli commented Oct 30, 2016

Quite a few things were added by 7999680 and cccc541. Documented here.

A few things that need to be solved.

  1. For now ambiguity resolution of the stdlib simply select the non-vmthread version of the libraries. We should try to detect if vmhtreads are being used by the toplevel.
  2. In general we need a way to be able to refine manually the load order via an opam x-field but this should be defined in the more general context of #9, since it's simply the ability to redefine the link order.
  3. Whenever packages are looked up recursively we should limit searches to deps: and depopts: field of packages, rather than all the global install base as is done now. Global lookup is unstable w.r.t. switch state (e.g. if a new package is installed that has a toplevel name clash, even though packages are not used toghether; EDIT that would only really be a problem if they implement the same interface though).
  4. For now unfound cmos are simply ignored. We should try to resolve their cmis and if found add their directory to the include paths. This could be a cmi only interface.
  5. Related to 4. we could always try to find cmis for cmos and if existing add their directory to the include path. For now we add the directory where the cmo (or its including cma) to the include path.
  6. We need to check for freshness on each Odig.* toplevel command. The user might have opam installed something new meanwhile and it is a curse to make her restart the toplevel to get an up-to-date view of the package install.

A few things that need to be improved:

  1. Interactive mode to resolve ambiguities for Odig.load (in which search will remain unrestricted).

dbuenzli added a commit that referenced this issue Mar 3, 2017

Improve Odig toplevel bootstrap (#10).
Thanks to Jeremie Dimino (@diml) for a few hints.

dbuenzli added a commit that referenced this issue Mar 7, 2017

Fix bug in toplevel loader (#10).
Dependency search and topological sort were done with different
dependency functions on the cmo objects leading to inconsistent
resolution maps.
@dbuenzli

This comment has been minimized.

Show comment
Hide comment
@dbuenzli

dbuenzli Jul 17, 2017

Owner

@Drup suggested that since 4.04 it is possible to hook into the cmi lookup which provides a way to intercept requests for a cmi and load the appropriate compilation unit and its dependencies.

Unfortunately this API is not generally available in ocaml it seems however to be available in utop-full.

A prototype implementation of this is to simply call Odig.load in that function before piggybacking on the internal implementation. Example:

> utop-full  -init /dev/null # prevent loading of odig.top otherwise ocamlcommon.cma get loaded                                                               
# #require "odig" (* We are using ocamlfind because odig.top was not read *)
# let () = 
    let load = !Env.Persistent_signature.load in
    Env.Persistent_signature.load := (fun ~unit_name -> Odig.load unit_name; load ~unit_name)

This does however currently interfere badly with utop.

  1. Users of #use odig.top load their own version of Env which prevent the above code of working.
  2. utop doesn't like Odig trying to output to stdout while it tries to complete.
  3. Random failures are being observed which seem to be due to a bad interaction with lwt (they look suspiciously like a double unix load, but it doesn't seem to be the case).

More generally it would also maybe good to try to separate/relayer the compilation object indexing from the larger odig infrastructure which pulls a bit too much deps into the toplevel.

Owner

dbuenzli commented Jul 17, 2017

@Drup suggested that since 4.04 it is possible to hook into the cmi lookup which provides a way to intercept requests for a cmi and load the appropriate compilation unit and its dependencies.

Unfortunately this API is not generally available in ocaml it seems however to be available in utop-full.

A prototype implementation of this is to simply call Odig.load in that function before piggybacking on the internal implementation. Example:

> utop-full  -init /dev/null # prevent loading of odig.top otherwise ocamlcommon.cma get loaded                                                               
# #require "odig" (* We are using ocamlfind because odig.top was not read *)
# let () = 
    let load = !Env.Persistent_signature.load in
    Env.Persistent_signature.load := (fun ~unit_name -> Odig.load unit_name; load ~unit_name)

This does however currently interfere badly with utop.

  1. Users of #use odig.top load their own version of Env which prevent the above code of working.
  2. utop doesn't like Odig trying to output to stdout while it tries to complete.
  3. Random failures are being observed which seem to be due to a bad interaction with lwt (they look suspiciously like a double unix load, but it doesn't seem to be the case).

More generally it would also maybe good to try to separate/relayer the compilation object indexing from the larger odig infrastructure which pulls a bit too much deps into the toplevel.

@dbuenzli

This comment has been minimized.

Show comment
Hide comment
@dbuenzli

dbuenzli Jul 17, 2017

Owner

Users of #use odig.top load their own version of Env which prevent the above code of working.

To be precise ocamlcommon.cma has to be loaded in the expunged utop but not in utop-full.

Owner

dbuenzli commented Jul 17, 2017

Users of #use odig.top load their own version of Env which prevent the above code of working.

To be precise ocamlcommon.cma has to be loaded in the expunged utop but not in utop-full.

@dbuenzli

This comment has been minimized.

Show comment
Hide comment
@dbuenzli

dbuenzli Jul 18, 2017

Owner

To be precise ocamlcommon.cma has to be loaded in the expunged utop but not in utop-full.

According to Jérémie there's currently no good way of detecting this.

Owner

dbuenzli commented Jul 18, 2017

To be precise ocamlcommon.cma has to be loaded in the expunged utop but not in utop-full.

According to Jérémie there's currently no good way of detecting this.

@dbuenzli

This comment has been minimized.

Show comment
Hide comment
@dbuenzli

dbuenzli Jun 13, 2018

Owner

odig's toplevel loaders are deprecated. Some of the problems mentioned above are resolved by omod in a more viable fashion.

Owner

dbuenzli commented Jun 13, 2018

odig's toplevel loaders are deprecated. Some of the problems mentioned above are resolved by omod in a more viable fashion.

@dbuenzli dbuenzli closed this Jun 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment