Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testsuite: stabilize internal type identifiers in expect tests #11318

Merged
merged 1 commit into from Jun 15, 2022

Conversation

Octachron
Copy link
Member

@Octachron Octachron commented Jun 14, 2022

Currently, the testsuite fails after a make install due to a shift in internal type identifiers in a handful of expect tests.

This is a subtle "bug" introduced by #11199 which made the toplevel reads the compiler-libs version of Topdirs when available in the installation directory of the compiler. Unfortunately, reading or not a cmi file modifies the internal type identifiers used by the typechecker. Consequently, the tests in the testsuite that were relying on those internal identifiers had their result dependent on the installation status of the compiler.

This PR fixes this issue by ensuring that expect_test reads topdirs.cmi a fixed number of time, independently of the installation status of the compiler.

@Octachron
Copy link
Member Author

Note that this PR is only short-term fix which has the advantages of only updating expect_test.ml and the affected tests. I expect that we may want to revisit the use of Topcommon.load_topdirs_signature at at a later stage.

Copy link
Member

@gasche gasche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an ugly hack, but it's better than unreliable tests. Approved.

let in_tree_topdirs_cmi =
Filename.concat root @@ Filename.concat "toplevel" "topdirs.cmi" in
ignore (Env.read_signature "Topdirs" in_tree_topdirs_cmi)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand how this works.
I would have expected priority to be given to the in tree file, but apparently priority is given to the installed one.
What happens if the installed file is obsolete?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about this more, I don't understand either. The code that we had before (still around, right after this function is called) is:

if not !Clflags.no_std_include then begin
match !repo_root with
| None -> ()
| Some dir ->
(* If we pass [-repo-root], use the stdlib from inside the
compiler, not the installed one. We use
[Compenv.last_include_dirs] to make sure that the stdlib
directory is the last one. *)
Clflags.no_std_include := true;
Compenv.last_include_dirs := [Filename.concat dir "stdlib"]
end;
Compmisc.init_path ~auto_include:Load_path.no_auto_include ();
Toploop.initialize_toplevel_env ();

Unless I am mistaken, ocamltest always call expect_test with the -repo-root option

let run_expect_once input_file principal log env =
let expect_flags = Sys.safe_getenv "EXPECT_FLAGS" in
let repo_root = "-repo-root " ^ Ocaml_directories.srcdir in
let principal_flag = if principal then "-principal" else "" in
let commandline =
[
Ocaml_commands.ocamlrun_expect_test;
expect_flags;
flags env;
repo_root;
principal_flag;
input_file
] in

so we should always have !Clflags_no_std_include = true, and I don't understand why the installed version of topdirs would ever get read by the environment-initialization code below

Compmisc.init_path ~auto_include:Load_path.no_auto_include ();
Toploop.initialize_toplevel_env ();

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens is that none of the topdirs.cmi files take priority currently: both files are always loaded, and inconsistent interfaces result in a Consistbl.Make(Module_name).Inconsistency("Topdirs", ...) error. This hack make ensure that we are always loading twice topdirs.cmi.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same issue affects the runtop target for launching a development toplevel. Definitely a bug to fix during the 5.0-cycle, but only affecting the development tree.

@gasche
Copy link
Member

gasche commented Jun 15, 2022

Thanks to @garrigue I realize that I also have questions about this PR. I would like to see the situation clarified (maybe there is a nicer fix than @Octachron's proposal?), but a working CI/testsuite is arguably even more important than technical debt in expect_test for now, so I propose to merge now and trust @Octachron to clarify things later.

@gasche gasche merged commit 50b0953 into ocaml:trunk Jun 15, 2022
@Octachron
Copy link
Member Author

This is PR is indeed only intended as a bug-fixing temporary hack.
Now that the alpha release of OCaml 5.0 is out of my todo list, I will be working on a proper fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants