Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge utils/Makefile into the root Makefile #11268

Merged
merged 9 commits into from
Sep 9, 2022

Conversation

shindere
Copy link
Contributor

The main thing this PR does is to modify the configure and build systems
so that ocamltest/ocamltest_config.ml and utils/config.ml (renamed to
utils/config_fragment.ml by this PR) are generated as part of the
configure stager, rather than during the build.

This PR consists in several commits which are best reviewed one after the
other.

Two questions regarding the configure module:

  1. Wouldn't it be a good moment to remove bytecode_c_compiler and the like, or do those need to be deprecated first?

  2. The code that computes mkexe, mkdll and mkmaindll is executed at runtime. Is that a feature, i.e. a behaviour that needs to be preserved as it it, or could we save a test and a few computations and generate the correct code at configure time?

@shindere
Copy link
Contributor Author

Is it okay to stop generating utils/config_boot.mli and
utils/config_main.mli as the PR does?

These files look not so useful since the corresponding implementations
won't actually be compiled as such but rather copied to config.ml
which will then be compiled and checked against config.mli.

@shindere shindere force-pushed the merge-utils-makefile branch 3 times, most recently from 4423681 to 8393bb7 Compare May 20, 2022 10:05
@dra27
Copy link
Member

dra27 commented May 24, 2022

Is it okay to stop generating utils/config_boot.mli and utils/config_main.mli as the PR does?

These files look not so useful since the corresponding implementations won't actually be compiled as such but rather copied to config.ml which will then be compiled and checked against config.mli.

These files are there so that if you add, say, Config.my_amazing_new_feature_flag to utils/config.mli and utils/config.mlp, then during the build you get:

File "/home/dra/ocaml/utils/config_boot.ml", line 1:
Error: The implementation utils/config_boot.ml
       does not match the interface utils/config_boot.cmi:
       The value `my_amazing_new_feature_flag' is required but not provided
       File "/home/dra/ocaml/utils/config_boot.mli", line 262, characters 0-17:
         Expected declaration

Otherwise you only find this out when make bootstrap is next run (hopefully in CI, of course, but nonetheless further down the line).

@gasche
Copy link
Member

gasche commented May 24, 2022

These files are there so that [..]

This reads like a design aspect that could be documented by a comment somewhere in the configuration / build system. (Maybe the PR could include such a comment?)

@dra27
Copy link
Member

dra27 commented May 24, 2022

These files are there so that [..]

This reads like a design aspect that could be documented by a comment somewhere in the configuration / build system. (Maybe the PR could include such a comment?)

Guilty as charged! It could/should indeed be documented with the definition of ALL_CONFIG_CMO in compilerlibs/Makefile.compilerlibs.

@shindere
Copy link
Contributor Author

shindere commented May 25, 2022 via email

@shindere shindere force-pushed the merge-utils-makefile branch 4 times, most recently from 740d98f to 1e83077 Compare May 31, 2022 14:06
@shindere
Copy link
Contributor Author

shindere commented May 31, 2022 via email

@shindere shindere force-pushed the merge-utils-makefile branch from 1e83077 to 5cc0e82 Compare June 27, 2022 06:27
@shindere
Copy link
Contributor Author

Although there was no conflict with trunk it felt a good idea to rebase.

@shindere shindere force-pushed the merge-utils-makefile branch from 5cc0e82 to 20b315a Compare July 1, 2022 14:17
@shindere
Copy link
Contributor Author

shindere commented Jul 1, 2022

Just rebased on latest trunk to resolve merge conflicts

Copy link
Member

@dra27 dra27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is really great to see the back of those sed expressions, and sorry it's taken me a while to review this. I've checked the various files generated commit-by-commit to try to try to catch the effect of all the moving parts.

A few of the comments are suggestions for clarifications or tweaks, but they're mostly optional: in particular, I think possibly a little too much has been removed from ocamltest/Makefile into configure.ac and there's a slightly muddying of the proverbial waters because there's some extant win32unix/unix code in ocamltest which you might want to excise at the same time.

@@ -97,7 +97,7 @@ O=@OBJEXT@
EXT_OBJ=.@OBJEXT@

### How to tell the C compiler to output an object file
OUTPUTOBJ=@outputobj@
OUTPUTOBJ=@outputobj@ $(EMPTY)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The space before $(EMPTY) here needs to go.

configure.ac Outdated
[PACKLD="link -lib -nologo $machine -out:"],
[PACKLD="$DIRECT_LD -r$PACKLD_FLAGS -o \$(EMPTY)"])],
[PACKLD="$PARTIALLD -o \$(EMPTY)"])
[pacld_ocaml="link -lib -nologo $machine -out:"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: this should be packld_ocaml (missing k) - although suggestion earlier that this separate variable isn't needed.

configure.ac Outdated
AC_SUBST([PACKLD])
AC_SUBST([packld_ocaml])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this variable is needed - $(EMPTY) is used because some linkers must have a space after -o. There's no harm for the Microsoft case having configure set PACKLD="link -lib -nologo $machine -out:" (no space at the end) and then have in Makefile.config.in PACKLD=@PACKLD@$(EMPTY)? @PACKLD@ can then be used directly in utils/config_fragment.ml.in

configure.ac Outdated
@@ -591,19 +652,18 @@ AS_IF(

## Determine which flags to use for the C compiler

outputobj='-o '
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this a little inconsistent: in the AS_CASE below, we very clearly set all outputobj, warn_error_flag, and cc_warnings for various different configurations.

If the default for outputobj goes here shouldn't the defaults for warn_error_flag and cc_warnings go here too?

configure.ac Outdated

# Absolute path to the toplevel source directory

# A sentinelle character (X) is added ad the end of the directory name
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two typos: at the end of the directory name and sentinel, rather than sentinelle. I would be more explicit on the exact reason for the sentinel: it's there because $(..) strips trailing \n characters (you could reference Eric's marvellous reply in https://lists.gnu.org/archive/html/autoconf/2019-07/msg00002.html !)

@@ -1,127 +0,0 @@
#**************************************************************************
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SUBST and SUBST_STRING macros in Makefile.common can also be deleted along with this file!

@@ -97,7 +97,7 @@ O=@OBJEXT@
EXT_OBJ=.@OBJEXT@

### How to tell the C compiler to output an object file
OUTPUTOBJ=@outputobj@
OUTPUTOBJ=@outputobj@ $(EMPTY)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The space before $(EMPTY) shouldn't be here (same in OUTPUTEXE on L192)

@@ -97,7 +97,7 @@ O=@OBJEXT@
EXT_OBJ=.@OBJEXT@

### How to tell the C compiler to output an object file
OUTPUTOBJ=@outputobj@
OUTPUTOBJ=@outputobj@ $(EMPTY)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The space before $(EMPTY) shouldn't be here (same in OUTPUTEXE on L192)


let systhreads = %%systhreads%%
let systhreads = @lib_systhreads@
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@systhread_support@ could be used, and the extra variable lib_systhreads eliminated.

@@ -2138,18 +2221,20 @@ AC_CONFIG_COMMANDS_PRE([
AS_IF([test -n "${LDFLAGS}"],
[for flag in ${LDFLAGS}; do
mkdll_ldflags="${mkdll_ldflags} ${mkexe_ldflags_prefix}${flag}"
done])
done
mkdll_ldflags_exp="$mkdll_ldflags"])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a mkexe_ldflags_exp is needed here, too (see later comment in utils/config_fragment.ml.in. It would want the oc_ldflags part below on L2231 as well but then not the oc_dll_ldflags bit on L2237

@shindere shindere force-pushed the merge-utils-makefile branch 5 times, most recently from 06799f4 to 384389a Compare July 19, 2022 06:40
@shindere shindere force-pushed the merge-utils-makefile branch 5 times, most recently from bceefe0 to e8b2315 Compare August 24, 2022 10:10
@shindere shindere force-pushed the merge-utils-makefile branch 2 times, most recently from 369e685 to d776671 Compare August 31, 2022 15:17
Add the $(EMPTY) space-preserving suffix to OUTPUTOBJ, OUTPUTEXE and PACKLD
in Makefile.config.in rather than in the configure script.

This makes it possible to substitute these variables in OCaml files
during the configure stage.

This commit also makes the mkdll_ldflags_exp variable available to
the build system and improves its computation to ensure it does not
contain superfluous spaces.

Finally, this commit instanciates CFLAGS and CPPFLAGS during the configure
stage, so that they can be passed to the OCaml configuraiton module.
It becomes thus impossible to override CFLAGS and CPPFLAGS
at build time.
This will be used to generate the OCaml configuration module as part of
the configure stage, rather than generating it during the build.
This is the fragment of the configure module which is presently generated
at build time from a template file but will ultimately be generated
at configure time.

Also make .gitignore more specific and explicit about what needs
to be ignored.
This commit also improves the way mkdll_ldflags_exp is computed so that it
contains no spurious spaces.
@shindere shindere force-pushed the merge-utils-makefile branch from d776671 to 35c93d2 Compare September 7, 2022 09:31
@shindere
Copy link
Contributor Author

shindere commented Sep 7, 2022

I just rebased this on latest trunk, after the merge of #11420.

Rebasing required to resolve three merge conflicts in configure.

That looks as another argument not to have configure versionned, except for
releases: it would simplify those rebases that modify configure.ac
or another of the M4 files.

This should be taken into account in #11343

@gasche
Copy link
Member

gasche commented Sep 7, 2022

Note: One simple way to resolve a conflict on a generated file is to re-generate it.

@shindere
Copy link
Contributor Author

shindere commented Sep 7, 2022 via email

Copy link
Member

@dra27 dra27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Precheck seems happy. With some slight adaptations, I persuaded this PR to produce utils/config.generated.ml files for the presently disabled Cygwin/MSVC ports and compared these with utils.config.ml from 4.14 (along with mingw).

What this revealed is some breakage from #10413 which we didn't pick up on (it affects mingw-w64 somewhat innocuously, but the faults break MSVC and Cygwin).

However, this PR doesn't make those problems on trunk/5.0 any worse, so I propose as a penance to open a PR to fix those to be merged after this one.

Nice work, @shindere! I'll aim not to shed a tear for the loss of a 50 line sed command :)

@dra27 dra27 merged commit 72fc081 into ocaml:trunk Sep 9, 2022
@shindere shindere deleted the merge-utils-makefile branch September 9, 2022 14:52
@shindere
Copy link
Contributor Author

shindere commented Sep 9, 2022 via email

@shindere
Copy link
Contributor Author

shindere commented Sep 9, 2022 via email

@Octachron
Copy link
Member

This PR broke bootstrapping: there are some dependencies on config_main that remain even when IN_COREBOOT_CYCLE is true which breaks the coreboot target.

@dra27
Copy link
Member

dra27 commented Sep 21, 2022

Oops - it's not to do with the IN_COREBOOT_CYCLE stuff, it's simply that utils/config.generated.ml is erased during partialclean where it should now only be erased during distclean. PR incoming.

@shindere
Copy link
Contributor Author

shindere commented Oct 11, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants