Support other make syntax than GNU (nmake, bsd)

[andreeis]

Needed for repositories like:
https://github.com/freebsd/freebsd.git
https://www.openbsd.org/
https://www.netbsd.org/

[rustedwizard]

upvote this one, hope it will work with nmake

[michoy]

I'm also waiting for nmake support

[andreeis]

@rustedwizard and @michoy, any GitHub repositories based on nmake that you recommend for testing when we get to implement this support?

I can think of a partial hacky way you can use Makefile Tools for nmake even now. I don't have an nmake project handy to verify, but if the build logs are not too crazy different, the extension may be able to work with having "makefile.buildLog" pointing to the output you save from a full clean build of your project. If cd/push/pop/entering/leaving directories is written plainly, there are no active logging silencing directives and compiler commands are outputted, you may even have full functionality working already (except the "live" configure from a fresh up to date "dry-run". the build.log is frozen and needs to be refreshed every time you make a relevant change for IntelliSense: add/remove file, add/remove/change compilation swtiches...).

We don't plan to implement this support soon but if you share with me a full clean build log of your project or a repository to clone from GitHub, I can quickly try and see how much it works already and help you with workarounds if at all possible.

[michoy]

Hi @andreeis, thanks for taking the time to look into this.

The project I'm working on is proprietary, so unfortunately I cannot give you a repository to test on. But I can show you my build log from .vscode\targets.log. The dry-run fails with same same error.

nmake.exe all --print-data-base --no-builtin-variables --no-builtin-rules --question

Microsoft (R) Program Maintenance Utility Version 12.00.21005.1
Copyright (C) Microsoft Corporation.  All rights reserved.

NMAKE : fatal error U1065: invalid option '-'
 
Stop.
 

[andreeis]

@michoy, yes because since we didn't have time to support nmake, we are not sending the right switches, we are not parsing for the right strings in the output, etc... We need to re-evaluate the arguments we send to "nmake", supposing there is even an equivalent for all the arguments that we ended up needing for "make".

To unblock you, you create manually, in the terminal, a full clean build log of your project with the "nmake" command that you use daily. Make sure there is no logging suppressing going on. And share that log output with us.

[michoy]

I'm not allowed to share the complete log. Sorry that I can't be more helpful, hope you figure it out anyways!

[andreeis]

@michoy, We will, for sure. It's not planned to happen yet.
Also, in case you know any other good nmake projects that we can clone from somewhere and which would give us a good build log, that would speed up things.

[ashemedai]

@andreeis I have a vested interest in working on bmake (BSD make) work. What's the easiest to start with, the syntax highlighting?

[andreeis]

@ashemedai, that is wonderful! :)

Not sure what you think of "syntax highlighting", this extension doesn't currently support that for regular make either. Language service support for makefile syntax is even further on our backlog.

What we support now is some level of parsing and understanding of a "make" log to extract from that information that is useful in constructing IntelliSense for C/C++. And so far, it works only with regular "make", no NMAKE, no BSD.

So, if you can help extend the support for BSD, let's find where bmake differs from make in its output and usage.

For "make", the following switches are very important and we need equivalents when supporting bmake: --dry-run, --keep-going, --always-make, --print-directory, --print-data-base, --no-builtin-variables, --no-builtin-rules, --question.

Once you identify the equivalent bmake args, let's see how the output differs so we adapt our parsers. Can you share an example of a build log when using these switches?

One log with --dry-run, --keep-going, --always-make, --print-directory (we extract compilation/linking commands from this output, we parse defines, includes, c/c++ standard, other compiler arguments relevant for IntelliSense and we also resolve paths based on how make tracks all the directories it goes into).

and one log with --print-data-base, --no-builtin-variables, --no-builtin-rules, --question (we extract makefile targets from this output)

[ashemedai]

@andreeis Have some initial data. Just need to generate some make logs.

--dry-run would be equivalent to -N:

Display the commands which would have been executed, but do not actually execute any of them; useful for debugging top-level makefiles without descending into subdirectories.

It could also be -n if you want certain targets to still be executed:

Display the commands that would have been executed, but do not actually execute them unless the target depends on the .MAKE special source (see below) or the command is prefixed with `+'.

--keep-going would be equivalent to -k:

Continue processing after errors are encountered, but only on those targets that do not depend on the target whose creation caused the error.

--print-directory would be equivalent to -w:

Print entering and leaving directory messages, pre and post processing.

--no-builtin-rules would be equivalent to -r:

Do not use the built-in rules specified in the system makefile.

For --print-data-base I would imagine some combination of -d[..] flags to accomplish this functionality, probably -dpv

p: Print debugging information about makefile parsing.
v: Print debugging information about variable assignment.

--question would be equivalent to -q:

Do not execute any commands, but exit 0 if the specified targets are up-to-date and 1, otherwise.

Still looking if there's equivalent functionality to --always-make, --no-builtin-variables

[andreeis]

@ashemedai, excellent initial data. Looking forward to the following findings.

[ashemedai]

@andreeis With regard to the logs, are small projects (to get started) just as valuable as big ones (which probably highlight some additional things)? Can give you that today then.

[ashemedai]

Apologies for the long delay.

Find attached an output with make -N -k -w for cp from the FreeBSD source tree. There is no --always-make functionality.
cp-nop.log

Since one of the use cases listed at the top is for the FreeBSD source tree, I tried to use it with the no built-in rules, but that actually breaks the use of make since it errors out in multiple places due to malformed conditionals, e.g.: make: "/usr/src/share/mk/bsd.cpu.mk" line 9: Malformed conditional (${MACHINE_CPUARCH} == "aarch64")

If you need to extract the target list, I wonder if a make -N -dm might not be better rather than trying to find an exact equivalent of what you asked me. The -dm is for print debugging information about making targets, including modification dates.
cp-targets.log

Contrast to the rough equivalent of --print-data-base, --no-builtin-variables, --no-builtin-rules, --question (make -dpv -r -q), which has various errors:
cp-fatal-error.log.gz

[andreeis]

@ashemedai, cp-nop.log looks promising :). If you set "makefile.dryrunSwitches": ["-N","-k","-w"] then the IntelliSense part should work as with normal "make" syntax (when --always-make is missing). Try that and if anything unexpected happens for larger projects let us know.

The problem with not having a functionality for --always-make is that if the project is not cleaned before being loaded in VSCode, IntelliSense will not know anything about the up-to-date source files. That's so inconvenient but it's by design. Alternatively you can produce yourself a full clean build log of the entire code base and feed that into the extension via "makefile.buildLog" (and make sure to refresh it any time you add/remove a source file or add/remove/edit a compilation switch).
Or maybe there is a way in bsd to mark an up-to-date source file as out-of-date? I imagine this would cause your incremental builds to not be incremental anymore.

For the targets part, unfortunately we don't have a setting that would allow other switches to be passed (-N -dm instead of --nobuiltint... print-database....etc) and also the way we parse for the target names is currently different (we can think whether as we add support for more syntaxes we make the parsing more configurable). But indeed, when we get to supporting BSD syntax, we can use a log like cp-targets.log. "MakeAddChild", "Make_ExpandUse" are makefile targets?

[ashemedai]

@andreeis First off, there was a bug in bmake's -q flag, which is now fixed upstream. So our exchange helped fix a 28 year-old bug. :)

With regard to the --always-make equivalent or otherwise mark out-of-date, looking into it.

My biggest challenge right now is how to get properly testing this. Unfortunately remote-ssh seems blocked for BSD hosts due to microsoft/vscode-remote-release#727
So I need to try to the unofficial VS Code from ports on FreeBSD and see how it works.

[ashemedai]

On the topic of nmake, from Visual Studio 2022:

gmake flag	nmake equivalent
--dry-run	/n
--keep-going	/k
--always-make	/a
--print-directory
--print-data-base
--no-builtin-variables
--no-builtin-rules	/r
--question	/q

For the evaluation of targets and such you probably would want something like nmake /n /nologo /p, see attached file. I used the ping example from a Microsoft network programming book found on https://github.com/esfand/winsock/tree/master/ping

nmake-nop.log