dmd -op -od broken

[dlangBugzillaToGithub]

Timothee Cour (@timotheecour) reported this on 2014-02-09T01:36:51Z

Transferred from https://issues.dlang.org/show_bug.cgi?id=12116

CC List

• Richard (Rikki) Andrew Cattermole
• Andrei Alexandrescu (@andralex)
• Andrej Mitrovic (@AndrejMitrovic)
• Mike Franklin
• Timothee Cour

Description

dmd -c -op -odbuild ../d01/fun.d
#will create  ./build, and write to ./d01/fun.o instead of ./build/d01/fun.o

dmd -c -op -odbuild /abspath/to/fun.d
#will not create ./build and not write any .o file, and not report any error either

Without -op (flat hierarchy), dmd will also produce wrong results:
dmd -c fun.d util/fun.d 
#oops, fun.o overwritten 

ldc has a better way to handle the object flattening:
here, it'll create:
fun.o
util.fun.o
with no ambiguity (since '.' is not a valid character in a module anyways, unlike '_', which was previously proposed).

We should have a flag to support the 'ldc' way.
translate paths a/b/c containing module foo.bar to:
build/a.b.c.o (choice 1, based on path given as input) or build/foo.bar.o (based on module name)
For choice 1, we need to worry about the case of input files given as '../foo/bar.d' but one way that would always work would be to 1st convert to absolute paths every input file.

Note, this is a blocker for incremental compilation in '1 pass' (ie where we don't compile each object file at a time but all in 1 go).

[dlangBugzillaToGithub]

andrej.mitrovich (@AndrejMitrovic) commented on 2014-02-10T04:57:23Z

Related or dupe of: http://d.puremagic.com/issues/show_bug.cgi?id=3541

[dlangBugzillaToGithub]

alphaglosined commented on 2014-07-31T13:09:17Z

Similar again to https://issues.dlang.org/show_bug.cgi?id=13206 but for generated interface files.

[dlangBugzillaToGithub]

timothee.cour2 commented on 2017-01-08T02:19:10Z

still completely broken:

dmd --version
DMD64 D Compiler v2.073.0-devel-a2b772f
rdmd | grep build
rdmd build 20170107

fun.d:
void main(){}

# with absolute path_to_file.d:
dmd -op -od/tmp/d01 /path/bug_12116/fun.d
creates ./fun and /path/bug_12116/fun.o (BUG: /tmp/d01 is ignored)

dmd -c -op -od/tmp/d01 /path/bug_12116/fun.d
creates /path/bug_12116/fun.o (BUG: /tmp/d01 is ignored)

rdmd -c -op -od/tmp/d01 /path/bug_12116/fun.d
creates /tmp/d01/fun.o (BUG: inconsistent with dmd)

# with .. in path_to_file.d:
mkdir temp && cd temp
dmd -op -od/tmp/d01 ../bug_12116/fun.d
creates ./fun and /tmp/bug_12116/fun.o (BUG: expected: something under /tmp/d01)

rdmd -op -od/tmp/d01 ../bug_12116/fun.d
creates /tmp/d01/fun (BUG: inconsistent with dmd) and the .o under */.rdmd-501/* (OK)


# with -c:
dmd -c -op -od/tmp/d01 ../bug_12116/fun.d
/tmp/bug_12116/fun.o (BUG)

rdmd -c -op -od/tmp/d01 ../bug_12116/fun.d
creates /tmp/d01/fun.o (BUG: inconsistent with dmd)


# with no absolute path and not .. in file.d:
dmd -c -op -od/tmp/d01 bug_12116/fun.d
creates /tmp/d01/bug_12116/fun.o (OK)

rdmd -c -op -od/tmp/d01 bug_12116/fun.d
creates /tmp/d01/fun.o (BUG:inconsistent with dmd)

Other argument why -op is much less useful compared to -oq:
* forces you be at the root of module import paths
* if you have multiple module roots, it can't work:

dmd -c -op -od/tmp/d01/ -Iroot1/import -Ipath2/root2/import/ root1/import/std/path.d path2/root2/import/core/stdio.d

=> will create:
/tmp/d01/root1/import/std/path.o
/tmp/d01/path2/root2/import/core/stdio.o

with -oq semantics are a lots easier, predictable, no weird edge cases, and easy to implement:
/tmp/d01/std.path.o
/tmp/d01/core.stdio.o

[dlangBugzillaToGithub]

andrei (@andralex) commented on 2017-01-08T02:32:29Z

(In reply to Timothee Cour from comment #3)
> still completely broken:
> 
> dmd --version
> DMD64 D Compiler v2.073.0-devel-a2b772f
> rdmd | grep build
> rdmd build 20170107
> 
> fun.d:
> void main(){}
> 
> # with absolute path_to_file.d:
> dmd -op -od/tmp/d01 /path/bug_12116/fun.d
> creates ./fun and /path/bug_12116/fun.o (BUG: /tmp/d01 is ignored)
> 
> dmd -c -op -od/tmp/d01 /path/bug_12116/fun.d
> creates /path/bug_12116/fun.o (BUG: /tmp/d01 is ignored)
> 
> rdmd -c -op -od/tmp/d01 /path/bug_12116/fun.d
> creates /tmp/d01/fun.o (BUG: inconsistent with dmd)
> 
> # with .. in path_to_file.d:
> mkdir temp && cd temp
> dmd -op -od/tmp/d01 ../bug_12116/fun.d
> creates ./fun and /tmp/bug_12116/fun.o (BUG: expected: something under
> /tmp/d01)
> 
> rdmd -op -od/tmp/d01 ../bug_12116/fun.d
> creates /tmp/d01/fun (BUG: inconsistent with dmd) and the .o under
> */.rdmd-501/* (OK)
> 
> 
> # with -c:
> dmd -c -op -od/tmp/d01 ../bug_12116/fun.d
> /tmp/bug_12116/fun.o (BUG)
> 
> rdmd -c -op -od/tmp/d01 ../bug_12116/fun.d
> creates /tmp/d01/fun.o (BUG: inconsistent with dmd)
> 
> 
> # with no absolute path and not .. in file.d:
> dmd -c -op -od/tmp/d01 bug_12116/fun.d
> creates /tmp/d01/bug_12116/fun.o (OK)
> 
> rdmd -c -op -od/tmp/d01 bug_12116/fun.d
> creates /tmp/d01/fun.o (BUG:inconsistent with dmd)
> 
> Other argument why -op is much less useful compared to -oq:
> * forces you be at the root of module import paths
> * if you have multiple module roots, it can't work:
> 
> dmd -c -op -od/tmp/d01/ -Iroot1/import -Ipath2/root2/import/
> root1/import/std/path.d path2/root2/import/core/stdio.d
> 
> => will create:
> /tmp/d01/root1/import/std/path.o
> /tmp/d01/path2/root2/import/core/stdio.o
> 
> with -oq semantics are a lots easier, predictable, no weird edge cases, and
> easy to implement:
> /tmp/d01/std.path.o
> /tmp/d01/core.stdio.o

Could you please paste this entire post along with what _should_ be the behavior? Thanks.

[dlangBugzillaToGithub]

timothee.cour2 commented on 2017-01-08T04:05:11Z

Expected behavior:

* dmd -h | grep '\-od' should be accurate (currently: "write object & library files to directory" is not true when source path is absolute or contains ..; but would be under my proposed points below)

* `rdmd -c ` should place .o/.a files in same place as `dmd -c` (not the case currently)

* dmd -op (path with ..)/fun.d should be an error: it's dangerous and unexpected to have something writing BELOW dir when user gives '-od=mydir'. But this won't restrict any use case, see below.

* `dmd` should be equivalent to `dmd -od=.`

* dmd -od=mydir -op /absolute_path/fun.d should write to: mydir/absolute_path/fun.o (eg: -od=/tmp/ => /tmp/absolute_path/fun.o)

this behavior is useful when users don't want to pollute a source repository with .o files, or when they want to keep multiple cached versions of object files (eg for different compilation options): the current behavior (/absolute_path/fun.o) won't allow that.

* Also introduce an optional value for -op=dir (multiple are possible, eg: -op=mydir1 -op=mydir2) to make paths be relative to the first (if any) directory where the absolute path is found:

dmd -c -od=/tmp/ -op=/absolute_path/ -op=import/ /absolute_path/std/fun.d import/core/bar.d source/core/bar.d
/tmp/std/fun.o (thanks to -op=/absolute_path/)
/tmp/core/bar.o (thanks to -op=import/)
/tmp/source/core/bar.o (no match inside -op=)

There is a value to both -oq and -op (with the above fixes):
-op: allows compiling multiple source files with same module name (eg: core.bar), which can be useful
-oq: should be used most of the time (when we don't expect such module name clashes), because all builds are simply under 1 directory, simpler to reason about.

[dlangBugzillaToGithub]

kinke commented on 2017-01-08T12:05:35Z

(In reply to Timothee Cour from comment #5)
> * dmd -od=mydir -op /absolute_path/fun.d should write to:
> mydir/absolute_path/fun.o (eg: -od=/tmp/ => /tmp/absolute_path/fun.o)
> 
> this behavior is useful when users don't want to pollute a source repository
> with .o files, or when they want to keep multiple cached versions of object
> files (eg for different compilation options): the current behavior
> (/absolute_path/fun.o) won't allow that.

Please do not forget about Windows - how would you propose handling `dmd -c -od=mydir -op C:\bla.d D:\bla.d` then? ;)

[dlangBugzillaToGithub]

timothee.cour2 commented on 2017-01-08T12:50:41Z

For windows:

 > Please do not forget about Windows - how would you propose handling `dmd -c -od=mydir -op C:\bla.d D:\bla.d` then? ;)

=> creates:
mydir/C/bla.o
mydir/D/bla.o

but again, as in the proposal user can also write:

dmd -c -od=mydir -op=C:\ C:\bla.d D:\bla.d
=> creates:
mydir/bla.o
mydir/D/bla.o

it's nice and consistent. Another possiblity is to disallow the combo(windows+op+absolute files), but that doesn't seem necessary.

But whatever is decided on windows, hopefully we can at least do what's proposed here on mac/linux.