Compilation of code outside of the Red/ path fails under Windows #366

PeterWAWood · 2013-01-04T00:57:56Z

When compiling code outside of the Red/ path under Windows, the compilation from Red/System to machine code fails.

Code

Red []
print 1

Output

>> do/args %red.r %../../Code/Red/test.red
Script: "Red command-line front-end" (none)
Script: "Red compiler" (none)
Script: "Red/System compiler" (none)
Script: "REBOL code profiling tool" (none)
Script: "REBOL 3 Forward Compatibility Functions" (23-Feb-2011)
Script: "Red/System integer! to binary! conversion library" (none)
Script: "Red/System IEEE-754 library" (none)
Script: "Red/System struct! datatype replacement library" (none)
Script: "secure-clean-path" (19-Sep-2002)
Script: "Red/System linker" (none)
Script: "Red/System code emitter" (none)
Script: "Red/System code emitter base object" (none)

-= Red Compiler =-
Compiling ../../Code/Red/test.red ...

...compilation time:     79 ms
Compiling to native code...

Script: "Red/System IA-32 code emitter" (none)
Script: "Red/System PE/COFF format emitter" (none)
*** Red/System Compiler Internal Error: Access Error : Cannot open /c/program files (x86)/re
bol/view/test.exe
*** Where: build
*** Near:  [write/binary/direct file job/buffer
if find
]

The text was updated successfully, but these errors were encountered:

dockimbel · 2013-01-04T22:38:27Z

It seems the problem is caused by your test.exe file been locked, so it cannot be overwritten (it was probably running already). Can you try it again?

PeterWAWood · 2013-01-04T23:24:04Z

I tried again and checked that test.exe was not running. I guess the problem is the Red/System compiler trying to write to the program files directory. But why does it do that? I'll show the full console log:

REBOL/View 2.7.8.3.1 1-Jan-2011
Copyright 2000-2011 REBOL Technologies.  All rights reserved.
REBOL is a trademark of REBOL Technologies. WWW.REBOL.COM


Type desktop to start the Viewtop.
>> change-dir %/e/Languages/Red
== %/e/Languages/Red/
>> do/args %red.r %../../Code/Red/test.red
Script: "Red command-line front-end" (none)
Script: "Red compiler" (none)
Script: "Red/System compiler" (none)
Script: "REBOL code profiling tool" (none)
Script: "REBOL 3 Forward Compatibility Functions" (23-Feb-2011)
Script: "Red/System integer! to binary! conversion library" (none)
Script: "Red/System IEEE-754 library" (none)
Script: "Red/System struct! datatype replacement library" (none)
Script: "secure-clean-path" (19-Sep-2002)
Script: "Red/System linker" (none)
Script: "Red/System code emitter" (none)
Script: "Red/System code emitter base object" (none)

-= Red Compiler =-
Compiling ../../Code/Red/test.red ...

...compilation time:     94 ms

Compiling to native code...

Script: "Red/System IA-32 code emitter" (none)
Script: "Red/System PE/COFF format emitter" (none)
*** Red/System Compiler Internal Error: Access Error : Cannot open /c/program files (x86)/re
bol/view/test.exe
*** Where: build
*** Near:  [write/binary/direct file job/buffer
if find
]

>> pwd
== %/e/Languages/Red/red-system/
>>

dockimbel · 2013-01-05T10:28:41Z

Thanks, I will investigate that further.

dockimbel · 2013-01-06T11:50:43Z

The error message is caused by Windows forbidding to write to ProgramFiles folder. Beyond that, I cannot reproduce your problem:

REBOL/View 2.7.8.3.1 1-Jan-2011
Copyright 2000-2011 REBOL Technologies.  All rights reserved.
REBOL is a trademark of REBOL Technologies. WWW.REBOL.COM


Type desktop to start the Viewtop.
>> change-dir %//dev/Red/
== %/C/dev/Red/
>> do/args %red.r "%../../tmp/tests/inc.red"
Script: "Red command-line front-end" (none)
Script: "Red compiler" (none)
Script: "Red/System compiler" (none)
Script: "REBOL code profiling tool" (none)
Script: "REBOL 3 Forward Compatibility Functions" (23-Feb-2011)
Script: "Red/System integer! to binary! conversion library" (none)
Script: "Red/System IEEE-754 library" (none)
Script: "Red/System struct! datatype replacement library" (none)
Script: "secure-clean-path" (19-Sep-2002)
Script: "Red/System linker" (none)
Script: "Red/System code emitter" (none)
Script: "Red/System code emitter base object" (none)

-= Red Compiler =-
Compiling ../../tmp/tests/inc.red ...

...compilation time:     97 ms

Compiling to native code...

Script: "Red/System IA-32 code emitter" (none)
Script: "Red/System PE/COFF format emitter" (none)
...compilation time:     4265 ms
...linking time:         125 ms
...output file size:     101376 bytes

You would need to put some PROBEs in %rsc.r and %compiler.r to see where Red/System does select such target path.

PeterWAWood · 2013-01-06T13:50:40Z

From a quick look at the source and a quick test, I'd guess the issue is line 101 in red.r:

opts/build-prefix: system/options/path

I'm guessing that you are running Rebol from the Red directory and so don't hit the problem.

The program compiles if I specify the output file. This console session

Type desktop to start the Viewtop.
>> systems/options/path
** Script Error: systems has no value
** Near: systems/options/path
>> system/options/path
== %/c/program%20files%20%28x86%29/rebol/view/
>> change-dir %/e/Languages/Red
== %/e/Languages/Red/
>> do/args %red.r "-o test %../../Code/Red/test.red"
Script: "Red command-line front-end" (none)
Script: "Red compiler" (none)
Script: "Red/System compiler" (none)
Script: "REBOL code profiling tool" (none)
Script: "REBOL 3 Forward Compatibility Functions" (23-Feb-2011)
Script: "Red/System integer! to binary! conversion library" (none)
Script: "Red/System IEEE-754 library" (none)
Script: "Red/System struct! datatype replacement library" (none)
Script: "secure-clean-path" (19-Sep-2002)
Script: "Red/System linker" (none)
Script: "Red/System code emitter" (none)
Script: "Red/System code emitter base object" (none)

-= Red Compiler =-
Compiling ../../Code/Red/test.red ...

...compilation time:     94 ms

Compiling to native code...

Script: "Red/System IA-32 code emitter" (none)
Script: "Red/System PE/COFF format emitter" (none)
...compilation time:     4734 ms
...linking time:         79 ms
...output file size:     101376 bytes
>> do/args %red.r %../../Code/Red/test.red
Script: "Red command-line front-end" (none)

-= Red Compiler =-
Compiling ../../Code/Red/test.red ...

...compilation time:     93 ms

Compiling to native code...

Script: "Red/System IA-32 code emitter" (none)
Script: "Red/System PE/COFF format emitter" (none)
*** Red/System Compiler Internal Error: Access Error : Cannot open /c/program files (x86)/re
bol/view/test.exe
*** Where: build
*** Near:  [write/binary/direct file job/buffer
if find
]

>

PeterWAWood · 2013-01-06T13:53:11Z

Sorry I clicked the wrong button :-(

If the fix is tricky, it's no problem for me to copy REBOL into the Red directory.

dockimbel · 2013-01-06T14:33:45Z

No, I run REBOL from %/c/dev/SDK/tools/.

dockimbel · 2013-01-14T20:04:49Z

Could you please re-test this issue with the last fixes from fix-issue-277 branch to see if those fixes are efficient for this specific issue?

PeterWAWood · 2013-01-15T02:49:48Z

I get this error on the fix-issue-277 branch:

>> pwd
== %/e/Languages/Red/
>> do/args %red.r %../../Code/Red/test.red
Script: "Red command-line front-end" (none)
*** Driver Internal Error: Script Error : .. has no value
*** Where: fail
*** Near:  [forall srcs [
if slash <> first srcs/1 [
srcs/1: clean-path join system/options/path srcs/1
]
unless exists? srcs/1 [
fail ["Cannot access source file:" src]
]
]
reduce [srcs opts]
]

Works on OS X

dockimbel · 2013-01-15T08:25:03Z

I have fixed this internal error. Now you should have a proper "Cannot access source file" error message.

PeterWAWood · 2013-01-15T08:51:38Z

Yes I now get the "Cannot access source file" error:

>> change-dir %/e/Languages/Red
== %/e/Languages/Red/
>> do/args %red.r %../../Code/Red/test.red
Script: "Red command-line front-end" (none)
Script: "Red compiler" (none)
Script: "Red/System compiler" (none)
Script: "REBOL code profiling tool" (none)
Script: "REBOL 3 Forward Compatibility Functions" (23-Feb-2011)
Script: "Red/System integer! to binary! conversion library" (none)
Script: "Red/System IEEE-754 library" (none)
Script: "Red/System struct! datatype replacement library" (none)
Script: "secure-clean-path" (19-Sep-2002)
Script: "Red/System linker" (none)
Script: "Red/System code emitter" (none)
Script: "Red/System code emitter base object" (none)
Cannot access source file: /c/program files (x86)/Code/Red/test.red

The compiler is now using the directory of the REBOL executable as its base directory, previously it could find the source but was trying to write the Red executable to the directory of the REBOL executable.

There is a simple workaround which is to use the fullpath of the source. It then demonstrates the issue with the Red executable:

>> do/args %red.r %/e/Code/Red/test.red
Script: "Red command-line front-end" (none)

-= Red Compiler =-
Compiling /e/Code/Red/test.red ...

...compilation time:     110 ms

Compiling to native code...

Script: "Red/System IA-32 code emitter" (none)
Script: "Red/System PE/COFF format emitter" (none)
*** Red/System Compiler Internal Error: Access Error : Cannot open /c/program files (x86)/re
bol/view/test.exe

Again, there is a simple workaround to specify the location of the executable:

>> do/args %red.r "-o /e/Code/Red/test /e/Code/Red/test.red"
Script: "Red command-line front-end" (none)

-= Red Compiler =-
Compiling /e/Code/Red/test.red ...

...compilation time:     93 ms

Compiling to native code...

Script: "Red/System IA-32 code emitter" (none)
Script: "Red/System PE/COFF format emitter" (none)
...compilation time:     4875 ms
...linking time:         63 ms
...output file size:     103936 bytes

I can happily live with the workarounds at this stage.

earl · 2013-01-15T16:27:30Z

If I understand correctly, the remaining problem is now due to differences between running the compiler via an OS command-line (rebol path/to/red.r path/to/script.red) vs running the compiler via the Rebol console (do/args %path/to/red.r %path/to/script.red).

dockimbel · 2013-01-15T17:27:34Z

Basically yes. But my own tests seem to show that some cases are still broken, I need to do more tests after solving this "command-line vs console" paths mismatching.

earl · 2013-01-15T18:16:22Z

If we can't come up with a safe heuristic to detect do/args vs external calls, maybe it would be better to just use a wrapper to execute the compiler from the Rebol console. Basically something like the following:

run-script: func [script [file!] args /local p r] [
    p: copy system/options/path
    system/options/path: what-dir
    r: do/args script args
    system/options/path: p
    r
]

dockimbel · 2013-01-15T18:43:16Z

It seems that system/options/script is set when run from command-line, but not from console.

dockimbel · 2013-01-16T10:30:44Z

Peter, could you give it a new try with the latest commit from fix-issue-277 branch please?

PeterWAWood · 2013-01-16T12:35:50Z

I've tested with the latest commit on the fix-issue-277 branch butI'm afraid that I'm still getting the cannot access source file error:

>> do/args %red.r %../../Code/Red/test.red
Script: "Red command-line front-end" (none)
Cannot access source file: /c/program files (x86)/Code/Red/test.red
>> system/options/script
== none
>> system/script/path
== %/e/Languages/Red/

If I change lines 101 & 121 of %red.r to use system/script/path then it works just fine. I also tested compiling the same test program from the command line under both Ubuntu and OS X, again it worked. I ran the complete test suite successfully with these patches in all three environments, Windows from the REBOL/View console, the others from the command line.

dockimbel · 2013-01-16T12:38:20Z

Thanks Peter, I will test your fix here, that might just be the right trick to finish this series of path fixes.

Actually, any [system/options/script system/script/path] at line 101, is probably what is required to make it work in all use cases.

…nder Windows)

PeterWAWood · 2013-01-16T13:40:54Z

Nenad

Your change also needs to be applied to line 121:

srcs/1: clean-path join system/options/path srcs/1  ;-- add working dir path

``

dockimbel · 2013-01-16T13:43:21Z

I saw that (too late), so I have pushed a new commit just now for that. Thanks for noticing.

PeterWAWood · 2013-01-16T14:00:10Z

I'm still getting the cannot access source file error. The 'ANY isn't working as expected because system/options/path is set to the dir of the REBOL executable (It seemed to be 'NONE before but I guess that must have been some unintended side effect). I've added some debug prints to show the effect of the inserted call of 'ANY:

Type desktop to start the Viewtop.
>> change-dir %/e/Languages/Red
== %/e/Languages/Red/
>> system/options/path
== %/c/program%20files%20%28x86%29/rebol/view/
>> do/args %red.r %../../Code/Red/test.red
Script: "Red command-line front-end" (none)
Script: "Red compiler" (none)
Script: "Red/System compiler" (none)
Script: "REBOL code profiling tool" (none)
Script: "REBOL 3 Forward Compatibility Functions" (23-Feb-2011)
Script: "Red/System integer! to binary! conversion library" (none)
Script: "Red/System IEEE-754 library" (none)
Script: "Red/System struct! datatype replacement library" (none)
Script: "secure-clean-path" (19-Sep-2002)
Script: "Red/System linker" (none)
Script: "Red/System code emitter" (none)
Script: "Red/System code emitter base object" (none)
system/options/path /c/program files (x86)/rebol/view/
system/script/path /e/Languages/Red/
base-path /c/program files (x86)/rebol/view/
Cannot access source file: /c/program files (x86)/Code/Red/test.red

I always seem to have problems with file paths in REBOL, I hope I'm not a jinx on them.

earl · 2013-01-16T18:50:15Z

Hmm, I don't think that system/options/script being set is a totally safe heuristic, but if you want to use that, line 99 should probably rather be:

either none? system/options/script [system/options/path] [system/script/path]

(I think the current any [system/options/path system/script/path] should do nothing, as system/options/path should always be set.)

dockimbel · 2013-01-16T21:09:15Z

Right Andreas, that shows you how confused I am with all the REBOL system paths and their confusing names...

dockimbel · 2013-01-16T21:10:26Z

Andreas, have you find use-cases, where system/options/script cannot be used to distinguish command-line from console script launching method?

dockimbel · 2013-01-16T21:41:26Z

Andreas, with your proposed fixed, the following use-case doesn't pass anymore:

>> do/args %Red/red.r %Red/red/tests/demo.red
Cannot access source file: /root/Bureau/Red/Red/red/tests/demo.red

It still passes when run from console with:

$ rebol -qws Red/red.r %Red/red/tests/demo.red

EDIT: sorry, I've inverted the test cases results, it's now in the right order, the one failing is the one from the console.

dockimbel · 2013-01-17T00:24:13Z

Ok, it seems the only solution to overcome the REBOL internal bad working dir issue is to re-sync with OS path in order to compensate for REBOL's internal "cd" when DOing a script. I will test that solution tomorrow.

PeterWAWood · 2013-01-17T02:49:27Z

Changing line 99 of red.r as below fixes the problem for me. I've tested under Windows 7 (REBOL/View console). Ubuntu (command line) and OS X (command line). Also run-all.r continues to work after the change.

from

base-path: any [system/options/path system/script/path]

to

base-path: either any [
    system/options/path = none
    system/options/path = first split-path system/options/boot 
][
    system/script/path
][
    system/options/path 
]

It's a little verbose. Testing system/options/path = none may not be strictly necessary.

earl · 2013-01-17T02:57:59Z

Hah! R2 has system/script/parent/path (which I never noticed before) which, from a few first tests, seems to be pretty reliable and exactly what we need: the current working directory a script was started from.

I did the following tests using R2/Core 2.7.8 on Linux:

cwd	shell	rebol
/tmp/foo/red	rebol2 -qws red.r red/tests/demo.red	-
/tmp/foo	rebol2 -qws red/red.r red/red/tests/demo.red	-
/tmp/foo/red	rebol2	do/args %red.r %red/tests/demo.red
/tmp/foo	rebol2	change-dir %red/ do/args %red.r %red/tests/demo.red
/tmp	rebol2	change-dir %foo/ do/args %red/red.r %red/red/tests/demo.red

So the fix would just be to use base-path: system/script/parent/path. Let's hope system/script/parent/path works on Windows/OSX as well.

PeterWAWood · 2013-01-17T03:07:06Z

The change that I suggested works in the REBOL console under OS X and Liunx provided that the pwd isn't changed by change-dir in the REBOL console session ... so it doesn't really work :-(

I'll test Andreas fix.

PeterWAWood · 2013-01-17T03:41:19Z

I've tested Andreas's monumental discovery and resultant fix and found that it works in the following situations:

Windows REBOL console
OS X & Ubuntu command line and REBOL Console (with and without using change-dir)

The change has no adverse affect on run-all.r

earl · 2013-01-17T03:44:59Z

As it really seems to work in all cases tested so far, I tentatively pushed a system/script/parent/path commit so that others can try directly using the fix-issue-277 branch.

dockimbel · 2013-01-17T11:17:18Z

Andreas, your fix works well with all my test cases. Great job! Thank you very much for your help!

Fixes following issues: #251 (Red doesn't find source in working directory) #252 (#system cannot find #include) #277 (Include system doesn't handle well files with same name in different directories) #366 (Compilation of code outside of the Red/ path fails under Windows) #381 (#system-global doesn't detect equal #include paths)

PeterWAWood closed this as completed Jan 6, 2013

PeterWAWood reopened this Jan 6, 2013

dockimbel added a commit that referenced this issue Jan 16, 2013

FIX: issue #366 (Compilation of code outside of the Red/ path fails u…

923abbc

…nder Windows)

dockimbel closed this as completed Jan 16, 2013

PeterWAWood reopened this Jan 16, 2013

dockimbel added a commit that referenced this issue Jan 16, 2013

FIX: extended last fix for #366.

59f7ec8

earl added a commit that referenced this issue Jan 17, 2013

FIX: use system/script/parent/path to tackle #366

3bcf184

dockimbel closed this as completed Jan 17, 2013

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compilation of code outside of the Red/ path fails under Windows #366

Compilation of code outside of the Red/ path fails under Windows #366

PeterWAWood commented Jan 4, 2013

dockimbel commented Jan 4, 2013

PeterWAWood commented Jan 4, 2013

dockimbel commented Jan 5, 2013

dockimbel commented Jan 6, 2013

PeterWAWood commented Jan 6, 2013

PeterWAWood commented Jan 6, 2013

dockimbel commented Jan 6, 2013

dockimbel commented Jan 14, 2013

PeterWAWood commented Jan 15, 2013

dockimbel commented Jan 15, 2013

PeterWAWood commented Jan 15, 2013

earl commented Jan 15, 2013

dockimbel commented Jan 15, 2013

earl commented Jan 15, 2013

dockimbel commented Jan 15, 2013

dockimbel commented Jan 16, 2013

PeterWAWood commented Jan 16, 2013

dockimbel commented Jan 16, 2013

PeterWAWood commented Jan 16, 2013

dockimbel commented Jan 16, 2013

PeterWAWood commented Jan 16, 2013

earl commented Jan 16, 2013

dockimbel commented Jan 16, 2013

dockimbel commented Jan 16, 2013

dockimbel commented Jan 16, 2013

dockimbel commented Jan 17, 2013

PeterWAWood commented Jan 17, 2013

earl commented Jan 17, 2013

PeterWAWood commented Jan 17, 2013

PeterWAWood commented Jan 17, 2013

earl commented Jan 17, 2013

dockimbel commented Jan 17, 2013

Compilation of code outside of the Red/ path fails under Windows #366

Compilation of code outside of the Red/ path fails under Windows #366

Comments

PeterWAWood commented Jan 4, 2013

dockimbel commented Jan 4, 2013

PeterWAWood commented Jan 4, 2013

dockimbel commented Jan 5, 2013

dockimbel commented Jan 6, 2013

PeterWAWood commented Jan 6, 2013

PeterWAWood commented Jan 6, 2013

dockimbel commented Jan 6, 2013

dockimbel commented Jan 14, 2013

PeterWAWood commented Jan 15, 2013

dockimbel commented Jan 15, 2013

PeterWAWood commented Jan 15, 2013

earl commented Jan 15, 2013

dockimbel commented Jan 15, 2013

earl commented Jan 15, 2013

dockimbel commented Jan 15, 2013

dockimbel commented Jan 16, 2013

PeterWAWood commented Jan 16, 2013

dockimbel commented Jan 16, 2013

PeterWAWood commented Jan 16, 2013

dockimbel commented Jan 16, 2013

PeterWAWood commented Jan 16, 2013

earl commented Jan 16, 2013

dockimbel commented Jan 16, 2013

dockimbel commented Jan 16, 2013

dockimbel commented Jan 16, 2013

dockimbel commented Jan 17, 2013

PeterWAWood commented Jan 17, 2013

earl commented Jan 17, 2013

PeterWAWood commented Jan 17, 2013

PeterWAWood commented Jan 17, 2013

earl commented Jan 17, 2013

dockimbel commented Jan 17, 2013