node.js - fs.js - writefilesync and renamesync - file doesn't exist before attempting to rename

[3blake7]

Compiling shader 1 of 1 (ChairFabric_shadowmap.frag.glsl).
Compiling shader 1 of 1 (ChairFabric_shadowmap.vert.glsl).
Compiling shader 1 of 1 (Coffee_mesh.frag.glsl).
Recompiling Coffee_mesh.frag
Compiling shader 1 of 1 (Coffee_mesh.vert.glsl).
fs.js:809
return binding.rename(pathModule._makeLong(oldPath),
^

Error: ENOENT: no such file or directory, rename 'C:\Users\blake\Documents\blender\build\html5-resources\Coffee_mesh.frag.essl.temp' -> 'C:\Users\blake\Documents\blender\build\html5-resources\Coffee_mesh.frag.essl'
at Error (native)
at Object.fs.renameSync (fs.js:809:18)
at ChildProcess.child.on (C:\Users\blake\Downloads\Armory3D\Armory\armsdk\win32\Kha\Tools\khamake\out\ShaderCompiler.js:356:44)
at emitTwo (events.js:106:13)
at ChildProcess.emit (events.js:191:7)
at maybeClose (internal/child_process.js:852:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:215:5)
Build failed, check console

I was getting this error with Armory3D

Here is my quote from Armory3D issue: #176

I was investigating this issue, my hunch was that it is asynchronous and the writing of the .temp is sometimes slow, depending on what's going on on my computer, which gives it the intermittent appearance. I found that the writing and renaming was done by the node.js plugin fs.js. I found the functions writefilesync and renamesync.

According to: https://nodejs.org/api/fs.html#fs_fs_writefilesync_file_data_options
Both writefilesync and renamesync are the synchronous versions of the functions.

Then I found this article:
http://www.daveeddy.com/2013/03/26/synchronous-file-io-in-nodejs/
Which did a little experiment and comes to the conclusion that the writefilesync and renamesync are synchronous within the node.js context but they tell the operating system to do asynchronous write and rename.

[luboslenco]

Hm, just noticed there is always Recompiling xyz.frag line before it errors out, maybe I should investigate that first.

[3blake7]

Looks like the same issue is happening to \win32\Kha\Tools\khamake\out\ImageTool.js as well. When it reexports, it overwrites the file, and renameSync is running before the rewrite operation is completed, giving the following error:

Exporting asset 1 of 1 (spacestation-residential-unit-d06b018.png).
Reexporting spacestation-residential-unit-d06b018
fs.js:809
  return binding.rename(pathModule._makeLong(oldPath),
                 ^

Error: EBUSY: resource busy or locked, rename 'C:\Users\blake\Documents\blender\build\html5\spacestation-residential-unit-d06b018.png.temp' -> 'C:\Users\blake\Documents\blender\build\html5\spacestation-residential-unit-d06b018.png'
    at Error (native)
    at Object.fs.renameSync (fs.js:809:18)
    at ChildProcess.process.on (C:\Users\blake\Downloads\Armory3D\Armory\armsdk\win32\Kha\Tools\khamake\out\ImageTool.js:63:16)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:191:7)
    at maybeClose (internal/child_process.js:852:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:215:5)
 Build failed, check console

[3blake7]

function fileready(file) {
	var i = 0;
	while ( i == 0 ) {
		if (!fs.access(file, function(err){
			return(err);
		  })) {
			i = 1;
		}
	}
}

Im running the above function right before fs.renameSync on ImageTool.js and ShaderCompiler.js.. it seems to be working. Probably not the best solution lol Im not a programmer by trade.

[RobDangerous]

Now I get it. This is running in --watch mode and happens while the program is running, right? Something like fileready might just be the right fix then.

[luboslenco]

I am still confused since there is no --watch used in armory but the Recompiling/Reexporting messages are signs of that. Can't get to reproduce on my machine for some reason. @fractalfederation does it error out even on the default scene with a single cube, or just more complex ones?

[3blake7]

Yea it errors out on default scene. I just got the error again with nothing but a cube. I guess I did my function wrong and it was just a coincidence that it stopped getting the error after I added it yesterday.

[3blake7]

function KhaRename(file) {
	var i = 0;
	while(i == 0) {
		try {
			fs.renameSync(file, file.slice(0,-5));
			i++;
		} catch (err) {
			console.log(err);
		}
	}
}

I've been using the above code. It still errors and sometimes it errors forever. I looked at the build/html5/ directory for the image and it was already there without the .temp at the end of the file name. Adding the .temp at the end of the image allowed the script to finish. This was with a fresh build, I ran Clean Project which deleted the build directory.

So maybe it is renaming file and trying a second time? or the file is created initially without the .temp and a rename is unnecessary?

I am such a noob lol

Here is the error:

{ Error: ENOENT: no such file or directory, rename 'C:\Users\blake\Documents\blender\build\html5\spacestation-residential-unit-d06b018.png.temp' -> 'C:\Users\blake\Documents\blender\build\html5\spacestation-residential-unit-d06b018.png'
    at Error (native)
    at Object.fs.renameSync (fs.js:809:18)
    at KhaRename (C:\Users\blake\Downloads\Armory3D\Armory\armsdk\win32\Kha\Tools\khamake\out\ImageTool.js:20:7)
    at ChildProcess.process.on (C:\Users\blake\Downloads\Armory3D\Armory\armsdk\win32\Kha\Tools\khamake\out\ImageTool.js:74:13)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:191:7)
    at maybeClose (internal/child_process.js:852:16)
    at Socket.<anonymous> (internal/child_process.js:323:11)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
  errno: -4058,
  code: 'ENOENT',
  syscall: 'rename',
  path: 'C:\\Users\\blake\\Documents\\blender\\build\\html5\\spacestation-residential-unit-d06b018.png.temp',
  dest: 'C:\\Users\\blake\\Documents\\blender\\build\\html5\\spacestation-residential-unit-d06b018.png' }

[luboslenco]

I got somewhat similar(but not the same) trouble on Windows 10 machine today targeting html5.

Using --to parameter to specify output dir in khamake and passing absolute path results in -webgl2 & -relaxed shaders being left with .temp extension. Passing relative path instead seems to fix the problem.

[3blake7]

It seems to be two different issues, it happens with the shader files and image files. Sometimes it tries to rename a file that doesn't exist, but if you wait and try again, then it exists and renames successfully. Other times, the file is already renamed.

Here is the workaround I am currently using:

function KhaRename(file) {
	var i = 0;
	while(i == 0) {
		try {
			fs.renameSync(file, file.slice(0,-5));
			i++;
		} catch (err) {
			console.log(err);
			if (fs.existsSync(file.slice(0, -5))) {
				i++;
			}
		}
	}
}

Hope this helps. If you figure out a fix, I subscribed and get the message right away, so I'll hop on and test it.

[carlito767]

Hi all :)

I encountered the problem on my project. After some tests, I discovered it was because of the dot in the folder name:

C:\Dames.chinoises

By changing the folder name to:

C:\Dames_chinoises

it was ok.

[3blake7]

I checked mine, I didn't see any periods but I do have hyphens, ampersands, underscores and even cyrillic characters.

Sure it wasn't a coincidence? When I ran build multiple times, it would create the missing file on the last attempt and no longer error out on that particular file. If you run it enough times, it eventually gets through all the missing files.

[carlito767]

There are probably several ways to get this error.

In my case, even if I run build multiple times and get the missing files, at the end I always get a black screen (html5 target).
I tried many times with different folder names, using Kode Studio or a batch file, and I still get the same result.

Unfortunately I don't know enough to analyze, I just hope it helps to understand and fix the problem.

[luboslenco]

(updated)
Running node Kha/make html5 on macOS High Sierra now also throws error when using . in path.

Compiling shader 1 of 8 (painter-colored.frag.glsl).
fs.js:773
  return binding.rename(pathModule._makeLong(oldPath),
                 ^

Error: ENOENT: no such file or directory, rename 'build/html5-resources/painter-colored.frag.essl.temp' -> 'build/html5-resources/painter-colored.frag.essl'

[RobDangerous]

The dot problem was a separate thing which is now also fixed.