lib/vfscore: Fix behavior of writable open on directories

[andreittr]

Description of changes

Previously the open syscall, when called on a directory with write mode would either fail with EINVAL or, worse, finish successfully, whereas the expected behavior is to return EISDIR.
This change corrects this behavior, bringing open() in line with expectations.

Prerequisite checklist

• Read the contribution guidelines regarding submitting new changes to the project;
• Tested your changes against relevant architectures and platforms;
• Ran the checkpatch.uk on your commit series before opening this PR;
• Updated relevant documentation.

Base target

• Architecture(s): all
• Platform(s): all
• Application(s): all

Additional configuration

Minimal test code; all 4 cases fail with EISDIR on Linux, none should open the dir successfully.

int f;
FILE *fp;

f = open("/tmp", O_WRONLY);
if (f >= 0) {
	puts("open tmp OK");
	close(f);
} else {
	perror("open tmp");
}

f = open("/tmp", O_WRONLY|O_CREAT);
if (f >= 0) {
	puts("open tmp CREAT OK");
	close(f);
} else {
	perror("open tmp CREAT");
}

fp = fopen("/tmp", "wb");
if (fp) {
	puts("fopen tmp OK");
	fclose(fp);
} else {
	perror("fopen tmp");
}

fp = fopen("/tmp", "a");
if (fp) {
	puts("fopen tmp a OK");
	fclose(fp);
} else {
	perror("fopen tmp a");
}

[unikraft-bot]

✅ Checkpatch passed

Beep boop! I ran Unikraft's checkpatch.pl support script on your pull request and it all looks good!

SHA	commit	checkpatch
ab99b0a	lib/vfscore: Fix behavior of writable open on dirs	✅

[dinhngtu]

Hi, I've left a comment.

[dinhngtu]

unikraft/lib/vfscore/syscalls.c

Line 197 in ab99b0a

goto out_drele;

should be goto out_unlock;

[andreittr]

Oh nice, good catch, thanks! Gonna fix asap.

[andreittr]

Pushed fix.

[dinhngtu]

That's all from me, thanks.

Reviewed-by: Tu Dinh Ngoc dinhngoc.tu@irit.fr

[razvand]

Approved-by: Razvan Deaconescu razvand@unikraft.io