Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib/*fs, lib/vfscore: Add proper dirent/dirent64 structure usage #963

Closed

Conversation

StefanJum
Copy link
Member

@StefanJum StefanJum commented Jun 29, 2023

Unikraft defines the dirent structure as identical to dirent64.
This works fine if not using Unikraft in binary compatibility mode,
since we have total control over the internal functions/structures that
are used.

However, if we use Unikraft in binary compatibility mode, we want to use
Linux dirent and dirent64 structures, in order to maintain compatibility
with older libc versions that do not use the *64 calls by default.
Since the filesystems READDIR vop uses dirent64, we will convert to a
dirent structure within the readdir_r call.

You can see the issue happening in #919
In order to test it with the app-elfloader, you can use the ls executable from the dynamic-apps repository, which uses an old libc version that calls getdents instead of getdents64.

Prerequisite checklist

  • Read the contribution guidelines regarding submitting new changes to the project;
  • Tested your changes against relevant architectures and platforms;
  • Ran the checkpatch.uk on your commit series before opening this PR;
  • Updated relevant documentation.

@StefanJum StefanJum requested review from a team as code owners June 29, 2023 10:17
@StefanJum StefanJum removed request for a team June 29, 2023 10:17
@StefanJum StefanJum added area/kconfig Part of the Unikraft KConfig option system lib/vfscore VFS Core Interface lib/nolibc Only neccessary subset of libc functionality labels Jun 29, 2023
@unikraft-bot unikraft-bot added area/lib Internal Unikraft Microlibrary lang/c Issues or PRs to do with C/C++ lib/9pfs 9p client lib/devfs devfs file system lib/ramfs labels Jun 29, 2023
@razvand razvand requested review from RaduNichita and mogasergiu and removed request for vladandrew June 29, 2023 12:22
@razvand razvand added this to the v0.14.0 (Prometheus) milestone Jun 29, 2023
Config.uk Outdated Show resolved Hide resolved
@mogasergiu
Copy link
Member

Just a more general comment, but would it be a good idea to use unlikely's on error path's here as well? 🤔

@StefanJum
Copy link
Member Author

Done @skuenzer, I moved the config option in vfscore, it will get enabled by default if the syscall handler is enabled.

Also added the unlikely's as @mogasergiu suggested .

Copy link
Contributor

@RaduNichita RaduNichita left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for providing the fix, @StefanJum. I have provided some nitpicks.

I will approve the PR after that! 💪

lib/nolibc/include/dirent.h Show resolved Hide resolved
lib/vfscore/main.c Outdated Show resolved Hide resolved
@StefanJum
Copy link
Member Author

Done @RaduNichita, thank you.

Copy link
Contributor

@RaduNichita RaduNichita left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good now. Thanks, @StefanJum! 💯

Reviewed-by: Radu Nichita radunichita99@gmail.com

Copy link
Member

@mogasergiu mogasergiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks cool to me, some comments for your notification box 📫.

lib/vfscore/main.c Show resolved Hide resolved
lib/vfscore/main.c Show resolved Hide resolved
lib/vfscore/main.c Show resolved Hide resolved
@StefanJum StefanJum force-pushed the StefanJum/fix-dents-calls branch 3 times, most recently from 39d3f14 to d30cc32 Compare August 10, 2023 07:38
@StefanJum
Copy link
Member Author

All done @skuenzer, sorry for the force-push spam, there were some checkpatch and ci/cd issues.

@StefanJum
Copy link
Member Author

WARNING:LINE_SPACING: Missing a blank line after declarations
#233: FILE: lib/vfscore/main.c:1080:
int (*cmp)(const struct dirent **, const struct dirent **))
__attribute__((alias("scandir64")));

The checkpatch is getting confused by the function pointer argument here 😆

@mogasergiu
Copy link
Member

WARNING:LINE_SPACING: Missing a blank line after declarations
#233: FILE: lib/vfscore/main.c:1080:
int (*cmp)(const struct dirent **, const struct dirent **))
attribute((alias("scandir64")));

The checkpatch is getting confused by the function pointer argument here laughing

The whole scandir64 function looks like a big checkpatch violation to be honest 😤. Maybe we could rewrite it better 🤔.

@StefanJum
Copy link
Member Author

The whole scandir64 function looks like a big checkpatch violation to be honest 😤. Maybe we could rewrite it better 🤔.

Yes, it's absolutely hideous, I just copied the scandir function. I'll try to do a rewrite if I catch some time.

@skuenzer
Copy link
Member

⚠️ Checkpatch passed with warnings

Beep boop! I ran Unikraft's checkpatch.pl support script on your pull request but it encountered warnings:
SHA commit checkpatch
cfc5c4c lib/vfscore: Remove readdir_r result from TLS ✅
093be54 lib/vfscore: Add non-largefile variant for dirent ⚠️

Truncated logs starting from first warning 093be54:

WARNING:LONG_LINE: line over 80 characters
#68: FILE: lib/nolibc/include/dirent.h:22:
+#if !defined(CONFIG_LIBVFSCORE_NONLARGEFILE) || (CONFIG_LIBVFSCORE_NONLARGEFILE <= 0)

WARNING:LINE_SPACING: Missing a blank line after declarations
#233: FILE: lib/vfscore/main.c:1080:
+	int (*cmp)(const struct dirent **, const struct dirent **))
+	__attribute__((alias("scandir64")));

total: 0 errors, 2 warnings, 377 lines checked

NOTE: For some of the reported defects, checkpatch may be able to
      mechanically convert to the typical style using --fix or --fix-inplace.

/tmp/build/a53d4c5b/patches/0002-lib-vfscore-Add-non-largefile-variant-for-dirent.patch has style problems, please review.

NOTE: Ignored message types: ASSIGN_IN_IF FILE_PATH_CHANGES NEW_TYPEDEFS OBSOLETE

NOTE: If any of the errors are false positives, please report
      them to the maintainer, see CHECKPATCH in MAINTAINERS.

View complete logs | Learn more about Unikraft's coding style and contribution guidelines.

Because I am seeing this here:
You do not need to do

#if !defined(CONFIG_LIBVFSCORE_NONLARGEFILE) || (CONFIG_LIBVFSCORE_NONLARGEFILE <= 0)

the following achieves exactly the same (except it does not check for < 0, but it is fine to assume a CONFIG to be set to 0(and undefined) or 1):

+#if !CONFIG_LIBVFSCORE_NONLARGEFILE

The preprocessor handles the case of being not defined correctly.

@StefanJum
Copy link
Member Author

The preprocessor handles the case of being not defined correctly.

I did not know that 👀 Pushed the change.

lib/nolibc/include/dirent.h Outdated Show resolved Hide resolved
lib/vfscore/main.c Outdated Show resolved Hide resolved
lib/vfscore/main.c Outdated Show resolved Hide resolved
lib/vfscore/main.c Outdated Show resolved Hide resolved
lib/vfscore/main.c Outdated Show resolved Hide resolved
lib/vfscore/main.c Show resolved Hide resolved
@StefanJum
Copy link
Member Author

Updated @skuenzer, thanks.

@razvand razvand requested a review from skuenzer August 11, 2023 16:34
@StefanJum StefanJum force-pushed the StefanJum/fix-dents-calls branch 2 times, most recently from 2487f8e to 9089d20 Compare August 14, 2023 17:16
Unikraft defines the `dirent` structure as identical to `dirent64`.
This works fine if not using Unikraft in binary compatibility mode,
since we have total control over the internal functions/structures that
are used.

However, if we use Unikraft in binary compatibility mode, we want to use
Linux `dirent` and `dirent64` structures in order to maintain compatibility
with older libc versions that do not use the `*64` calls by default.
Since the filesystems `READDIR` vop uses `dirent64`, we will convert to a
`dirent` structure within the `readdir_r` call.

Signed-off-by: Stefan Jumarea <stefanjumarea02@gmail.com>
GitHub-Closes: unikraft#919
@unikraft-bot
Copy link
Member

⚠️ Checkpatch passed with warnings

Beep boop! I ran Unikraft's checkpatch.pl support script on your pull request but it encountered warnings:

SHA commit checkpatch
cfc5c4c lib/vfscore: Remove readdir_r result from TLS
fe58a9a lib/vfscore: Add non-largefile variant for dirent ⚠️

Truncated logs starting from first warning fe58a9a:

WARNING:LINE_SPACING: Missing a blank line after declarations
#231: FILE: lib/vfscore/main.c:1080:
+	int (*cmp)(const struct dirent **, const struct dirent **))
+	__attribute__((alias("scandir64")));

total: 0 errors, 1 warnings, 389 lines checked

NOTE: For some of the reported defects, checkpatch may be able to
      mechanically convert to the typical style using --fix or --fix-inplace.

/tmp/build/a53d4c5b/patches/0002-lib-vfscore-Add-non-largefile-variant-for-dirent.patch has style problems, please review.

NOTE: Ignored message types: ASSIGN_IN_IF FILE_PATH_CHANGES NEW_TYPEDEFS OBSOLETE

NOTE: If any of the errors are false positives, please report
      them to the maintainer, see CHECKPATCH in MAINTAINERS.

View complete logs | Learn more about Unikraft's coding style and contribution guidelines.

Copy link
Member

@skuenzer skuenzer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for your work!

Reviewed-by: Simon Kuenzer simon@unikraft.io
Approved-by: Simon Kuenzer simon@unikraft.io

unikraft-bot pushed a commit that referenced this pull request Aug 16, 2023
Unikraft defines the `dirent` structure as identical to `dirent64`.
This works fine if not using Unikraft in binary compatibility mode,
since we have total control over the internal functions/structures that
are used.

However, if we use Unikraft in binary compatibility mode, we want to use
Linux `dirent` and `dirent64` structures in order to maintain compatibility
with older libc versions that do not use the `*64` calls by default.
Since the filesystems `READDIR` vop uses `dirent64`, we will convert to a
`dirent` structure within the `readdir_r` call.

Signed-off-by: Stefan Jumarea <stefanjumarea02@gmail.com>
GitHub-Closes: #919
Reviewed-by: Radu Nichita <radunichita99@gmail.com>
Reviewed-by: Sergiu Moga <sergiu@unikraft.io>
Reviewed-by: Simon Kuenzer <simon@unikraft.io>
Approved-by: Simon Kuenzer <simon@unikraft.io>
Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #963
@unikraft-bot unikraft-bot added the ci/merged Merged by CI label Aug 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/kconfig Part of the Unikraft KConfig option system area/lib Internal Unikraft Microlibrary ci/merged Merged by CI lang/c Issues or PRs to do with C/C++ lib/devfs devfs file system lib/nolibc Only neccessary subset of libc functionality lib/ramfs lib/vfscore VFS Core Interface lib/9pfs 9p client
Projects
Status: Done!
Development

Successfully merging this pull request may close these issues.

None yet

6 participants