allocating file with size (IDFGH-12035)

[djdisodo]

Is your feature request related to a problem?

sometimes i have to create a file of certain size (for fseek)

it's not a major problem on conventional OSs since posix's ftruncate supports extending, iirc windows has similar feature too
If fildes refers to a regular file, the ftruncate() function shall cause the size of the file to be truncated to length. If the size of the file previously exceeded length, the extra data shall no longer be available to reads on the file. If the file previously was smaller than this size, ftruncate() shall increase the size of the file. If the file size is increased, the extended area shall appear as if it were zero-filled.

linux even supports fallocate() which creates a file without initialization

even if filesystem doesn't have a way to do this effectively it should just fill zero to ensure consistency

i found fatfs will reject if i try to extend the file using ftruncate

i don't know if it's also the case for other filesystems

Describe the solution you'd like.

support extending file with ftruncate for fatfs and other filesystems

better if we can get fallocate support

Describe alternatives you've considered.

extending file using such apis will be faster than writing zeros to the file
also preventing fragmentations

Additional context.

https://pubs.opengroup.org/onlinepubs/9699919799/functions/ftruncate.html

https://man7.org/linux/man-pages/man2/fallocate.2.html

[igrr]

i found fatfs will reject if i try to extend the file using ftruncate

Could you please provide more information about the issue you faced: version of ESP-IDF, the code to reproduce the issue, log output (if any), sdkconfig. Thanks in advance.

[djdisodo]

i found fatfs will reject if i try to extend the file using ftruncate

Could you please provide more information about the issue you faced: version of ESP-IDF, the code to reproduce the issue, log output (if any), sdkconfig. Thanks in advance.

i was using esp-rs with esp-idf version of 5.1.2
what i tried to do was use rust File::set_len to extend the file which uses truncate internally

i got error
PermissionDenied
with message "Not owner"
and didn't saw anything other than the set_len function returning an error

maybe this is just because of my logging settings

the message doesn't make any sense but i think it's just that there's something wrong handling esp-idf error

i tried to track down where did this message came from but couldn't find it

however it seems straightforward that fatfs will not truncate to extend the file length, instead return Permission error

esp-idf/components/fatfs/vfs/vfs_fat.c

Lines 1036 to 1042 in 7cbee80

	long sz = f_size(file);
	if (sz < length) {
	ESP_LOGD(TAG, "ftruncate does not support extending size");
	errno = EPERM;
	ret = -1;
	goto out;
	}

[adokitkat]

Hello. You should be able to use lseek to expand a file (and maybe then ftruncate/truncate?), although it will not behave exactly like a POSIX lseek - it uses f_lseek from FATFS, read more here: http://elm-chan.org/fsw/ff/doc/lseek.html.

If an offset beyond the file size is specified in write mode, the file size is expanded to the specified offset in this function.
Be careful of the difference in this behavior between FatFs and POSIX.
The file data in the expanded part is undefined, because no data is written to the file in this process.
This is suitable to pre-allocate a data area to the file quickly for fast write operation.

However I am not very familiar with Rust, so I don't know is this API is exposed to you.

[djdisodo]

@adokitkat it seems to extend the file length and a lot faster than writing zeros