Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File_system: support for file modification timestamps #1784

Closed
cnuke opened this issue Nov 22, 2015 · 20 comments
Closed

File_system: support for file modification timestamps #1784

cnuke opened this issue Nov 22, 2015 · 20 comments
Labels

Comments

@cnuke
Copy link
Member

cnuke commented Nov 22, 2015

By now having support for something like st_ctim and st_mtim sounds reasonable. Adding this feature would first of all involve extending File_system::Status as well as adding the required functionality to each file system or rather enabling the usage by tapping into the existing API (e.g. rump_fs) and probably the layers above (VFS). I am not sure if it is actually necessary to support it in all pseudo file system (or even ram_fs for that matter) but at least supporting in the ones used for persistent storage would be nice.

@cnuke
Copy link
Member Author

cnuke commented Mar 18, 2019

The plan for implementing the functionality is as follows:

  1. Add a Timestamp member to File_system::Status which saves the modification time as signed 64bit integer. If a more structured type is needed, we will introduce convenience utilities.
  2. Add a new packet type to the File_system session called WRITE_TIMESTAMP which allows clients to update the modification timestamp of a node. The rough plan is to send this packet whenenver a “dirty” node is closed or before a SYNC packets but that's up to the client, e.g. VFS/libc.

Setting the modification time of a node should be a deliberate decision of the client rather than an implicit side-effect of the file system provider, which may or may not has access to a time source.

Afterwards we'll deal with the VFS and libc side of things.

@nfeske
Copy link
Member

nfeske commented Mar 18, 2019

Thank you for capturing the outcome of our offline discussion, @cnuke.

@cnuke
Copy link
Member Author

cnuke commented Mar 21, 2019

My file_system_modtime__wip branch contains the first sketch (be warned I haven't even tried to compile it yet).

@nfeske
Copy link
Member

nfeske commented Mar 25, 2019

The commits already very nice. Please see my inline comments.

@cnuke
Copy link
Member Author

cnuke commented Apr 10, 2019

I updated my wip branch. It now contains a semi-functioning implementation, open issue to consider:

  • For one it relies on doing the right thing at the client-side, i.e., calling utimes(). This does not happen a newly created files (that includes cping as well) as that is normally the job of the kernel .
  • Apart from the that, in case of a libc component, the modification time always gets updated whenever _vfs_sync() is executed. Since we do not track if a file was opened r/o or rather if it was modified (the same is true for noux) we now change it accidently as a side-effect.
  • We normally discourage writing to directories but we now might still update their modification time.
  • Sophisticated VFS plugins, e.g., vfs_rump, might ignore our attempt to set the modification time and will change it own their own.

So far I will ignore the last two points and focus on the first two.

@ehmry
Copy link
Contributor

ehmry commented Apr 11, 2019

The eager sync has already been identified as a unnecessary and don't see any problems with skipping it for unmodified files.

@cnuke
Copy link
Member Author

cnuke commented Apr 11, 2019

Yes and the tracking of unmodified files is the thing missing so far.

@cnuke
Copy link
Member Author

cnuke commented Apr 18, 2019

I've force pushed the branch again and it now contains commits that address the first two points of my previous post. Together with some small adjustments in the *genodians.org` appliance, lighttpd serves resources with the proper modification time now. I'll work on cleaning it up for staging next week.

@cnuke
Copy link
Member Author

cnuke commented Apr 23, 2019

The current WIP state is now live on genodians.org.

@cnuke
Copy link
Member Author

cnuke commented Apr 24, 2019

Running it on genodians.org revealed a bug in the current implementation (I didn't catch the Insufficient_buffer exception of the Fs_file_system in the libc) and I changed it to use the same mechanism as the sync packet now. That being said, I reverted genodians.org back to an older version b/c there are still some issue left.

@cnuke
Copy link
Member Author

cnuke commented May 7, 2019

I put the commits on top of the current master branch and they are live again on genodians.org.

@m-stein you may give thunderbird's RSS tool a try.

@m-stein
Copy link
Contributor

m-stein commented May 7, 2019

@cnuke Ah thanks!

@m-stein
Copy link
Contributor

m-stein commented May 7, 2019

@cnuke Seems to work - thanks a lot!

@cnuke
Copy link
Member Author

cnuke commented May 16, 2019

And the mtime commits are off again, there are apparently still some problems left (genodians.org sometimes still gets stuck, setting verbose="yes" in noux seems to make it more reliable while interactive usage works fine).

cnuke added a commit to cnuke/genode that referenced this issue Jun 26, 2019
cnuke added a commit to cnuke/genode that referenced this issue Jun 26, 2019
cnuke added a commit to cnuke/genode that referenced this issue Jun 26, 2019
cnuke added a commit to cnuke/genode that referenced this issue Jun 26, 2019
cnuke added a commit to cnuke/genode that referenced this issue Jun 26, 2019
cnuke added a commit to cnuke/genode that referenced this issue Jun 26, 2019
cnuke added a commit to cnuke/genode that referenced this issue Jun 26, 2019
... and set initial time by using RTC session.

Up to now Noux used a monotic clock whose initial start value always
was '0' (which correlates to the start of the UNIX epoch) to provide
a notion of time. In addition it is now possible to use the RTC
session to set the initial value for use-case where having a proper real
world time matters. It does not do any time-zone conversion - the returned
time depends on the component provide the RTC service.

To use the RTC session the 'rtc' attribute of the '<config>' node
must be set to 'yes'. Thereby the session becomes a mandatory
dependency as Noux will not start without it.

Issue genodelabs#1784.
cnuke added a commit to cnuke/genode that referenced this issue Jun 26, 2019
nfeske pushed a commit to nfeske/genode that referenced this issue Sep 23, 2019
nfeske pushed a commit to nfeske/genode that referenced this issue Sep 23, 2019
nfeske pushed a commit to nfeske/genode that referenced this issue Sep 23, 2019
nfeske pushed a commit to nfeske/genode that referenced this issue Sep 23, 2019
nfeske pushed a commit to nfeske/genode that referenced this issue Sep 23, 2019
nfeske pushed a commit to nfeske/genode that referenced this issue Sep 23, 2019
... and set initial time by using RTC session.

Up to now Noux used a monotic clock whose initial start value always
was '0' (which correlates to the start of the UNIX epoch) to provide
a notion of time. In addition it is now possible to use the RTC
session to set the initial value for use-case where having a proper real
world time matters. It does not do any time-zone conversion - the returned
time depends on the component provide the RTC service.

To use the RTC session the 'rtc' attribute of the '<config>' node
must be set to 'yes'. Thereby the session becomes a mandatory
dependency as Noux will not start without it.

Issue genodelabs#1784.
chelmuth pushed a commit that referenced this issue Sep 30, 2019
By specifying <libc update_mtime="no"...>, the modification-time update
on VFS-sync operations (as issued whenever a written file is closed)
can explicitly be disabled.

Issue #1784
chelmuth pushed a commit that referenced this issue Sep 30, 2019
With the added modification-time support in the libc, the extract tool
requires a timer session, which is not plausible for the purpose of the
program.

This behavior stems from the fact that the libc implicitly writes the
mtime when closing a written file. For this update, it implicitly calls
'clock_gettime', which in turn initializes the timer subsystem within
the libc (creating a timer session).

For the extract tool, the implicitly updated mtime is useless because
the extract tool overwrites this modification time with the mtime stored
in the archive anyway. However, the dependency from a timer service
remains.

This patch explicitly disables the libc's implicit updating of the
file-modification when closing a written file.

Issue #1784
nfeske pushed a commit to nfeske/genode that referenced this issue Oct 1, 2019
nfeske pushed a commit to nfeske/genode that referenced this issue Oct 1, 2019
nfeske pushed a commit to nfeske/genode that referenced this issue Oct 1, 2019
nfeske pushed a commit to nfeske/genode that referenced this issue Oct 1, 2019
nfeske pushed a commit to nfeske/genode that referenced this issue Oct 1, 2019
nfeske pushed a commit to nfeske/genode that referenced this issue Oct 1, 2019
... and set initial time by using RTC session.

Up to now Noux used a monotic clock whose initial start value always
was '0' (which correlates to the start of the UNIX epoch) to provide
a notion of time. In addition it is now possible to use the RTC
session to set the initial value for use-case where having a proper real
world time matters. It does not do any time-zone conversion - the returned
time depends on the component provide the RTC service.

To use the RTC session the 'rtc' attribute of the '<config>' node
must be set to 'yes'. Thereby the session becomes a mandatory
dependency as Noux will not start without it.

Issue genodelabs#1784.
nfeske pushed a commit to nfeske/genode that referenced this issue Oct 1, 2019
nfeske pushed a commit to nfeske/genode that referenced this issue Oct 1, 2019
nfeske added a commit to nfeske/genode that referenced this issue Oct 1, 2019
chelmuth pushed a commit that referenced this issue Nov 19, 2019
chelmuth pushed a commit that referenced this issue Nov 19, 2019
chelmuth pushed a commit that referenced this issue Nov 19, 2019
chelmuth pushed a commit that referenced this issue Nov 19, 2019
... and set initial time by using RTC session.

Up to now Noux used a monotic clock whose initial start value always
was '0' (which correlates to the start of the UNIX epoch) to provide
a notion of time. In addition it is now possible to use the RTC
session to set the initial value for use cases where having a proper
real-world time matters.

To use the RTC session the 'rtc' attribute of the '<config>' node
must be set to 'yes'. Thereby the session becomes a mandatory
dependency as Noux will not start without it.

Issue #1784
chelmuth pushed a commit that referenced this issue Nov 19, 2019
chelmuth pushed a commit that referenced this issue Nov 19, 2019
chelmuth pushed a commit that referenced this issue Nov 19, 2019
By specifying <libc update_mtime="no"...>, the modification-time update
on VFS-sync operations (as issued whenever a written file is closed)
can explicitly be disabled.

Issue #1784
chelmuth pushed a commit that referenced this issue Nov 19, 2019
With the added modification-time support in the libc, the extract tool
requires a timer session, which is not plausible for the purpose of the
program.

This behavior stems from the fact that the libc implicitly writes the
mtime when closing a written file. For this update, it implicitly calls
'clock_gettime', which in turn initializes the timer subsystem within
the libc (creating a timer session).

For the extract tool, the implicitly updated mtime is useless because
the extract tool overwrites this modification time with the mtime stored
in the archive anyway. However, the dependency from a timer service
remains.

This patch explicitly disables the libc's implicit updating of the
file-modification when closing a written file.

Issue #1784
@nfeske
Copy link
Member

nfeske commented Nov 19, 2019

Fixed in master.

@nfeske nfeske closed this as completed Nov 19, 2019
nfeske added a commit that referenced this issue Nov 21, 2019
This is a follow-up patch for issue #1784 that solves two
inconsistencies.

- The Vfs::Timestamp::INVALID matches File_system::Timestamp::INVALID
- The Noux libc plugin tests for Timestamp::INVALID instead of a
  positive value.

The patch fixes the mtime info as shown in directory listings in
Sculpt's inspect window.
cnuke added a commit to cnuke/genode that referenced this issue Jan 4, 2022
Although support was added some time ago (genodelabs#1784) the commit missed
executing the code from the 'Vfs::File_system' interface.
cnuke added a commit to cnuke/genode that referenced this issue Jan 10, 2022
Although support was added some time ago (genodelabs#1784) the commit missed
executing the code from the 'Vfs::File_system' interface.

Fixes genodelabs#4371.
nfeske pushed a commit that referenced this issue Jan 10, 2022
Although support was added some time ago (#1784) the commit missed
executing the code from the 'Vfs::File_system' interface.

Fixes #4371.
chelmuth pushed a commit that referenced this issue Jan 19, 2022
Although support was added some time ago (#1784) the commit missed
executing the code from the 'Vfs::File_system' interface.

Fixes #4371.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants