Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Do real revalidation of "remote" file systems #4

Closed
nlacasse opened this issue Apr 26, 2018 · 2 comments
Closed

Do real revalidation of "remote" file systems #4

nlacasse opened this issue Apr 26, 2018 · 2 comments
Assignees
@nlacasse
Labels
area: container runtime Issue related to docker, kubernetes, OCI runtime area: filesystem Issue related to filesystem area: integration Issue related to third party integrations priority: p2 Normal priority type: bug Something isn't working

Comments

@nlacasse
Copy link
Collaborator

nlacasse commented Apr 26, 2018 •
edited by fvoznika

Re this item in the FAQ:

I can’t see a file copied with docker cp.

For performance reasons, gVisor caches directory contents, and therefore it may not realize a new file was copied to a given directory. To invalidate the cache and force a refresh, create a file under the directory in question and list the contents again.

DirentOperations (akin to dentry_operations in Linux; embedded in MountOperations, which are about to be renamed MountSourceOperations, and are akin to super_operations) currently have Revalidate and Keep methods.

In Linux, revalidation will stat the inode to see if anything has changed. If it has, it re-walks to that node.

In gVisor, revalidation always returns false for host files, so the sandbox will notice the new file either when it falls out of the Dirent cache or when you readdir the containing directory. For Gofer files, revalidation is controlled by whether Dirents are cached or not, which should also not be the case.

We should be following Linux more closely here, which would also resolve the above problem. For Gofers, we should provide a configuration option to say "don't revalidate" that one can use as a performance optimization if they know only gVisor has access to that mount and nothing else can add/change/remove files.
shentubot pushed a commit that referenced this issue Jul 3, 2018
@dvyukov @shentubot
runsc/boot/filter: permit SYS_TIME for race
6f6f14f 
glibc's malloc also uses SYS_TIME. Permit it.

#0  0x0000000000de6267 in time ()
#1  0x0000000000db19d8 in get_nprocs ()
#2  0x0000000000d8a31a in arena_get2.part ()
#3  0x0000000000d8ab4a in malloc ()
#4  0x0000000000d3c6b5 in __sanitizer::InternalAlloc(unsigned long, __sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator32<0ul, 140737488355328ull, 0ul, __sanitizer::SizeClassMap<3ul, 4ul, 8ul, 17ul, 64ul, 14ul>, 20ul, __sanitizer::TwoLevelByteMap<32768ull, 4096ull, __sanitizer::NoOpMapUnmapCallback>, __sanitizer::NoOpMapUnmapCallback> >*, unsigned long) ()
#5  0x0000000000d4cd70 in __tsan_go_start ()
#6  0x00000000004617a3 in racecall ()
#7  0x00000000010f4ea0 in runtime.findfunctab ()
#8  0x000000000043f193 in runtime.racegostart ()

Signed-off-by: Dmitry Vyukov <dvyukov@google.com>
[mpratt@google.com: updated comments and commit message]
Signed-off-by: Michael Pratt <mpratt@google.com>

Change-Id: Ibe2d0dc3035bf5052d5fb802cfaa37c5e0e7a09a
PiperOrigin-RevId: 203042627
dvyukov added a commit to dvyukov/gvisor that referenced this issue Jul 4, 2018
@dvyukov @shentubot
runsc/boot/filter: permit SYS_TIME for race
6144751 
glibc's malloc also uses SYS_TIME. Permit it.

#0  0x0000000000de6267 in time ()
google#1  0x0000000000db19d8 in get_nprocs ()
google#2  0x0000000000d8a31a in arena_get2.part ()
google#3  0x0000000000d8ab4a in malloc ()
google#4  0x0000000000d3c6b5 in __sanitizer::InternalAlloc(unsigned long, __sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator32<0ul, 140737488355328ull, 0ul, __sanitizer::SizeClassMap<3ul, 4ul, 8ul, 17ul, 64ul, 14ul>, 20ul, __sanitizer::TwoLevelByteMap<32768ull, 4096ull, __sanitizer::NoOpMapUnmapCallback>, __sanitizer::NoOpMapUnmapCallback> >*, unsigned long) ()
google#5  0x0000000000d4cd70 in __tsan_go_start ()
google#6  0x00000000004617a3 in racecall ()
google#7  0x00000000010f4ea0 in runtime.findfunctab ()
google#8  0x000000000043f193 in runtime.racegostart ()

Signed-off-by: Dmitry Vyukov <dvyukov@google.com>
[mpratt@google.com: updated comments and commit message]
Signed-off-by: Michael Pratt <mpratt@google.com>

Change-Id: Ibe2d0dc3035bf5052d5fb802cfaa37c5e0e7a09a
PiperOrigin-RevId: 203042627
@fvoznika fvoznika added the type: bug Something isn't working label Jan 11, 2019
tonistiigi pushed a commit to tonistiigi/gvisor that referenced this issue Jan 30, 2019
@dvyukov @shentubot
runsc/boot/filter: permit SYS_TIME for race
a617331 
glibc's malloc also uses SYS_TIME. Permit it.

#0  0x0000000000de6267 in time ()
#1  0x0000000000db19d8 in get_nprocs ()
#2  0x0000000000d8a31a in arena_get2.part ()
#3  0x0000000000d8ab4a in malloc ()
google#4  0x0000000000d3c6b5 in __sanitizer::InternalAlloc(unsigned long, __sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator32<0ul, 140737488355328ull, 0ul, __sanitizer::SizeClassMap<3ul, 4ul, 8ul, 17ul, 64ul, 14ul>, 20ul, __sanitizer::TwoLevelByteMap<32768ull, 4096ull, __sanitizer::NoOpMapUnmapCallback>, __sanitizer::NoOpMapUnmapCallback> >*, unsigned long) ()
google#5  0x0000000000d4cd70 in __tsan_go_start ()
google#6  0x00000000004617a3 in racecall ()
google#7  0x00000000010f4ea0 in runtime.findfunctab ()
google#8  0x000000000043f193 in runtime.racegostart ()

Signed-off-by: Dmitry Vyukov <dvyukov@google.com>
[mpratt@google.com: updated comments and commit message]
Signed-off-by: Michael Pratt <mpratt@google.com>

Change-Id: Ibe2d0dc3035bf5052d5fb802cfaa37c5e0e7a09a
PiperOrigin-RevId: 203042627
Upstream-commit: 6144751
@hugelgupf hugelgupf assigned nlacasse and unassigned hugelgupf Mar 25, 2019
@ianlewis ianlewis added area: filesystem Issue related to filesystem priority: p2 Normal priority labels Apr 13, 2019
tanjianfeng added a commit to tanjianfeng/gvisor that referenced this issue Aug 2, 2019
@tanjianfeng
fix wrong SizeOfTCPInfo
8815a98 
Below command under hostinet network will lead to panic:

  $ cat /proc/net/tcp

It's caused by the wrong SizeOfTCPInfo.

  #0 runtime.panicindex()
  google#1 encoding/binary.littleEndian.Uint64
  google#2 encoding/binary.(*littleEndian).Uint64
  google#3 gvisor.dev/gvisor/pkg/binary.unmarshal
  google#4 gvisor.dev/gvisor/pkg/binary.unmarshal
  google#5 gvisor.dev/gvisor/pkg/binary.Unmarshal
  google#6 gvisor.dev/gvisor/pkg/sentry/socket/hostinet.(*socketOperations).State
  google#7 gvisor.dev/gvisor/pkg/sentry/fs/proc.(*netTCP).ReadSeqFileData

Correct SizeOfTCPInfo from 104 to 192 to fix it.

Signed-off-by: Jianfeng Tan <henry.tjf@antfin.com>
@tanjianfeng tanjianfeng mentioned this issue Aug 2, 2019
Closed
tanjianfeng added a commit to tanjianfeng/gvisor that referenced this issue Aug 2, 2019
@tanjianfeng
fix wrong SizeOfTCPInfo
0927085 
Below command under hostinet network will lead to panic:

  $ cat /proc/net/tcp

It's caused by the wrong SizeOfTCPInfo.

  #0 runtime.panicindex()
  google#1 encoding/binary.littleEndian.Uint64
  google#2 encoding/binary.(*littleEndian).Uint64
  google#3 gvisor.dev/gvisor/pkg/binary.unmarshal
  google#4 gvisor.dev/gvisor/pkg/binary.unmarshal
  google#5 gvisor.dev/gvisor/pkg/binary.Unmarshal
  google#6 gvisor.dev/gvisor/pkg/sentry/socket/hostinet.(*socketOperations).State
  google#7 gvisor.dev/gvisor/pkg/sentry/fs/proc.(*netTCP).ReadSeqFileData

Correct SizeOfTCPInfo from 104 to 192 to fix it.

Fixes google#640

Signed-off-by: Jianfeng Tan <henry.tjf@antfin.com>
amscanne referenced this issue in amscanne/gvisor Nov 14, 2019
@iangudger @amscanne
Tweaks to architecture guide (#4)
cf172c7
@ianlewis ianlewis mentioned this issue Jul 6, 2020
Closed
@ianlewis ianlewis added the area: container runtime Issue related to docker, kubernetes, OCI runtime label Jul 8, 2020
@ianlewis ianlewis added the area: integration Issue related to third party integrations label Aug 14, 2020
craig08 pushed a commit to craig08/gvisor that referenced this issue Aug 20, 2020
@boyuan-he
Merge pull request google#4 from ridwanmsharif/ridwanmsharif/fix-read…
948b19a 
…dir-test

fuse: use safe go_marshal API for FUSE
@nlacasse
Copy link
Collaborator Author

nlacasse commented Jun 8, 2021

This is fixed in VFS2.

@nlacasse nlacasse closed this as completed Jun 8, 2021
@mtaku3
Copy link

mtaku3 commented Jul 19, 2023

I'm aware that this issue isn't fixed.
Creating container and copying files works just fine, but after executing some commands using execInstance it suddenly became unable to copy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nlacasse nlacasse

Labels
area: container runtime Issue related to docker, kubernetes, OCI runtime area: filesystem Issue related to filesystem area: integration Issue related to third party integrations priority: p2 Normal priority type: bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@ianlewis @nlacasse @hugelgupf @fvoznika @mtaku3