Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fsync support #150

Closed
shoober420 opened this issue Jul 31, 2019 · 6 comments
Closed

fsync support #150

shoober420 opened this issue Jul 31, 2019 · 6 comments
Assignees
@damentz

Comments

@shoober420
Copy link

Valve’s Proton has recently added support for fsync. It’s said to reduce CPU overhead, and improve performance.

https://steamcommunity.com/games/221410/announcements/detail/2957094910196249305

Can this option be added to the zen kernel, or is it being added upstream in the mainline kernel down the line?
@damentz
Copy link
Member

damentz commented Aug 1, 2019

I'm interested in this patch set as well, but I think it's wise to wait till it's past RFC before we merge it into zen-kernel. There's still on-going discussion so what's that complete and a second revision is up that gets more praise, we can merge it early before it's upstreamed.

@damentz damentz self-assigned this Aug 1, 2019
@damentz
Copy link
Member

damentz commented Aug 1, 2019

Assigned to myself so I can keep an eye on the updates to this patch.

https://lkml.org/lkml/2019/7/30/1398

@shoober420
Copy link
Author

Cool, this is pretty exciting for Wine/Proton emulation. I honestly think with this and Vulkan, we will see close to matching native Windows performance. Great times indeed in the Linux world.

Thanks @damentz for the info on the getting merged into mainline. It would be a big plus if you merge it earlier like you said.

@damentz
Copy link
Member

damentz commented Aug 25, 2019

Well, it seems like discussion died down and there hasn't been any more versions. The patches in the LKML link have been merged to 5.2/master through 5.2/misc. If a new version comes out I'll refresh the patches.

@damentz damentz closed this as completed Aug 25, 2019
@damentz
Copy link
Member

damentz commented Aug 25, 2019

Nevermind, re-opening. Kernel fails to build for:

  CC      kernel/futex.o
kernel/futex.c: In function ‘do_futex_wait_multiple’:
kernel/futex.c:2708:7: error: implicit declaration of function ‘futex_setup_timer’ [-Werror=implicit-function-declaration]
 2708 |  to = futex_setup_timer(abs_time, &timeout, flags,
      |       ^~~~~~~~~~~~~~~~~
kernel/futex.c:2708:5: warning: assignment to ‘struct hrtimer_sleeper *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
 2708 |  to = futex_setup_timer(abs_time, &timeout, flags,
      |     ^
cc1: some warnings being treated as errors
make[1]: *** [scripts/Makefile.build:279: kernel/futex.o] Error 1
make: *** [Makefile:1081: kernel] Error 2

Looks like we need to backport futex patches from 5.3 WIP.

@damentz damentz reopened this Aug 25, 2019
@damentz
Copy link
Member

damentz commented Aug 25, 2019 •
edited

Resolved with f39367f, kernel builds successfully. Refactored patches into branch, 5.2/futex-backports

@damentz damentz closed this as completed Aug 25, 2019
@damentz damentz mentioned this issue Sep 24, 2019
Closed
heftig pushed a commit that referenced this issue Dec 31, 2019
@gregkh
media: vivid: media_device_cleanup was called too early
12d8deb 
[ Upstream commit 8ffd573 ]

Running the contrib/test/test-media script in v4l-utils with the vivid argument
will cause this kernel warning:

[  104.748720] videodev: v4l2_release
[  104.748731] ------------[ cut here ]------------
[  104.748750] DEBUG_LOCKS_WARN_ON(lock->magic != lock)
[  104.748790] WARNING: CPU: 6 PID: 1823 at kernel/locking/mutex.c:938 __mutex_lock+0x919/0xc10
[  104.748800] Modules linked in: rc_cec vivid v4l2_tpg videobuf2_dma_contig cec rc_core v4l2_dv_timings videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 videobuf2_common videodev mc vmw_balloon vmw_vmci button vmwgfx
[  104.748845] CPU: 6 PID: 1823 Comm: sleep Not tainted 5.4.0-rc1-test-no #150
[  104.748853] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
[  104.748867] RIP: 0010:__mutex_lock+0x919/0xc10
[  104.748878] Code: 59 83 e8 9a fc 16 ff 44 8b 05 23 61 38 01 45 85 c0 0f 85 ef f7 ff ff 48 c7 c6 a0 1f 87 82 48 c7 c7 a0 1e 87 82 e8 cd bb
f7 fe <0f> 0b e9 d5 f7 ff ff f6 c3 04 0f 84 3b fd ff ff 49 89 df 41 83 e7
[  104.748886] RSP: 0018:ffff88811a357b80 EFLAGS: 00010286
[  104.748895] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[  104.748902] RDX: 0000000000000003 RSI: 0000000000000004 RDI: ffffed102346af62
[  104.748910] RBP: ffff88811a357cf0 R08: ffffffff81217c91 R09: fffffbfff061c271
[  104.748917] R10: fffffbfff061c270 R11: ffffffff830e1383 R12: ffff8881a46103c0
[  104.748924] R13: 0000000000000000 R14: ffff8881a4614f90 R15: ffff8881a46153d0
[  104.748933] FS:  0000000000000000(0000) GS:ffff8881b6780000(0000) knlGS:0000000000000000
[  104.748940] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  104.748949] CR2: 00007f163fc9ca20 CR3: 0000000003013004 CR4: 00000000001606e0
[  104.749036] Call Trace:
[  104.749051]  ? _raw_spin_unlock+0x1f/0x30
[  104.749067]  ? llist_add_batch+0x33/0x50
[  104.749081]  ? tick_nohz_tick_stopped+0x19/0x30
[  104.749130]  ? v4l2_release.cold+0x6c/0xd6 [videodev]
[  104.749143]  ? mutex_lock_io_nested+0xb80/0xb80
[  104.749153]  ? vprintk_emit+0xf2/0x220
[  104.749191]  ? vivid_req_validate+0x40/0x40 [vivid]
[  104.749201]  ? printk+0xad/0xde
[  104.749211]  ? kmsg_dump_rewind_nolock+0x54/0x54
[  104.749226]  ? locks_remove_file+0x78/0x2b0
[  104.749248]  ? __fsnotify_update_child_dentry_flags.part.0+0x170/0x170
[  104.749281]  ? vivid_req_validate+0x40/0x40 [vivid]
[  104.749321]  ? v4l2_release.cold+0x6c/0xd6 [videodev]
[  104.749361]  v4l2_release.cold+0x6c/0xd6 [videodev]
[  104.749378]  __fput+0x15a/0x390
[  104.749393]  task_work_run+0xb2/0xe0
[  104.749407]  do_exit+0x4d0/0x1200
[  104.749422]  ? do_user_addr_fault+0x367/0x610
[  104.749431]  ? release_task+0x990/0x990
[  104.749449]  ? rwsem_spin_on_owner+0x170/0x170
[  104.749463]  ? vmacache_find+0xb2/0x100
[  104.749476]  do_group_exit+0x85/0x130
[  104.749487]  __x64_sys_exit_group+0x23/0x30
[  104.749500]  do_syscall_64+0x5e/0x1c0
[  104.749511]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  104.749520] RIP: 0033:0x7f163fc5c9d6
[  104.749536] Code: Bad RIP value.
[  104.749543] RSP: 002b:00007ffe6f3bec58 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  104.749553] RAX: ffffffffffffffda RBX: 00007f163fd4d760 RCX: 00007f163fc5c9d6
[  104.749560] RDX: 0000000000000000 RSI: 000000000000003c RDI: 0000000000000000
[  104.749567] RBP: 0000000000000000 R08: 00000000000000e7 R09: ffffffffffffff80
[  104.749574] R10: 00007ffe6f3beb24 R11: 0000000000000246 R12: 00007f163fd4d760
[  104.749581] R13: 0000000000000002 R14: 00007f163fd56428 R15: 0000000000000000
[  104.749597] ---[ end trace 66f20f73fc0daf79 ]---

This is caused by media_device_cleanup() which destroys
v4l2_dev->mdev->req_queue_mutex. But v4l2_release() tries to lock
that mutex after media_device_cleanup() is called.

By moving media_device_cleanup() to the v4l2_device's release function it is
guaranteed that the mutex is valid whenever v4l2_release is called.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
heftig pushed a commit that referenced this issue Dec 31, 2019
@gregkh
media: vicodec: media_device_cleanup was called too early
f5f8420 
[ Upstream commit 693c5f1 ]

Running the contrib/test/test-media script in v4l-utils with the vicodec argument
will cause this kernel warning:

[  372.298824] ------------[ cut here ]------------
[  372.298848] DEBUG_LOCKS_WARN_ON(lock->magic != lock)
[  372.298896] WARNING: CPU: 11 PID: 2220 at kernel/locking/mutex.c:938 __mutex_lock+0x919/0xc10
[  372.298907] Modules linked in: vicodec v4l2_mem2mem vivid rc_cec v4l2_tpg videobuf2_dma_contig cec rc_core v4l2_dv_timings videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev mc vmw_balloon vmw_vmci button vmwgfx [last unloaded: vimc]
[  372.298961] CPU: 11 PID: 2220 Comm: sleep Not tainted 5.4.0-rc1-test-no #150
[  372.298970] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
[  372.298983] RIP: 0010:__mutex_lock+0x919/0xc10
[  372.298995] Code: 59 83 e8 9a fc 16 ff 44 8b 05 23 61 38 01 45 85 c0 0f 85 ef f7 ff ff 48 c7 c6 a0 1f 87 82 48 c7 c7 a0 1e 87 82 e8 cd bb f7 fe <0f> 0b e9 d5 f7 ff ff f6 c3 04 0f 84 3b fd ff ff 49 89 df 41 83 e7
[  372.299004] RSP: 0018:ffff8881b400fb80 EFLAGS: 00010286
[  372.299014] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[  372.299022] RDX: 0000000000000003 RSI: 0000000000000004 RDI: ffffed1036801f62
[  372.299030] RBP: ffff8881b400fcf0 R08: ffffffff81217c91 R09: fffffbfff061c271
[  372.299038] R10: fffffbfff061c270 R11: ffffffff830e1383 R12: ffff88814761dc80
[  372.299046] R13: 0000000000000000 R14: ffff88814761cbf0 R15: ffff88814761d030
[  372.299055] FS:  0000000000000000(0000) GS:ffff8881b68c0000(0000) knlGS:0000000000000000
[  372.299063] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  372.299071] CR2: 00007f606d78aa20 CR3: 0000000003013002 CR4: 00000000001606e0
[  372.299153] Call Trace:
[  372.299176]  ? __kasan_slab_free+0x12f/0x180
[  372.299187]  ? kmem_cache_free+0x9b/0x250
[  372.299200]  ? do_exit+0xcdf/0x1200
[  372.299210]  ? do_group_exit+0x85/0x130
[  372.299220]  ? __x64_sys_exit_group+0x23/0x30
[  372.299231]  ? do_syscall_64+0x5e/0x1c0
[  372.299241]  ? entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  372.299295]  ? v4l2_release+0xed/0x190 [videodev]
[  372.299309]  ? mutex_lock_io_nested+0xb80/0xb80
[  372.299323]  ? find_held_lock+0x85/0xa0
[  372.299335]  ? fsnotify+0x5b0/0x600
[  372.299351]  ? locks_remove_file+0x78/0x2b0
[  372.299363]  ? __fsnotify_update_child_dentry_flags.part.0+0x170/0x170
[  372.299383]  ? vidioc_querycap+0x50/0x50 [vicodec]
[  372.299426]  ? v4l2_release+0xed/0x190 [videodev]
[  372.299467]  v4l2_release+0xed/0x190 [videodev]
[  372.299484]  __fput+0x15a/0x390
[  372.299499]  task_work_run+0xb2/0xe0
[  372.299512]  do_exit+0x4d0/0x1200
[  372.299528]  ? do_user_addr_fault+0x367/0x610
[  372.299538]  ? release_task+0x990/0x990
[  372.299552]  ? rwsem_spin_on_owner+0x170/0x170
[  372.299567]  ? vmacache_find+0xb2/0x100
[  372.299580]  do_group_exit+0x85/0x130
[  372.299592]  __x64_sys_exit_group+0x23/0x30
[  372.299602]  do_syscall_64+0x5e/0x1c0
[  372.299614]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  372.299624] RIP: 0033:0x7f606d74a9d6
[  372.299640] Code: Bad RIP value.
[  372.299648] RSP: 002b:00007fff65364468 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  372.299658] RAX: ffffffffffffffda RBX: 00007f606d83b760 RCX: 00007f606d74a9d6
[  372.299666] RDX: 0000000000000000 RSI: 000000000000003c RDI: 0000000000000000
[  372.299673] RBP: 0000000000000000 R08: 00000000000000e7 R09: ffffffffffffff80
[  372.299681] R10: 00007fff65364334 R11: 0000000000000246 R12: 00007f606d83b760
[  372.299689] R13: 0000000000000002 R14: 00007f606d844428 R15: 0000000000000000
[  372.299704] ---[ end trace add7d62ca4bc65e3 ]---

This is caused by media_device_cleanup() which destroys
v4l2_dev->mdev->req_queue_mutex. But v4l2_release() tries to lock
that mutex after media_device_cleanup() is called.

By moving media_device_cleanup() to the v4l2_device's release function it is
guaranteed that the mutex is valid whenever v4l2_release is called.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
damentz pushed a commit that referenced this issue Apr 27, 2022
@gregkh
jbd2: fix a potential race while discarding reserved buffers after an…
ae3fe72 
… abort

commit 23e3d7f upstream.

we got issue as follows:
[   72.796117] EXT4-fs error (device sda): ext4_journal_check_start:83: comm fallocate: Detected aborted journal
[   72.826847] EXT4-fs (sda): Remounting filesystem read-only
fallocate: fallocate failed: Read-only file system
[   74.791830] jbd2_journal_commit_transaction: jh=0xffff9cfefe725d90 bh=0x0000000000000000 end delay
[   74.793597] ------------[ cut here ]------------
[   74.794203] kernel BUG at fs/jbd2/transaction.c:2063!
[   74.794886] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[   74.795533] CPU: 4 PID: 2260 Comm: jbd2/sda-8 Not tainted 5.17.0-rc8-next-20220315-dirty #150
[   74.798327] RIP: 0010:__jbd2_journal_unfile_buffer+0x3e/0x60
[   74.801971] RSP: 0018:ffffa828c24a3cb8 EFLAGS: 00010202
[   74.802694] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[   74.803601] RDX: 0000000000000001 RSI: ffff9cfefe725d90 RDI: ffff9cfefe725d90
[   74.804554] RBP: ffff9cfefe725d90 R08: 0000000000000000 R09: ffffa828c24a3b20
[   74.805471] R10: 0000000000000001 R11: 0000000000000001 R12: ffff9cfefe725d90
[   74.806385] R13: ffff9cfefe725d98 R14: 0000000000000000 R15: ffff9cfe833a4d00
[   74.807301] FS:  0000000000000000(0000) GS:ffff9d01afb00000(0000) knlGS:0000000000000000
[   74.808338] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   74.809084] CR2: 00007f2b81bf4000 CR3: 0000000100056000 CR4: 00000000000006e0
[   74.810047] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   74.810981] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[   74.811897] Call Trace:
[   74.812241]  <TASK>
[   74.812566]  __jbd2_journal_refile_buffer+0x12f/0x180
[   74.813246]  jbd2_journal_refile_buffer+0x4c/0xa0
[   74.813869]  jbd2_journal_commit_transaction.cold+0xa1/0x148
[   74.817550]  kjournald2+0xf8/0x3e0
[   74.819056]  kthread+0x153/0x1c0
[   74.819963]  ret_from_fork+0x22/0x30

Above issue may happen as follows:
        write                   truncate                   kjournald2
generic_perform_write
 ext4_write_begin
  ext4_walk_page_buffers
   do_journal_get_write_access ->add BJ_Reserved list
 ext4_journalled_write_end
  ext4_walk_page_buffers
   write_end_fn
    ext4_handle_dirty_metadata
                ***************JBD2 ABORT**************
     jbd2_journal_dirty_metadata
 -> return -EROFS, jh in reserved_list
                                                   jbd2_journal_commit_transaction
                                                    while (commit_transaction->t_reserved_list)
                                                      jh = commit_transaction->t_reserved_list;
                        truncate_pagecache_range
                         do_invalidatepage
			  ext4_journalled_invalidatepage
			   jbd2_journal_invalidatepage
			    journal_unmap_buffer
			     __dispose_buffer
			      __jbd2_journal_unfile_buffer
			       jbd2_journal_put_journal_head ->put last ref_count
			        __journal_remove_journal_head
				 bh->b_private = NULL;
				 jh->b_bh = NULL;
				                      jbd2_journal_refile_buffer(journal, jh);
							bh = jh2bh(jh);
							->bh is NULL, later will trigger null-ptr-deref
				 journal_free_journal_head(jh);

After commit 96f1e09, we no longer hold the j_state_lock while
iterating over the list of reserved handles in
jbd2_journal_commit_transaction().  This potentially allows the
journal_head to be freed by journal_unmap_buffer while the commit
codepath is also trying to free the BJ_Reserved buffers.  Keeping
j_state_lock held while trying extends hold time of the lock
minimally, and solves this issue.

Fixes: 96f1e09("jbd2: avoid long hold times of j_state_lock while committing a transaction")
Signed-off-by: Ye Bin <yebin10@huawei.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Link: https://lore.kernel.org/r/20220317142137.1821590-1-yebin10@huawei.com
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@damentz damentz

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@damentz @shoober420