Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mieru v1.9.0 oom-kill #44

Closed
foobartech opened this issue Dec 14, 2022 · 4 comments
Closed

mieru v1.9.0 oom-kill #44

foobartech opened this issue Dec 14, 2022 · 4 comments
Labels
bug Something isn't working

Comments

@foobartech
Copy link

在路由器 (Redmi Router AX6S 256M 内存 openwrt 系统) 上运行 mieru，一开始内存占用较低，随时间逐渐增加并最终导致 oom-kill，考虑是否存在内存泄漏问题。

系统日志：

Wed Dec 14 22:37:32 2022 kern.warn kernel: [114296.480768] mieru invoked oom-killer: gfp_mask=0x100dca(GFP_HIGHUSER_MOVABLE|__GFP_ZERO), order=0, oom_score_adj=0
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.491245] CPU: 0 PID: 2476 Comm: mieru Not tainted 5.10.138 #0
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.497329] Hardware name: Xiaomi Redmi Router AX6S (DT)
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.502717] Call trace:
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.505242]  0xffffffc01000a310
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.508460]  0xffffffc01000a4d4
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.511679]  0xffffffc0107f5b50
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.514897]  0xffffffc0107f418c
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.518115]  0xffffffc01011bf50
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.521332]  0xffffffc01011c938
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.524550]  0xffffffc01015f75c
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.527767]  0xffffffc01015fb74
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.530986]  0xffffffc010146e2c
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.534203]  0xffffffc0100167c8
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.537421]  0xffffffc010016aa4
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.540638]  0xffffffc010016c6c
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.543857]  0xffffffc0107fb154
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.547074]  0xffffffc0107fb584
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.550292]  0xffffffc0100025c8
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.553609] Mem-Info:
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.555994] active_anon:16456 inactive_anon:17931 isolated_anon:0
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.555994]  active_file:5 inactive_file:8 isolated_file:0
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.555994]  unevictable:0 dirty:0 writeback:0
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.555994]  slab_reclaimable:1998 slab_unreclaimable:6606
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.555994]  mapped:969 shmem:2768 pagetables:417 bounce:0
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.555994]  free:4645 free_pcp:0 free_cma:0
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.587724] Node 0 active_anon:65824kB inactive_anon:71724kB active_file:20kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:3840kB dirty:0kB writeback:0kB shmem:11072kB writeback_tmp:0kB kernel_stack:1728kB all_unreclaimable? yes
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.611165] DMA free:18580kB min:16384kB low:20480kB high:24576kB reserved_highatomic:4096KB active_anon:65824kB inactive_anon:71724kB active_file:140kB inactive_file:152kB unevictable:0kB writepending:0kB present:262144kB managed:233328kB mlocked:0kB pagetables:1668kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.639481] lowmem_reserve[]: 0 0 0 0
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.643229] DMA: 499*4kB (UMEH) 93*8kB (UMEH) 14*16kB (UME) 2*32kB (UE) 2*64kB (M) 3*128kB (UM) 1*256kB (M) 1*512kB (U) 4*1024kB (ME) 3*2048kB (ME) 1*4096kB (M) = 18644kB
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.658557] 2769 total pagecache pages
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.662384] 0 pages in swap cache
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.665782] Swap cache stats: add 62488, delete 62488, find 204/10973
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.672298] Free swap  = 0kB
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.675261] Total swap = 115708kB
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.678653] 65536 pages RAM
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.681523] 0 pages HighMem/MovableOnly
Wed Dec 14 22:37:32 2022 kern.debug kernel: [114296.685440] 7204 pages reserved
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.688658] Tasks state (memory values in pages):
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.693439] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.702150] [    925]    81   925      314       25    36864        8             0 ubusd
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.710412] [    926]     0   926      212        2    32768       11             0 askfirst
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.718933] [    961]     0   961      246        2    28672       14             0 urngd
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.727301] [   1354]   514  1354      305       17    32768       21             0 logd
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.735478] [   1408]     0  1408      553        2    36864       65             0 rpcd
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.743654] [   1645]     0  1645      262       13    32768        3             0 dropbear
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.752170] [   1755]     0  1755      644        2    36864       25             0 hostapd
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.760605] [   1756]     0  1756      644        2    32768       25             0 wpa_supplicant
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.769648] [   1778]   101  1778     1127       14    40960       19             0 wpa_supplicant
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.778691] [   1780]   101  1780     1179       41    40960       14             0 hostapd
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.787129] [   1820]     0  1820      460       17    28672       32             0 netifd
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.795479] [   1865]     0  1865      367       23    36864        9             0 odhcpd
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.803828] [   2005]     0  2005      403        2    32768       39             0 uhttpd
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.812171] [   2328]     0  2328      251       10    32768        7             0 odhcp6c
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.820607] [   2450]     0  2450      644        2    32768       27             0 ntpd
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.828781] [   2462]   123  2462      312       13    32768        2             0 ntpd
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.888416] [  11052]     0 11052      328       19    28672       18             0 pppd
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.913114] [   2472]     0  2472      104       51    32768        3             0 ipt2socks
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.921724] [   2473]     0  2473   231713    30805   593920    19566             0 mieru
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.946768] [   2703]     0  2703      644        2    36864       27             0 dnsmasq
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.955203] [   2709]   453  2709      361       32    36864        8             0 dnsmasq
Wed Dec 14 22:37:32 2022 kern.info kernel: [114296.972635] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=mieru,pid=2473,uid=0
Wed Dec 14 22:37:32 2022 kern.err kernel: [114296.985108] Out of memory: Killed process 2473 (mieru) total-vm:926852kB, anon-rss:120588kB, file-rss:4kB, shmem-rss:2628kB, UID:0 pgtables:580kB oom_score_adj:0

mieru 运行日志 (DEBUG) ：
20221214_0347_2473_脱敏.log
@enfein enfein added the bug Something isn't working label Dec 14, 2022
@enfein
Copy link
Owner

enfein commented Dec 14, 2022

按一个内存页 4 KB 大小来计算，120 MB 的 RSS 有些太大了。

从日志上看

[metrics - connections] ActiveOpens=2755 CurrEstablished=1078 MaxConn=1078 PassiveOpens=0

可能是网络连接泄露导致的。我会想办法用容器集成测试重现这个问题。

此外，客户端为了防止镜面反射攻击有一个重放缓存。极限情况下会存储 3.6M 个 int64。这个对 openwrt 的内存占用是不小的。考虑到我没有找到 GFW 有实施这种攻击的证据，或许默认去除客户端的重放缓存是个好主意。关于镜面反射攻击：shadowsocks/shadowsocks-org#183

@foobartech
Copy link
Author

此外，客户端为了防止镜面反射攻击有一个重放缓存。极限情况下会存储 3.6M 个 int64。这个对 openwrt 的内存占用是不小的。考虑到我没有找到 GFW 有实施这种攻击的证据，或许默认去除客户端的重放缓存是个好主意。

或者默认保留缓存，给一个option，小内存使用场景的用户可以按需关闭，毕竟大部分用户应该是使用 SagerNet 客户端的，安卓手机内存不是问题。

@enfein
Copy link
Owner

enfein commented Dec 16, 2022

@foobartech 我尝试修复了这个问题，在集成测试环境里，内存占用从 300 MB 下降到了 2 MB。

请用 https://github.com/enfein/mieru/actions/runs/3709833071 里面的安装包验证一下。如果没有问题，会推出下个版本。

@foobartech
Copy link
Author

测试了几小时，内存占用没有持续升高的现象，确认已经修复了，感谢！

enfein added a commit that referenced this issue Dec 16, 2022
@enfein
v1.9.1 release
28b2a04 
1. Fix UDP associate connection leak (issue #44).
2. Provide official armv7 build for old openwrt devices.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@foobartech @enfein