FS#3366 - router keeps running out of memory, have to power cycle it periodically, need help figuring out why. #6675
Comments
|
wrighrc: Sorry for the poor formatting. I wish I could fix that but it seems I can't edit my own post. Here's a comparison from yesterday and this morning.
Wed Sep 30 08:23:26 EDT 2020
08:23:26 up 10 days, 10:05, load average: 0.06, 0.01, 0.01
Mem: 101308K used, 22464K free, 844K shrd, 1980K buff, 4356K cached
CPU: 0% usr 0% sys 0% nic 100% idle 0% io 0% irq 0% sirq
Load average: 0.06 0.02 0.02 2/71 25149
PID PPID USER STAT VSZ %VSZ %CPU COMMAND
2175 2174 root S 4912 4% 0% ocserv-sm
2174 1 root S 4740 4% 0% ocserv-main
931 1 root S 2324 2% 0% /sbin/rpcd -s /var/run/ubus.sock -t 30
2001 1 root S 1784 1% 0% /usr/sbin/hostapd -s -P /var/run/wifi-phy0.pid -B /var/run/hostapd-phy0.conf
1077 1 root S 1748 1% 0% /sbin/netifd
1 0 root S 1572 1% 0% /sbin/procd
PID USER VSZ STAT COMMAND
2174 root 4740 S ocserv-main
2175 root 4912 S ocserv-sm
total used free shared buff/cache available
Mem: 123772 95004 22432 844 6336 1588
Swap: 262140 1544 260596
MemTotal: 123772 kB
MemFree: 22376 kB
MemAvailable: 1532 kB
Buffers: 1980 kB
Cached: 4356 kB
SwapCached: 360 kB
Active: 4904 kB
Inactive: 3432 kB
Active(anon): 908 kB
Inactive(anon): 1936 kB
Active(file): 3996 kB
Inactive(file): 1496 kB
Unevictable: 0 kB
Mlocked: 0 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 123772 kB
LowFree: 22376 kB
SwapTotal: 262140 kB
SwapFree: 260596 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 1672 kB
Mapped: 2228 kB
Shmem: 844 kB
Slab: 11800 kB
SReclaimable: 1968 kB
SUnreclaim: 9832 kB
KernelStack: 624 kB
PageTables: 352 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 324024 kB
Committed_AS: 6680 kB
VmallocTotal: 1040376 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
|
|
kofec: In my case it was this: |
|
bjonglez: The commit you mention fixes something that had not been backported to 19.07. So, it seems surprising that it would fix your issue. Can you give more details about this "fix"? |
aparcar
added
the
release/19.07
wrighrc:
Supply the following if possible:
netgear r6350
OpenWrt 19.07.4 r11208-ce6496d796 / LuCI openwrt-19.07 branch git-20.247.75781-0d0ab01
I installed ocserv too but I don't think its the problem
Just let router run, maybe will stay up for a week and then at some point gui tells me can't fork, I can't login to router with ssh and I have to power cycle it.
I added swap space on a USB stick to try to keep it from locking up.
I've ssh'ed into and run the following to try to figure out what's going on.
I expected to find a process growing but nothing looks obvious to me. I just see free memory keep going down, and I see a bit of swap space used.
Any suggestions on what else I could do to figure out what's going on with the memory usage?
looking at meminfo it shows about ~100M used but the numbers don't seem to add up to that.
while sleep 60 do date uptime top -b -n 1 | head -n 10 ps w | awk '$3 > 4000' free -m cat /proc/meminfo printf "\n" doneHere's what 10 days looks like:
Wed Sep 30 00:56:16 EDT 2020 00:56:16 up 10 days, 2:38, load average: 0.00, 0.02, 0.00 Mem: 100276K used, 23496K free, 1076K shrd, 1832K buff, 4600K cached CPU: 0% usr 0% sys 0% nic 100% idle 0% io 0% irq 0% sirq Load average: 0.01 0.02 0.00 2/71 27832 PID PPID USER STAT VSZ %VSZ %CPU COMMAND 27831 7096 root R 1224 1% 0% top -b -n 1 2175 2174 root S 4912 4% 0% ocserv-sm 2174 1 root S 4740 4% 0% ocserv-main 931 1 root S 2324 2% 0% /sbin/rpcd -s /var/run/ubus.sock -t 30 2001 1 root S 1784 1% 0% /usr/sbin/hostapd -s -P /var/run/wifi-phy0.pid -B /var/run/hostapd-phy0.conf 1077 1 root S 1748 1% 0% /sbin/netifd PID USER VSZ STAT COMMAND 2174 root 4740 S ocserv-main 2175 root 4912 S ocserv-sm total used free shared buff/cache available Mem: 123772 93796 23544 1076 6432 2604 Swap: 262140 1024 261116 MemTotal: 123772 kB MemFree: 23500 kB MemAvailable: 2560 kB Buffers: 1832 kB Cached: 4600 kB SwapCached: 232 kB Active: 5184 kB Inactive: 3364 kB Active(anon): 1120 kB Inactive(anon): 2072 kB Active(file): 4064 kB Inactive(file): 1292 kB Unevictable: 0 kB Mlocked: 0 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 123772 kB LowFree: 23500 kB SwapTotal: 262140 kB SwapFree: 261116 kB Dirty: 0 kB Writeback: 0 kB AnonPages: 1960 kB Mapped: 2136 kB Shmem: 1076 kB Slab: 11756 kB SReclaimable: 1912 kB SUnreclaim: 9844 kB KernelStack: 624 kB PageTables: 308 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 324024 kB Committed_AS: 6680 kB VmallocTotal: 1040376 kB VmallocUsed: 0 kB VmallocChunk: 0 kBThe text was updated successfully, but these errors were encountered: