[Bug]: Redis crashes after attaching scope when setting value.

[the-data-sherpa]

Steps To Reproduce

1. Install Redis apt install redis -y

2. Fresh install of Appscope https://appscope.dev/docs/downloading

3. Confirm Redis is running
   redis 1218513 1 0 17:25 ? 00:00:00 /usr/bin/redis-server 127.0.0.1:6379

4. Attach Appscope to the process
   

5. Set value in Redis and cause Redis to crash (new PID)
   

6. Same behavior when using GET command for Redis
   

7. No issue when running commands without Appscope attached
   

From Redis-server.log

1219969:M 29 Jan 2023 17:33:58.058 # Server initialized
1219969:M 29 Jan 2023 17:33:58.058 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1219969:M 29 Jan 2023 17:33:58.058 * Loading RDB produced by version 6.0.16
1219969:M 29 Jan 2023 17:33:58.058 * RDB age 560 seconds
1219969:M 29 Jan 2023 17:33:58.058 * RDB memory usage when created 0.77 Mb
1219969:M 29 Jan 2023 17:33:58.058 * DB loaded from disk: 0.000 seconds
1219969:M 29 Jan 2023 17:33:58.058 * Ready to accept connections


=== REDIS BUG REPORT START: Cut & paste starting from here ===
1219969:M 29 Jan 2023 17:35:41.400 # Redis 6.0.16 crashed by signal: 11, si_code: 2
1219969:M 29 Jan 2023 17:35:41.400 # Crashed running the instruction at: 0x7f0942a4a140
1219969:M 29 Jan 2023 17:35:41.400 # Accessing address: 0x7f0942a4a140
1219969:M 29 Jan 2023 17:35:41.400 # Failed assertion: <no assertion failed> (<no file>:0)

Environment

root@NGinx:/opt/appscope# ./scope version
Version: v1.2.2
Build Date: 2023-01-11T15:23:50Z

root@NGinx:/opt/appscope# cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.1 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

root@NGinx:/opt/appscope# uname -m
x86_64

root@NGinx:/opt/appscope# uname -r
5.15.0-58-generic

Requested priority

Medium

Relevant log output

No response

[michalbiesek]

Hi @the-data-sherpa
Thank You for reaching us and reporting this issue, thanks for the excellent description and reproduction steps.

I was able to reproduce Your issue.

From the description You have provided, I concluded that in Your environment the redis is running as a service.
If I am correct regarding Your settings, one of Your setting is MemoryDenyWriteExecute=true which impacts the AppScope logic.
As a workaround, You can disable MemoryDenyWriteExecute in for redis service:

# Find if redis service is running
systemctl status redis
# `/lib/systemd/system/redis-server.service`  in next line is extracted from `Loaded: loaded` entry presented in redis service status above
cat /lib/systemd/system/redis-server.service
# Check `MemoryDenyWriteExecute` settings - by default it should be `true`
grep MemoryDenyWriteExecute /lib/systemd/system/redis-server.service
# Please edit the entry above to MemoryDenyWriteExecute=false
sudo systemctl daemon-reload
sudo systemctl restart redis

Details description of the issue (developer notes):

Redis-server received SIGSEGV on following part of code

EXPORTOFF ssize_t
write(int fd, const void *buf, size_t count)
{
    WRAP_CHECK(write, -1);
    if (g_ismusl == FALSE) {
        return __write_libc(fd, buf, count); <-----
    }

After the attach I was able to figure out that the following code fails:

rc = funchook_install(funchook, 0); 
ERROR: failed to install SSL_read hook. (Failed to protect page 0x7fa724aaa000 (size=4096, error=Operation not permitted))

The message above can be misleading but the fun hook applies also to __write_libc. Our code must cover the case when funchook install fails

@the-data-sherpa I will get back to You.

[the-data-sherpa]

Thank you for the quick work on providing the workaround. You are correct that redis is running as a service on my server(s). I've validated the setting as mentioned by you, and have made the appropriate modification. After running a reload-daemon and restarting the redis service, I have validated that the redis process no longer crashes when attaching AppScope.