Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Active Defrag not working for me #5188

Closed
timeassistant opened this Issue Jul 30, 2018 · 6 comments

Comments

Projects
None yet
3 participants
@timeassistant
Copy link

commented Jul 30, 2018

Hello,

I have installed Redis 4.0.10 from Remi Repo, I then uninstalled Jemalloc via RPM so that I could remove that without removing Redis. I then downloaded the Redis Source and extracted the Jemalloc dependancy and installed it. However I cannot activate active defrag.

Also Redis still seems to think it is using jemalloc 3.6.0 despite me uninstalling that and installing the newer version.

Did I miss something? or Is there a spefic place I should have installed Jemalloc? I initially uploaded the folder to /opt/ and ran from there but I'm guessing that was wrong now

Is there a full step by step for how to replace Jemalloc with the one referred to in the docs for Active Defrag? I feel like I have done it wrong

Thanks

@oranagra

This comment has been minimized.

Copy link
Contributor

commented Jul 31, 2018

Hi,
Redis is usually compiled in a way that it statically links with jemalloc, and not dependent of an external one.
I'm not sure how they built it, but you can see what dynamic libraries it needs by running ldd redis-server.

just to be clear, you're getting this error, Active defragmentation cannot be enabled: it requires a ..., right?

this actually means that redis itself was compiled without active defrag support, since at the time it was compiled, the header files with the modified jemalloc were not available.

@timeassistant

This comment has been minimized.

Copy link
Author

commented Jul 31, 2018

Hi there,

Thank you for the response @oranagra . You are correct I am getting that message

127.0.0.1:6379> config set activedefrag yes
(error) ERR Active defragmentation cannot be enabled: it requires a Redis server compiled with a modified Jemalloc like the one shipped by default with the Redis source distribution

So I guess what you are saying is that what I was trying basically isn't possible. I'd need to install Redis completely from source rather than using a yum repo like EPEL or Remi?

I presume the only method that will work is to completely remove Redis and then build all from source? The reason I want to avoid that is because I want to have the active defrag enabled on a production server. If I need to manually upgrade redis each time then I will obviously have more down time and there is a greater possibility for things to go wrong (even if I do test the upgrade via test server) Where as updating via YUM has its clear advantages - assuming you have an up to date repo

@oranagra

This comment has been minimized.

Copy link
Contributor

commented Jul 31, 2018

what did ldd show?
i'm not sure how they build redis and why is active defrag not included there..

i can understand that if they use an external (dynamic link) jemalloc, then they can't use defrag.. but redis doesn't normally built / tested this way.

for instance, redis 5.0 is usually built and tested only with jemalloc 5.0.1.

so especially if they're using static link, i don't see any reason defrag will not be supported.

maybe you should look into that (how they build redis), and / or contact them.

@timeassistant

This comment has been minimized.

Copy link
Author

commented Jul 31, 2018

Hello again

Please see ldd output below - it had me stumped for a while until I realised I had to feed it the full redis path!

[root@test-server /]# ldd /usr/bin/redis-server
        linux-vdso.so.1 =>  (0x00007ffd85f13000)
        libm.so.6 => /lib64/libm.so.6 (0x00007faef98b2000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007faef96ae000)
        libjemalloc.so.1 => /lib64/libjemalloc.so.1 (0x00007faef947a000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007faef925e000)
        libc.so.6 => /lib64/libc.so.6 (0x00007faef8e91000)
        /lib64/ld-linux-x86-64.so.2 (0x00007faef9ed4000)

When you say how they build redis, do you mean the maintainers of the Repos such as Remi? I have actually already posted in the Remi forums to see if they can do anything as remi seems to be the only one who has a repo with redis 4.x in it at the moment !

Well I don't think I'll be building Jemalloc from source again any time soon. The Make file doesnt seem to support make uninstall so its refusing to remove itself nicely ! The other reason I like using Yum ! :P

Edit: Just saw a post on Remi's site - The repo is on standby until mid aug as Remi is moving house ! So I doubt I will get any response til then sadly !

Par Remi le vendredi 20 juillet 2018, 07:46 - General

Holidays time !

This evening I will be offline, my computers dismantled, on the way to a new house.

The move will take some time, as the cleaning of the flat, and the resettlement in the new house. I should be back online around mid of August, hoping the 4G router will keep its promises (quite hard, used to have a fiber connection).

The repository will be on standby during this period.

@oranagra

This comment has been minimized.

Copy link
Contributor

commented Jul 31, 2018

yes, i meant the maintainer(s) and since we see they're using a dynamically linked jemalloc, and obviously not one containing the defrag patch I added to the jemalloc inside the redis source folder, the redis they build cannot be used with defrag.

i would have advised Remi to build redis in a way that it contains embedded (statically linked) jemalloc... which i consider the standard / recommended way.

your only other option is seems to be to build redis yourself, and in that case you don't need to worry about building/installing/uninstalling jemalloc. the redis makefile will make the static lib, and there's no need to install it.

@timeassistant

This comment has been minimized.

Copy link
Author

commented Jul 31, 2018

Yeah ok this makes sense now - I guess I was just being too hopeful in my attempt to replace default jemalloc with your one from redis source.

The whole reason I was doing this in the first place (after testing) was for a production server as I said above so I will need to hold off and hope that Remi will be interested in building Redis from your source with modified jemalloc and adding that to the Repo.

I personally find make files can be rather problematic especially when make uninstall doesn't work!! so I'll need to just make do without active defrag for now.

I have linked this issue onto my forum post on Remi. So whenever Remi is back online hopefully this will be considered.

The forums post is here incase anyone would like to add their comments or track it etc.
https://forum.remirepo.net/viewtopic.php?pid=10111

@antirez antirez closed this Aug 1, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.