Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement: Implement Redis cache backend #534

Closed
cofin opened this issue Sep 30, 2022 · 7 comments
Closed

Enhancement: Implement Redis cache backend #534

cofin opened this issue Sep 30, 2022 · 7 comments
Assignees
Labels
Enhancement This is a new feature or request

Comments

@cofin
Copy link
Member

cofin commented Sep 30, 2022

What's the feature you'd like to ask for.
Implement a Redis cache backend that can be utilized as an alternate to the SimpleBackend.

Additional context
We should probably extend the CacheProtocol and SimpleCacheBackend to have additional methods that are similar to redis.

For instance, we could add some or all the following implementations.

  • scan,keys,rpush,expire,exists, mget, lrange, delete_keys, ping.

Here is a working Redis backend with the methods above from the sample project in my github:

class RedisAsyncioBackend(CacheBackendProtocol):  # pragma: no cover
    """Cache Backend for Redis"""

    async def get(self, key: str) -> "Any":  # pylint: disable=invalid-overridden-method
         return await redis.get(key)

    async def set(  # pylint: disable=invalid-overridden-method
        self, key: str, value: "Any", expiration: "Optional[Union[int, timedelta]]"
    ) -> "Any":
         return await redis.set(key, value, expiration)

    async def delete(self, key: str) -> "Any":  # pylint: disable=invalid-overridden-method
        return await redis.delete(key)

    async def scan(self, match: str, count: "Optional[int]" = None) -> "Tuple[int, list]":
        """Execute Redis SCAN command with pattern matching ."""
        return await redis.scan(match=match, count=count)

    async def keys(self, pattern: str) -> "list":
        return await redis.keys(pattern)

    async def rpush(self, key: str, value: str, expiration: "Optional[Union[int, timedelta]]" = None) -> int:
        if expiration:
            async with redis.pipeline(transaction=True) as pipe:
                push = await pipe.rpush(key, value)
                await pipe.expire(key, expiration).execute()
                return cast("int", push)
        else:
            return await redis.rpush(key, value)

    async def expire(self, key: str, expiration: "Union[int, timedelta]") -> bool:
        return await redis.expire(key, expiration)

    async def exists(self, key: str) -> bool:
        cnt = await redis.exists(key)
        return cnt > 0

    async def mget(self, keys: "List[str]") -> "list[str | None]":
        return await redis.mget(keys)

    async def lrange(self, key: str, start: int, end: int) -> "list":
        return await redis.lrange(key, start, end)

    async def delete_keys(self, pattern: str) -> int:
        async with redis.pipeline(transaction=True) as pipe:
            keys = await pipe.keys(pattern)
            deleted = await pipe.delete(*keys).execute()
            return cast("int", deleted)

    async def ping(self) -> bool:
        try:
            return await redis.ping()
        except RedisError:
            return False
@cofin cofin added Enhancement This is a new feature or request Triage Required 🏥 This requires triage and removed Triage Required 🏥 This requires triage labels Sep 30, 2022
@cofin
Copy link
Member Author

cofin commented Sep 30, 2022

The integration should also include hiredis in addition to redis for performance

@Goldziher
Copy link
Contributor

Should see have redis at all? Why not simply to with hiredis to begin with?

@cofin
Copy link
Member Author

cofin commented Sep 30, 2022

I thought hiredis integrated with the redis library, but I could be wrong.

@seladb
Copy link
Contributor

seladb commented Oct 1, 2022

@cofin I think I can work on this unless you already started, please let me know

@cofin
Copy link
Member Author

cofin commented Oct 1, 2022

@seladb, please go for it.

@seladb seladb self-assigned this Oct 1, 2022
@seladb
Copy link
Contributor

seladb commented Oct 12, 2022

@cofin @Goldziher basic redis and memcached support was added.

As @cofin mentioned we can extend the existing cache to do more than get(), set() and delete() (and extend the various backends accordingly). We can use Django cache as a reference. Would you like me to work on that or do you think it's not needed or important?

@Goldziher
Copy link
Contributor

@cofin @Goldziher basic redis and memcached support was added.

As @cofin mentioned we can extend the existing cache to do more than get(), set() and delete() (and extend the various backends accordingly). We can use Django cache as a reference. Would you like me to work on that or do you think it's not needed or important?

Since this is now user facing through request.cache I'm fine with extending it. But we need to make it simple and solid so the protocol cannot become too bloated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement This is a new feature or request
Projects
None yet
Development

No branches or pull requests

3 participants