Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]:【python】the data appears as garbled characters on Redis. #2260

Closed
YmaYeah opened this issue Jun 28, 2023 · 14 comments · Fixed by #2681
Closed

[Bug]:【python】the data appears as garbled characters on Redis. #2260

YmaYeah opened this issue Jun 28, 2023 · 14 comments · Fixed by #2681
Assignees
Labels
issue Something isn't working

Comments

@YmaYeah
Copy link

YmaYeah commented Jun 28, 2023

Preconditions (Any important steps we need to know)
1.【not sure】pip install django-redis
2.【must be】Configurate file configuration(in file settings)
CACHES = {
"default": {
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://127.0.0.1:6379",
"OPTIONS": {
"CLIENT_CLASS": "django_redis.client.DefaultClient",
"CONNECTION_POOL_KWARGS": {"max_connections": 100}
}
}
}

Steps to reproduce (How to reproduce what you found step by step)
1.[Configurate url]
urlpatterns = [path('redis_test/',RedisTestView.as_view())]

2.[Configurate view]
class RedisTestView(APIView):
def get(self,request):
cache.set('star',[1,2,3])
return ApiResponse()

3.[Test in Postman]
http://127.0.0.1:8000/api/v1/home/redis_test/
Actual behavior (A short description of what you found)
the data appears as garbled characters on Redis
key:":1:star" val:'����]�(K�K�K�e.'
Expected behavior (A short description of what you expected to find)
key:":1:star" val:"[1, 2, 3]"by pickle.loads

Screenshots (Paste or drag-and-drop a screenshot or a link to a recording)
image

Additional context (Operating system, version of RedisInsight, Redis database version, Redis module version, database type, connection type, logs, or any other information)
Operating system:macOS Ventura 13.0
version of RedisInsight:2.26.0
Redis database version:7.0.11 (bottled), HEAD
Redis module version:redis 7.0.11_1
connection type:NoSQL
version of Django:3.2.12
version of django-redis:5.3.0

@YmaYeah YmaYeah added the issue Something isn't working label Jun 28, 2023
@YmaYeah
Copy link
Author

YmaYeah commented Jun 28, 2023

Please make sure it's correct, thank you!

@ViktarStarastsenka
Copy link
Collaborator

hi @YmaYeah , can you please share an example of this data (but please remove all the sensitive information)

@YmaYeah
Copy link
Author

YmaYeah commented Jun 28, 2023

luffy_api.zip
pls check luffy_api--->apps--->home--->views.py--->RedisTestView(APIView)

@YmaYeah
Copy link
Author

YmaYeah commented Jun 28, 2023

hi @YmaYeah , can you please share an example of this data (but please remove all the sensitive information)
done ,pls check

@ViktarStarastsenka
Copy link
Collaborator

@YmaYeah , can you please provide just an example of the data you see with garbled characters? it will be helpful to look at it in ASCII format

@YmaYeah
Copy link
Author

YmaYeah commented Jun 28, 2023

@YmaYeah , can you please provide just an example of the data you see with garbled characters? it will be helpful to look at it in ASCII format

okkkk, I write [1,2,3]it is a list,redis shows 8004950b000000000000005d94284b014b024b03652e by HEX
����]�(K�K�K�e. by Unicode
\x80\x04\x95\x0b\x00\x00\x00\x00\x00\x00\x00]\x94(K\x01K\x02K\x03e.by ASCII

@ViktarStarastsenka
Copy link
Collaborator

thank you, we need to look deeper into this, so it may take some time

@DhanushR24
Copy link

Hi there,
I am facing this issue as well.

Also, on hovering on the "garbled characters" it shows Failed to convert to Pickle message.

@rad-pat
Copy link

rad-pat commented Oct 9, 2023

The problem I experience with this that results in the Failed to convert to Pickle is that my keys are stored pickled with Protocol 5 in Python and the library pickleparser used for RedisInsight does not currently support Protocol 5.
https://github.com/ewfian/pickleparser/blob/main/SUPPORTED_OPCODES.md

@GnaneshKunal
Copy link
Collaborator

Hey @rad-pat,

You are right with it. The library we currently use doesn't support Pickle Protocol 5.

@ewfian
Copy link
Contributor

ewfian commented Oct 14, 2023

Protocol 5 will be supported in https://github.com/ewfian/pickleparser/pull/4/files

@GnaneshKunal
Copy link
Collaborator

@YmaYeah and @rad-pat, the pickle parser library now supports pickle protocol 5. Can you please try out the views again?

@ViktarStarastsenka
Copy link
Collaborator

a new version (with support for pickle protocol 5) is not available right now, it will be released with 2.36 in 1-2 weeks.

@ViktarStarastsenka
Copy link
Collaborator

available with 2.36 (released today)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
issue Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants