Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Binary data broken? #74

Open
jgwinner opened this issue Oct 25, 2020 · 3 comments
Open

Binary data broken? #74

jgwinner opened this issue Oct 25, 2020 · 3 comments

Comments

@jgwinner
Copy link

Is your feature request related to a problem? Please describe.
I can't store binary data in Redis via cpp_redis, although I can with other languages

Describe the solution you'd like
Is there a way to call "set" with something other than a std::string / i.e. a raw buffer and size?

Describe alternatives you've considered
I'm temporarily (for about 40 seconds) storing some PNG files into Redis. Unfortunately, the .set command uses std::string. Normally I would applaud this, but in this case it's causing a double allocation and a corrupt file.

ALL of the std::string initializers either walk the string and set every character to zero, or do a deep copy and null terimate the copy. They also null terminate the string. A binary file, which IS compatible with Redis, doesn't have to be null terminated. If I copy a PNG file with a null at the end, it may cause the file to be corrupt.

I'm dealing with a few hundred K, with the largest about a meg and a half, so creating a C++ string by allocating bytes that will subsequently be immedietely overwritten by OpenGL buffer copies seems wasteful.

So essentially, cpp_redis cannot store data that Redis is capable of storing? Or is there a way around this?

I looked at std::string_view but I can't quite coerce that into std:string. I'm using QT library, which does have a way to initialize a raw buffer to a QString, but I found out that .toStdString will also do a deep copy to put the null terminated string.

Regarding other languages, for example, with php/Predis:

$client = new Predis\Client();
$client->set('profile_picture', file_get_contents("profile.png"));//value is binary

Additional context

I'm using OpenGL to create some small thumbnails for a graphical editor. Right now, it can take up to 30 seconds at hosting to open a webcache and dump the data there, and we're using the premium service. It's mostly as there are dozens of small files, and each "file/open" takes a long time. We were attempting to create a Redis Cashe for this data until the data is persisted to disk. We are would lose clients dramatically if every change took 30 seconds until it was visible.

@jgwinner
Copy link
Author

jgwinner commented Nov 2, 2020

Anyone? :(

@jgwinner jgwinner changed the title Can cpp_redis store binary data? Binary data broken? Nov 9, 2020
@jgwinner
Copy link
Author

jgwinner commented Nov 9, 2020

It was a critical feature for us. Taking a look at a QT library that does Redis.

@larkwiot
Copy link

larkwiot commented Mar 1, 2023

I think you can create a std::string from bytes data?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants