Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Add fall-through for bytesToHuman for very large values (just print the bytes) #858

Closed
wants to merge 2 commits into from

2 participants

@dvdplm

We have seen an issue where an instance had a very large peak memory usage: used_memory_peak:18446744073706301432. The bytesToHuman() function can't stringify that properly (we get: used_memory_peak_human:\x90\f\x83\xFB\x83\u007F) so here's a version that resorts to simply printing the bytes for too-large values.

@charsyam

@dvdplm I just have a question about your environment. Do you really use more than 15EB memory?

It is because 15EB overflow double's range. but, I really wonder in your environment.

@dvdplm

It's an AWS m2.xlarge instance, so yeah, it has 17.1Gb. There are plenty of 64Gb servers out there, so it's not like this is a particularly exotic situation right?

@charsyam

Hmm. it's somewhat weird. because "used_memory_peak" is more than 15EB not 15GB in your report.

@dvdplm

Hehe, you're right, it is a weirdly high value (and I have no explanation for it, unfortunately). Maybe that's another unrelated bug?
Either way, I think the bytesToHuman should be able to deal with any kind of crazy value. I mean, it's simply a string formatting helper function and it should simply work and not encode/require any knowledge about what "valid" values are here.

@charsyam

@dvdplm Yes, I think that it is first to look for what makes it strange.

In my personal opinion, "used_memory_peak" already shows not encoded value.

so, I think adding "TB", "PB", "EB", "ZB", "YB" is better.

@dvdplm

@charsyam from PB onwards I'm getting compiler warnings about comparison of unsigned expression < 0 is always false and potential division by zero issues, so I stopped at TB and PB (and kept the final rescue with a note saying it's an exceptional case). You like better?

@charsyam

@dvdplm maybe there is no possibility to use more than PB size memory in next 10 years. :)
I just worry about what makes "used_memory_peak" weird.

@dvdplm dvdplm referenced this pull request in redis/redis-rb
Closed

Downgrade INFO string to ASCII #306

@mattsta mattsta referenced this pull request from a commit in mattsta/redis
@dvdplm dvdplm Extend range of bytesToHuman to TB and PB
Also adds a fallthrough case for when given
large values (like overflow numbers of 2^64 by mistake).

Closes #858
27b8bd7
@mattsta mattsta referenced this pull request from a commit in mattsta/redis
@dvdplm dvdplm Extend range of bytesToHuman to TB and PB
Also adds a fallthrough case for when given
large values (like overflow numbers of 2^64 by mistake).

Closes #858
6d1b51d
@mattsta mattsta referenced this pull request
Closed

ALL simple issue fixes #1906

@mattsta mattsta referenced this pull request from a commit in mattsta/redis
@dvdplm dvdplm Extend range of bytesToHuman to TB and PB
Also adds a fallthrough case for when given
large values (like overflow numbers of 2^64 by mistake).

Closes #858
80c24cd
@antirez antirez closed this pull request from a commit
@dvdplm dvdplm Extend range of bytesToHuman to TB and PB
Also adds a fallthrough case for when given
large values (like overflow numbers of 2^64 by mistake).

Closes #858
100c331
@antirez antirez closed this in 100c331
@antirez antirez referenced this pull request from a commit
@dvdplm dvdplm Extend range of bytesToHuman to TB and PB
Also adds a fallthrough case for when given
large values (like overflow numbers of 2^64 by mistake).

Closes #858
55672e0
@antirez antirez referenced this pull request from a commit
@dvdplm dvdplm Extend range of bytesToHuman to TB and PB
Also adds a fallthrough case for when given
large values (like overflow numbers of 2^64 by mistake).

Closes #858
8c0743d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
Showing with 9 additions and 0 deletions.
  1. +9 −0 src/redis.c
View
9 src/redis.c
@@ -1890,6 +1890,15 @@ void bytesToHuman(char *s, unsigned long long n) {
} else if (n < (1024LL*1024*1024*1024)) {
d = (double)n/(1024LL*1024*1024);
sprintf(s,"%.2fG",d);
+ } else if (n < (1024LL*1024*1024*1024*1024)) {
+ d = (double)n/(1024LL*1024*1024*1024);
+ sprintf(s,"%.2fT",d);
+ } else if (n < (1024LL*1024*1024*1024*1024*1024)) {
+ d = (double)n/(1024LL*1024*1024*1024*1024);
+ sprintf(s,"%.2fP",d);
+ } else {
+ /* Let's hope we never need this */
+ sprintf(s,"%lluB",n);
}
}
Something went wrong with that request. Please try again.