Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

False positives #6

Open
robredpath opened this issue Apr 8, 2014 · 45 comments
Open

False positives #6

robredpath opened this issue Apr 8, 2014 · 45 comments

Comments

@robredpath
Copy link

@robredpath robredpath commented Apr 8, 2014

Thanks for a very useful tool :D

However, we've had some issues using it whereby the website would produce false positives (ie, consider that a server was vulnerable when it wasn't) on servers the first time it was run, but not subsequently. This included servers running OpenSSL 0.9.8. Annoyingly, it's stopped happening now, but I couldn't see a commit where this issue was directly addressed, so I'm not sure if it's fixed itself (urgh) or if it's something fixed by the developer.

We've not seen this when running it on the command line.

Anyone else seen this? Anyone still seeing this?

@plentz
Copy link

@plentz plentz commented Apr 8, 2014

@robredpath I saw this a few minutes ago as well. 1 false positive e another false negative.

@plentz
Copy link

@plentz plentz commented Apr 8, 2014

update: now it seems to be giving green light for every site(tested 3 sites that I know that aren't ok yet and all 3 received "all good")

@robredpath
Copy link
Author

@robredpath robredpath commented Apr 8, 2014

Are you getting the same behaviour between the site and the CLI tool?

@morsik
Copy link

@morsik morsik commented Apr 8, 2014

I have a lot of false-positives using command line too.
If you run:

for i in $(cat ssl_hosts)
do
    (
        curl -s "http://localhost/bleed/$i" &
    )
done

you'll get also a lot of false positives. Local server can't handle this correcty.

When i do linear checking - results looks correct.

@robredpath
Copy link
Author

@robredpath robredpath commented Apr 8, 2014

Awesome. Thanks for your work on this, @FiloSottile !

That's slightly different from what I was seeing this morning - false reds followed by lots of correct greens.

If it's possible, could some mechanism to verify that the result displayed is definitely from my server be provided? I'm not entirely sure what that would look like but it would help immensely with confidence in results.

@FiloSottile
Copy link
Owner

@FiloSottile FiloSottile commented Apr 8, 2014

Be careful, I can't think of anything that could cause false reds.

More probably you have false greens, check better...

2014-04-08 15:56 GMT+02:00 Rob Redpath notifications@github.com:

Awesome. Thanks for your work on this, @FiloSottilehttps://github.com/FiloSottile!

That's slightly different from what I was seeing this morning - false reds
followed by lots of correct greens.

If it's possible, could some mechanism to verify that the result displayed
is definitely from my server be provided? I'm not entirely sure what that
would look like but it would help immensely with confidence in results.

Reply to this email directly or view it on GitHubhttps://github.com//issues/6#issuecomment-39850310
.

@malgorithms
Copy link

@malgorithms malgorithms commented Apr 8, 2014

I believe I saw a couple false positives on the site, too. @FiloSottile you said the command line tool is unaffected, and it's reporting keybase.io:443 as safe:

# passes repeated tests, no FAILS
>./Heartbleed keybase.io:443
2014/04/08 11:06:03 keybase.io:443 - SAFE

And so is the site now. But 20 minutes ago I got a FAIL on the site, reloaded a few more successes, and got a FAIL again. And someone tweeted at me they did too. Now I cannot reproduce this again.

Max updated to 1.0.1g last night.

@pielgrzym
Copy link

@pielgrzym pielgrzym commented Apr 8, 2014

Nice tool. I'm getting a false positive (red) via web ui / command line for a site running Openssl 0.9.8.

@FiloSottile
Copy link
Owner

@FiloSottile FiloSottile commented Apr 8, 2014

@malgorithms @pielgrzym

I can't honestly think of how a false positive (let's define it as a RED w/ dumped memory including YELLOW SUBMARINE) could happen.

Can you please provide hosts - logs?

@malgorithms
Copy link

@malgorithms malgorithms commented Apr 8, 2014

I can't reproduce it again, but what's your take on Keybase's server right now? We haven't changed anything in the last hour since someone else and I both saw the positive. Can you get a positive?

I never saw a false positive in the command line app. Just on the website. Is there a chance there was a bug (during heavy load? or a race condition?) where site A was getting site B's results back? That would in effect be a presentation bug...

@mboylan
Copy link

@mboylan mboylan commented Apr 8, 2014

I also saw a false positive a single time to a web server we have running OpenSSL 9.8. I cannot reproduce it on the website and the CLI tool is continuously reporting back that everything is safe. Unfortunately, for obvious reasons, I cannot share the URL of the server I was testing.

@egp
Copy link

@egp egp commented Apr 8, 2014

I did see one red fail on http://filippo.io/Heartbleed/#keybase.io:443 recently(after Chris announced it was fixed), but now all I get is green pass.

@egp
Copy link

@egp egp commented Apr 8, 2014

Oh, the Irony. http://filippo.io/Heartbleed/#openssl.org:443 gets a red fail

@malgorithms
Copy link

@malgorithms malgorithms commented Apr 8, 2014

Thanks @FiloSottile - I'll have the net tab open on future tests, but so far I can't get it to happen again.

@yourcelf
Copy link

@yourcelf yourcelf commented Apr 8, 2014

I have a debian 0.9.8o-4squeeze14 server that is reporting as vulnerable, both with the command line and the hosted service. That OpenSSL version /should/ be safe according to http://heartbleed.com. Either this tool or heartbleed.com's vulnerability table seems to be wrong.

@FiloSottile
Copy link
Owner

@FiloSottile FiloSottile commented Apr 8, 2014

@yourcelf Host please?

@yourcelf
Copy link

@yourcelf yourcelf commented Apr 8, 2014

@FiloSottile You have a private channel I can send to you?

@FiloSottile
Copy link
Owner

@FiloSottile FiloSottile commented Apr 8, 2014

@yourcelf My GH profile email, grab the PGP keys if you want

@r15ch13
Copy link

@r15ch13 r15ch13 commented Apr 8, 2014

I have the same problem as @yourcelf with version 0.9.8k-7ubuntu8.15

@yourcelf
Copy link

@yourcelf yourcelf commented Apr 8, 2014

OK, figured it out: a backported mod-spdy was pulling in a newer version of OpenSSL that was vulnerable.

@r15ch13
Copy link

@r15ch13 r15ch13 commented Apr 8, 2014

@yourcelf ha! disabling mod-spdy did the trick. thanks

@FiloSottile FiloSottile mentioned this issue Apr 8, 2014
@mmckinst
Copy link

@mmckinst mmckinst commented Apr 8, 2014

For people affected by mod-spdy, you'll need to disable mod_spdy.so and the mod_ssl_with_npn.so module it comes with. Then load mod_ssl.so instead of mod_ssl_with_npn.so .

mod-spdy bug report: https://code.google.com/p/mod-spdy/issues/detail?id=85

@bradspry
Copy link

@bradspry bradspry commented Apr 8, 2014

Tool reports red fail against Debian openssl 1.0.1e-2+deb7u6

But Debian security tracker reports openssl 1.0.1e-2+deb7u6 status as FIXED:
https://security-tracker.debian.org/tracker/CVE-2014-0160

Please advise.

@schlamar
Copy link

@schlamar schlamar commented Apr 8, 2014

@bradspry Did you forget to reboot after update? =)

@bradspry
Copy link

@bradspry bradspry commented Apr 8, 2014

Where I went wrong was using the 'apt-get install --only-upgrade openssl' method, which did not upgrade package 'libssl1.0.0'. Updating package 'openssl' was not enough... You must also update package 'libssl1.0.0'.

@takinbo
Copy link

@takinbo takinbo commented Apr 8, 2014

One other thing to note are binaries that have been statically compiled with vulnerable versions of the library. You want to make sure that those are checked as well.

I had an nginx binary that I had to replace to fix the problem.

@pielgrzym
Copy link

@pielgrzym pielgrzym commented Apr 8, 2014

Easiest way to check what @takinbo mentioned is to run nginx -V and look for custom library paths.

@cjolowicz
Copy link

@cjolowicz cjolowicz commented Apr 9, 2014

False positive for dealloc.org.

OpenSSL 0.9.8o (0.9.8o-4squeeze14)
Apache 2.2.16 (2.2.16-6+squeeze12)

([]uint8) {
 00000000  02 00 79 68 65 61 72 74  62 6c 65 65 64 2e 66 69  |..yheartbleed.fi|
 00000010  6c 69 70 70 6f 2e 69 6f  59 45 4c 4c 4f 57 20 53  |lippo.ioYELLOW S|
 00000020  55 42 4d 41 52 49 4e 45  a9 74 e8 72 ad 0d 71 7e  |UBMARINE.t.r..q~|
 00000030  28 0e 67 ef fe ab 05 c2  44 58 37 86 45 3b 61 af  |(.g.....DX7.E;a.|
 00000040  14 87 3c 78 70 71 14 d8  f2 05 78 f9 49 13 d5 54  |..<xpq....x.I..T|
 00000050  da d5 d3 72 cd 9e 1f 2f  3c eb fb 2f e6 80 90 37  |...r.../<../...7|
 00000060  76 97 40 7f fd 5c 6d ee  96 61 60 92 5b f3 86 f8  |v.@..\m..a`.[...|
 00000070  c5 7d 92 bd 2d 2c 9d ab  e1 8d 7d 7b 6c 84 07 80  |.}..-,....}{l...|
 00000080  50 29 91 7c b1 6c 08 c5  3e 35 3f 98              |P).|.l..>5?.|
}
@horsepunchkid
Copy link

@horsepunchkid horsepunchkid commented Apr 9, 2014

@cjolowicz And what version of libssl? Do you use mod-spdy as described above or something else that may be causing your server to use some other version? Did you restart apache?

@JackZielke
Copy link

@JackZielke JackZielke commented Apr 9, 2014

Mine was a positive positive with openssl 0.9.8 (Ubuntu lucid). I had installed mod_spdy which includes a patched, vulnerable, ssl. I did not have spdy enabled, just installed. Updating mod-spdy-beta provided me with a non-vulnerable ssl. I just thought I would mention this.

@paprika60
Copy link

@paprika60 paprika60 commented Apr 9, 2014

I checked a site and it is coming back consistantly red. But when I brought it to the attention of the technical team they said they aren't using OpenSSL. I'm not sure how to respond to them, and don't want to be a total dick if it's a false positive, but does that sound posible? A false positive if they aren't using OpenSSL?

@timayhelen
Copy link

@timayhelen timayhelen commented Apr 9, 2014

Upon my initial test, my website cam back as Red. All subsequent tests, however, return the timeout error. This is the expected result, due to the fact that I am using Litespeed and it blocks the request. I cannot figure out why the original test returned a red result, however.

I flushed my firewall blocks before re-testing, and each test comes back with the timeout error. All other tools that I use to test also confirm that the website is not vulnerable, such as:

https://www.ssllabs.com/ssltest/analyze.html

Do you have any idea why an initial test would come back positive? Could this be a false positive?

@iamthegrackle
Copy link

@iamthegrackle iamthegrackle commented Apr 9, 2014

I used the filippo tool and my site came back marked as Vulnerable. However when I got on a chat with my hosting company to ask them about patching it, after the tech checked my server her reported that:

I can see that your OpenSSL version is
root@vps-1123238-14690 [~]# rpm -q openssl
openssl-1.0.1e-16.el6_5.7.x86_64
The vulnerability is for
openssl-1.0.1e-16.el6_5.4.0.1 and below.
It seems you are not affected.

Is this correct?

@horsepunchkid
Copy link

@horsepunchkid horsepunchkid commented Apr 9, 2014

@iamthegrackle That's probably a question you should ask on a RedHat forum instead, but RedHat's security advisory suggests that your openssl version is good:

https://rhn.redhat.com/errata/RHSA-2014-0376.html (under "Updated packages", openssl-1.0.1e-16.el6_5.7.x86_64.rpm is mentioned as good)

You may have some other package that includes its own bad version of libssl. See the above comments for some examples.

@cjolowicz
Copy link

@cjolowicz cjolowicz commented Apr 9, 2014

@horsepunchkid I do not use mod-spdy. The version of libssl is 0.9.8o (0.9.8o-4squeeze14). I have not been able to reproduce the false positive, i.e. the host is now always reported as unaffected.

@davidkitfriedman
Copy link

@davidkitfriedman davidkitfriedman commented Apr 9, 2014

@timayhelen

Just wanted to add that the test from Qualys is seeking to test for Heartbleed. Wasn't sure about that at first.

@0100011101001000
Copy link

@0100011101001000 0100011101001000 commented Apr 10, 2014

I tested my OpenSSL using websites, all ok after patching. I had installed the Chrome extension though, and strangely enough, I happened to look down and it had found my IIS site support.ccnva.com as being vulnerable! Since I know this is not possible as IIS isn't effected, I just thought I would bring it up. Testing the main web tool does not show it as vulnerable, so the error is in the chrome extension.

@eryno
Copy link

@eryno eryno commented Apr 10, 2014

@gbhoover, IIS isn't affected, but OpenSSL could be somewhere else in your server stack. For example, an nginx load balancer. Read the section titled "Is it only sites on Apache and nginx that are affected?" in http://www.troyhunt.com/2014/04/everything-you-need-to-know-about.html for more info.

@tsilen
Copy link

@tsilen tsilen commented Apr 10, 2014

It would seem that there's some concurrency issue and the response is from time to time somehow leaked from one check to another, or something like that.

Queries directly to http://bleed-1161785939.us-east-1.elb.amazonaws.com/bleed/:

{"code":1,"data":"","error":"","host":"www.google.com"}
{"code":1,"data":"","error":"","host":"www.google.com"}
{"code":0,"data":"([]uint8) {\n 00000000 02 00 79 68 65 61 72 74 62 6c 65 65 64 2e 66 69 |..yheartbleed.fi|\n 00000010 6c 69 70 70 6f 2e 69 6f 59 45 4c 4c 4f 57 20 53 |lippo.ioYELLOW S|\n 00000020 55 42 4d 41 52 49 4e 45 4e b0 94 1e 5
b 12 db 2a |UBMARINEN...[.._|\n 00000030 f1 f0 2d 00 a3 af eb 83 83 e6 3d 3a 03 03 03 03 |..-.......=:....|\n 00000040 bc b3 72 46 5c 4f 91 80 2e 34 3a b5 52 3f 81 f3 |..rF\O...4:.R?..|\n 00000050 a6 58 0d 8d 69 07 e5 5e 30 4a d9 27
2e 76 7b 01 |.X..i..^0J.'.v{.|\n 00000060 61 07 9b 39 f5 e4 0d 10 5e 2b e7 53 4b b4 e2 fc |a..9....^+.SK...|\n 00000070 6b a6 98 9c df ac 98 2a 57 77 c5 12 26 b8 15 f1 |k......_Ww..\u0026...|\n 00000080 35 a6 f1 09 46 88 83 6b 7a a7
b4 70 |5...F..kz..p|\n}\n","error":"","host":"www.google.com"}
{"code":1,"data":"","error":"","host":"www.google.com"}
{"code":1,"data":"","error":"","host":"www.google.com"}

{"code":1,"data":"","error":"","host":"example.com"}
{"code":0,"data":"([]uint8) {\n 00000000 02 00 79 68 65 61 72 74 62 6c 65 65 64 2e 66 69 |..yheartbleed.fi|\n 00000010 6c 69 70 70 6f 2e 69 6f 59 45 4c 4c 4f 57 20 53 |lippo.ioYELLOW S|\n 00000020 55 42 4d 41 52 49 4e 45 d1 3c fa 8c 7
6 c0 c6 be |UBMARINE.\u003c..v...|\n 00000030 50 b0 50 fc e6 13 46 1b 32 46 30 34 25 32 46 32 |P.P...F.2F04%2F2|\n 00000040 30 31 34 26 64 61 74 65 54 6f 3d 31 32 25 32 46 |014\u0026dateTo=12%2F|\n 00000050 30 35 25 32 46 32 30 31 34
26 72 65 74 75 72 6e |05%2F2014\u0026return|\n 00000060 46 72 6f 6d 3d 26 72 65 74 75 72 6e 54 6f 3d 26 |From=\u0026returnTo=\u0026|\n 00000070 66 6c 79 44 61 79 73 25 35 42 25 35 cc 97 5f 49 |flyDays%5B%5.._I|\n 00000080 d5 06 99 ea
1c d6 74 3b 7e f1 e6 b6 |......t;~...|\n}\n","error":"","host":"example.com"}
{"code":1,"data":"","error":"","host":"example.com"}
{"code":1,"data":"","error":"","host":"example.com"}

I've seen it happen with multiple domains from time to time (not very often). Probably requires that somebody does a check against a vulnerable one at the same time or just before.

@iamthegrackle
Copy link

@iamthegrackle iamthegrackle commented Apr 12, 2014

@horsepunchkid

you said, "You may have some other package that includes its own bad version of libssl. See the above comments for some examples."

I have brought this thread and the examples mentioned to the attention of support at my hosting company, and they insist that everything is fine as long as I ran the server updates they outlined, which I did, yet my site still shows vulnerable using the filippo.io tool. The rep closed by saying,

"Also, I have tested from the following link and it says site is not vulnerable.

https://lastpass.com/heartbleed/?h=sales.mx

You can just ignore any warning from 3rd party sites as openSSL version in your server is already updated."

So either the filippo tool is wrong, or else they are totally disregarding the issues raised here and I would wonder just how many of the VPS customers at this very large host are still vulnerable ...

@FiloSottile
Copy link
Owner

@FiloSottile FiloSottile commented Apr 12, 2014

@iamthegrackle

That site is vulnerable. Full stop.

Feel free to link support to this comment and if they have different evidence they can get in contact with me over email. The address is in my profile.

@iamthegrackle
Copy link

@iamthegrackle iamthegrackle commented Apr 12, 2014

thanks. I let them know. they responded with a link to filippo.io with my IP address (ip::2087) instead of my domain name like I was doing. And when the 'ignore certificate' is checked, then the result is that it is fixed or unaffected.

So it my ip:2087 is used then it says fixed, if my domain is used then it says vulnerable.

@FiloSottile
Copy link
Owner

@FiloSottile FiloSottile commented Apr 12, 2014

Have a look at the usual: load balancers, SSL terminators, different
host-based setup, CDN, mod_SPDY...

2014-04-12 23:29 GMT+02:00 Michael notifications@github.com:

thanks. I let them know. they responded with a link to filippo.io with my
IP address (ip::2087) instead of my domain name like I was doing. And when
the 'ignore certificate' is checked, then the result is that it is fixed or
unaffected.

So it my ip:2087 is used then it says fixed, if my domain is used then it
says vulnerable.

Reply to this email directly or view it on GitHubhttps://github.com//issues/6#issuecomment-40292266
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.