Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

7.7+ issue with ssl-cert returning certificate when http-title returns "401 Authorization Required" #1685

Open
jr0792 opened this issue Aug 11, 2019 · 14 comments

Comments

@jr0792
Copy link

commented Aug 11, 2019

On version 6.4 ssl-cert (nmap -sV -sC) will return a certificate when the http-title returns "401 Authorization Required" but on verison 7.7 or 7.8 it stops after the title and does not return a certificate. This occurs on every device that has a pop-up sign in window before you can access it's HTTP page. I have to continue running version 6.4 until this is corrected because this severely limits Nmaps abilities.

Example of 'nmap -sV -sC'
Version 6.4:

PORT STATE SERVICE VERSION
21/tcp open ftp oftpd
| ftp-anon: Anonymous FTP login allowed (FTP code 202)
|Can't get directory listing: Can't parse PASV response: "Command not implemented."
23/tcp open telnet?
80/tcp open http?
| http-auth:
| HTTP/1.0 401 Unauthorized
|
Digest nonce=5ff518ef19b57b3722f6253570cf155a stale=FALSE algorithm=MD5 opaque= realm=Use 'live' as User Name in order to log in to the respective level qop=auth
|_http-methods: No Allow or Public header in OPTIONS response (status code 400)
|_http-title: 401 Authorization Required
443/tcp open ssl/openvas OpenVAS server
|_http-methods: No Allow or Public header in OPTIONS response (status code 400)
|http-title: 401 Authorization Required
| ssl-cert: Subject: commonName=local.myboschcam.net/organizationName=Bosch Sicherheitssysteme GmbH/stateOrProvinceName=Bayern/countryName=DE/localityName=Grasbrunn
| Issuer: commonName=local.myboschcam.net/organizationName=Bosch Sicherheitssysteme GmbH/stateOrProvinceName=Bayern/countryName=DE/localityName=Grasbrunn
| Public Key type: rsa
| Public Key bits: 1024
| Not valid before: 2012-11-05T13:05:29+00:00
| Not valid after: 2015-11-05T13:05:29+00:00
| MD5: a013 caa0 6b80 0788 c507 93e3 29fd 04b2
| SHA-1: 7c82 82ae 3d92 e609 75da 3e46 9d0f f957 896c 3512
| -----BEGIN CERTIFICATE-----
| MIICwDCCAimgAwIBAgIJAIl4oj8F35HTMA0GCSqGSIb3DQEBBQUAMHkxCzAJBgNV
| BAYTAkRFMQ8wDQYDVQQIDAZCYXllcm4xEjAQBgNVBAcMCUdyYXNicnVubjEmMCQG
| A1UECgwdQm9zY2ggU2ljaGVyaGVpdHNzeXN0ZW1lIEdtYkgxHTAbBgNVBAMMFGxv
| Y2FsLm15Ym9zY2hjYW0ubmV0MB4XDTEyMTEwNTEzMDUyOVoXDTE1MTEwNTEzMDUy
| OVoweTELMAkGA1UEBhMCREUxDzANBgNVBAgMBkJheWVybjESMBAGA1UEBwwJR3Jh
| c2JydW5uMSYwJAYDVQQKDB1Cb3NjaCBTaWNoZXJoZWl0c3N5c3RlbWUgR21iSDEd
| MBsGA1UEAwwUbG9jYWwubXlib3NjaGNhbS5uZXQwgZ8wDQYJKoZIhvcNAQEBBQAD
| gY0AMIGJAoGBALNVdJWLOjs6zTjYkB8zNdWppgdBzIwTYtLGSmEqe2l0i5asbHIM
| DSZk+9OcuVh0kasup4AO6w7Q9hG3E2gX2mnNWIhOXnIJ9No3EsseWgLYN0R6Vda1
| srjsjAFHcfsZoFibmybsOtOPR61uPWzlikJ1HSXoLXY8MSq/vHWhBnszAgMBAAGj
| UDBOMB0GA1UdDgQWBBTK669jgSwEYeAjMR1YpmkR6v9y7zAfBgNVHSMEGDAWgBTK
| 669jgSwEYeAjMR1YpmkR6v9y7zAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBBQUA
| A4GBACuMVPYOlnXaWJWxRD87WoFHBrbfLMocgTAoahbpH0kzL9MCDbWSCx3cS5LT
| lu6ty0Vuh2ih1pauTWfzdKJnYt0znVxuHNs4vzgOqv8ntjzVkJT5105pdF8zQOQ2
| F+0TAeLeIUKBXwzhwVnPvFzPg2ZAVnBlw3pKj3PgRSEQAdFI
|
-----END CERTIFICATE-----
554/tcp open rtsp Apple AirTunes rtspd
|rtsp-methods: ERROR: Script execution failed (use -d to debug)
3260/tcp open iscsi?
| iscsi-info:
| Target: iqn.2005-12.com.bosch:unit00075f7f9419.trg-000-sdcard
| Address: xxxxx:3260,1
|
Authentication: Authentication failure
2 services unrecognized despite returning data. If you know the service/version, please submit the following fingerprints at http://www.insecure.org/cgi-bin/servicefp-submit.cgi :

Version 7.8:

PORT STATE SERVICE REASON VERSION
21/tcp open ftp syn-ack ttl 61 oftpd
|ftp-anon: ERROR: Script execution failed (use -d to debug)
|ftp-bounce: ERROR: Script execution failed (use -d to debug)
|ftp-syst: ERROR: Script execution failed (use -d to debug)
23/tcp open telnet syn-ack ttl 61 HP LaserJet telnetd
| fingerprint-strings:
| GenericLines:
| Welcome to NDC-265-P xx.xx.xx.xx from xx.xx.xx.xx
| telnet last socket error 0x00000133
| telnet -1 !!!
| invalid username
| enter username ->
| GetRequest:
| Welcome to NDC-265-P xx.xx.xx.xx from xx.xx.xx.xx
| second telnet session from xx.xx.xx.xx discarded
| HTTP
| input too long
| again:
| Help:
| Welcome to NDC-265-P xx.xx.xx.xx from xx.xx.xx.xx
| second telnet session from xx.xx.xx.xx discarded
| HELP
| invalid username
| enter username ->
| NCP:
| Welcome to NDC-265-P xx.xx.xx.xx from xx.xx.xx.xx
| second telnet session from xx.xx.xx.xx discarded
| DmdT
| NULL:
| Welcome to NDC-265-P xx.xx.xx.xx from xx.xx.xx.xx
| telnet last socket error 0x00000133
|
telnet -1 !!!
80/tcp open http syn-ack ttl 61 Bosch DINION IP Bullet 5000 webcam http admin
| http-auth:
| HTTP/1.0 401 Unauthorized\x0D
|
Digest stale=FALSE algorithm=MD5 realm=Use 'live' as User Name in order to log in to the respective level nonce=347e12f33487322b7f6522720a2d5433 opaque= qop=auth
| http-methods:
|
Supported Methods: GET POST
|_http-title: 401 Authorization Required
443/tcp open ssl/openvas syn-ack ttl 61 OpenVAS server
|_ssl-date: 2019-08-11T09:00:08+00:00; -5h07m37s from scanner time.
554/tcp open rtsp syn-ack ttl 61 Apple AirTunes rtspd
|rtsp-methods: ERROR: Script execution failed (use -d to debug)
3260/tcp open iscsi? syn-ack ttl 61
| iscsi-info:
| iqn.2005-12.com.bosch:unit00075f7f9419.trg-000-sdcard:
| Address: xx.xx.xx.xx:3260,1
| Authentication: required
|
Auth reason: Authentication failure
1 service unrecognized despite returning data. If you know the service/version, please submit the following fingerprint at https://nmap.org/cgi-bin/submit.cgi?new-service :

@jr0792

This comment has been minimized.

Copy link
Author

commented Aug 11, 2019

Here are the scans with more detailed debug parameters:
nmap -n -T3 -d9 -Pn -sS -p 443 --script ssl-cert ip address

Attached as txt files:
Version 6.4: Scan6dot4Good.txt
Scan6dot4Good.txt

Version 7.8: Scan7dot8Error.txt
Scan7dot8Error.txt

@jr0792

This comment has been minimized.

Copy link
Author

commented Aug 11, 2019

I installed every version of Nmap and determined 6.47 is the first version the issue begins to occur.

@nnposter

This comment has been minimized.

Copy link

commented Aug 12, 2019

I have sampled a handful of sites with the following scan:

nmap -n -Pn -sT -T2 -p443 --script ssl-cert -d <target>

All of the sites return HTTP status 401 while script ssl-cert worked as expected, returning the certificate.

The test environments were:

  • nmap 7.70 and 7.80 on Ubuntu 18.04 x64 (Azure)
  • nmap 7.70SVN on Ubuntu 18.04 x64 (VMware)
  • nmap 7.70SVN on Win7 x64 (physical)

These results suggest that the issue is not universal but rather related to your scanning environment or specific targets.

The following part of your output is the key:

NSOCK DEBUG FULL [30.2900s] epoll_loop(): wait for events
NSOCK DEBUG FULL [30.2910s] epoll_loop(): wait for events
NSOCK DEBUG FULL [30.2910s] epoll_loop(): wait for events
NSOCK DEBUG FULL [30.2910s] epoll_loop(): wait for events
NSOCK DEBUG FULL [30.2910s] process_event(): Processing event 9 (timeout in 0ms, done=0)
NSOCK DEBUG FULL [30.2910s] process_event(): NSE #9: Sending event
NSOCK INFO [30.2910s] nsock_trace_handler_callback(): Callback: SSL-CONNECT TIMEOUT for EID 9 [xx.xx.xx.xx:443]

This shows that the HTTP status plays no role here. The TLS connection never completed the initial handshake.

@jr0792

This comment has been minimized.

Copy link
Author

commented Aug 12, 2019

What is odd is that version 6.46 works but 6.47+ does not.

curl -vvIk https://_ipaddress_
and
openssl s_client -connect _ipaddress_:443
both work as expected all the time.

I am running CentOS 7.6.1810 on VMware. I have also tried it on Windows 10 x64 physical.

I can provide -d 9 if that would help determine the cause.

@nnposter

This comment has been minimized.

Copy link

commented Aug 12, 2019

The detailed log will probably not help. I have casually reviewed the SVN commits between 6.46 and 6.47, noting these culprit candidates:

  • The OpenSSL library got upgraded
  • Cipher suites offered during the TLS handshake have changed

I would suggest the following:

  1. Run nmap 7.80 with -sT, instead of -sS, just to make sure that basic TCP connectivity works.
  2. Assuming it does, run nmap script ssl-enum-ciphers, determining which cipher suites and TLS/SSL versions are supported by the target.
  3. If your target is publicly facing you might also use third-party services, such as SSL Labs, for the same.
  4. Compare these results with what nmap uses when making TLS connections.
@jr0792

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

Below is the result of the ssl-enum-ciphers. Is there a list of what cipher suites are offered?

PORT     STATE SERVICE REASON
21/tcp   open  ftp     syn-ack ttl 61
23/tcp   open  telnet  syn-ack ttl 61
80/tcp   open  http    syn-ack ttl 61
443/tcp  open  https   syn-ack ttl 61
| ssl-enum-ciphers: 
|   TLSv1.0: 
|     ciphers: 
|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 1024) - A
|       TLS_RSA_WITH_DES_CBC_SHA (rsa 1024) - D
|     compressors: 
|       NULL
|     cipher preference: client
|     warnings: 
|       64-bit block cipher DES vulnerable to SWEET32 attack
|       Weak certificate signature: SHA1
|_  least strength: D
554/tcp  open  rtsp    syn-ack ttl 61
3260/tcp open  iscsi   syn-ack ttl 61
@nnposter

This comment has been minimized.

Copy link

commented Aug 13, 2019

At first glance the cipher suites do not appear to be a problem. The first one, TLS_RSA_WITH_AES_128_CBC_SHA, is normally offered by Nmap.

There are two known issues related to connecting to some older TLS implementations:

  • Some servers do not work properly when a client proposes TLS 1.2 but the server only supports 1.0.
  • Some servers choke up when a client offers "too many" cipher suites.

The second case you can generally test for by trying to connect with OpenSSL. This typically works:

openssl s_client -connect <target>

However, the following does not:

openssl s_client -connect <target> -cipher ALL

For the first case you need to inspect the Nmap TLS handshake in Wireshark, looking for symptoms described in issue #511.

@jr0792

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

Ok I tested those two issues. The second case had 0 errors with both openssl connections. The first case I am seeing Client Hello with no Server Hello. I attached a screenshot below. Does this mean that I will be unable to use Nmap 6.47+ if I would like to retrieve certificates from hosts that only support TLS 1.0?

image

@nnposter

This comment has been minimized.

Copy link

commented Aug 13, 2019

No, it does not. If a client proposes TLS 1.2 then the server is free to reply with any version up to 1.2, such as 1.0. It is then up to the client to decide if it is willing to continue with lower version. These days Nmap typically accept TLS 1.0, 1.1, and 1.2. Support for SSLv3 is spotty, depending on the OpenSSL library used.

What it does mean is that some TLS implementations are not fully compliant with the TLS standard and Nmap is not compensating for all the quirks that are out there.

@jr0792

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

That doesn't make sense though. Nmap is using OpenSSL, which has no issues getting the certificate from this host. I have captured Wireshark while running OpenSSL and Nmap on the same host. OpenSSL which Client Hello and Server Hello while Nmap has issues. This was on Windows 10 x64.

This is OpenSSL 1.1.1c running openssl s_client -connect <target> -cipher ALL :
OpenSSL_CipherAll_Poll

This is Nmap 7.70 running nmap -sV -p 443 --script ssl-cert ipaddress :
NmapSSL_Poll

If Nmap is using OpenSSL then why does it have an issue with this? Beyond that, Nmap is a tool used for finding vulnerabilities so wouldn't it make sense to at least report or exploit that TLS is out of date?

@jr0792

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

I can actually see the certificate inside of Wireshark when I run nmap -sV -p 443 --script ssl-cert ipaddress.

@nnposter

This comment has been minimized.

Copy link

commented Aug 13, 2019

I do not believe I can help with throubleshooting the issue any further without having access to the target. I have validated that the original claim of "This (issue) occurs on every device" is not true.

@jr0792

This comment has been minimized.

Copy link
Author

commented Aug 13, 2019

@nnposter I would be willing to provide you with network access to the host, or mail you the hardware as it's very small if you would like to help further.

@nnposter

This comment has been minimized.

Copy link

commented Aug 16, 2019

I cannot make promises that I will be able to help but feel free to send me a pertinent IP address and port.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.