I often see MS SQL running on non-default ports when exposed externally over TCP and it would be good if Nmap could identify it without having to specify --version-all. Rarity 8 to me implies something uncommon, but Microsoft SQL Server is a very popular product.
I also agree with djcater. I often find myself running --version-all scans to get the version of SQL servers in odd ports. Unless anyone else disagrees, i think we should decrease the rarity level of the TCP probe to be at least 7 (default).
@cldrn Anyone could introduce the progress of determining the rarity of probe? Or any detailed data which could help us make accurate value on it? While I have found it changes rapidly according to network.
@jkryanchou "rarity" is poorly named. It ought to be something more like "threshold" because it is the level at which Nmap will send the probe to a non-ports-matching port. Some factors that go into deciding a probe's rarity are
likelihood of getting a response from many services
relative prevalence of services that give good (version-distinguishable) responses to that probe.
By now, Nmap has so many different probes that most of the time adding a new one is only useful in cases of services that simply will not respond to any other probe. When these services are nearly always on the same port number, giving these new probes a high (8 or more) rarity ensures they are not sent to other ports unless the user increases --version-intensity. Most services that do not respond to the first 7 levels of probes will not respond to any of the others, either.
All that said, I think we can reduce this to 7. It would be the 28th probe to be sent at that level or sooner. I do ask that you help us out if you have experience finding these services:
If there is/are some other port/s that is often hosting this service, let us know so we can add it to the ports line. This lets Nmap send the probe much sooner in the scan, making for faster scan times.
If (without this modification) Nmap prints a service fingerprint for submission, please either submit it online or paste it here so we can make softmatch lines in any other probes that may elicit a response. This speeds up matching by letting Nmap eliminate probes that are unlikely to also match the same service name.
@dmiller-nmap OK, Thanks for your detailed answer. I have got to know what the rarity means. While Is there any continuous integration system internal for our submission fingerprint？How to set the proper rarity from those fingerprint we submitted. Any manual scripts we could probably do it by ourself locally?
Rarity level decreased to 7 in r36956 for MSSQL. For common probes setting the rarity level to 7 is fine. When the service submissions get processed the values could be adjusted if they need to be. Feel free to send any question about other service signatures that you consider should be updated.