ssl-cert NSE can cause elastic parsing exceptions

[tophertimzen]

Affecting:

Server

Version:
5fc55c4

Describe the bug

When an ssl cert is parsed by the Natlas server from the result of the ssl-cert NSE, if there are blank fields where it is expected for them to be filled in elastic search will have exceptions.

To Reproduce

1. Stand up a new natlas server and agents
2. Add some items to scope
3. add ssl-cert to agent namp scripts
4. Wait for an SSL cert to come back with missing fields, and elastic search will error

Expected behavior
Natlas should handle

Screenshots
See exception below

Additional context

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/opt/natlas/natlas-server/app/auth/wrappers.py", line 44, in decorated_function
    return f(*args, **kwargs)
  File "/opt/natlas/natlas-server/app/api/routes.py", line 151, in submit
    current_app.elastic.new_result(newhost)
  File "/opt/natlas/natlas-server/app/elastic/interface.py", line 59, in new_result
    self.client.execute_index(index='nmap_history', body=host)
  File "/opt/natlas/natlas-server/app/elastic/client.py", line 111, in execute_index
    results = self._execute_raw_query(self.es.index, doc_type='_doc', **kwargs)
  File "/opt/natlas/natlas-server/app/elastic/client.py", line 119, in _execute_raw_query
    return func(**kwargs)
  File "/usr/lib/python3.6/site-packages/elasticsearch/client/utils.py", line 76, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/usr/lib/python3.6/site-packages/elasticsearch/client/__init__.py", line 319, in index
    _make_path(index, doc_type, id), params=params, body=body)
  File "/usr/lib/python3.6/site-packages/elasticsearch/transport.py", line 318, in perform_request
    status, headers_response, data = connection.perform_request(method, url, params, body, headers=headers, ignore=ignore, timeout=timeout)
  File "/usr/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 186, in perform_request
    self._raise_error(response.status, raw_data)
  File "/usr/lib/python3.6/site-packages/elasticsearch/connection/base.py", line 125, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: RequestError(400, 'mapper_parsing_exception', 'failed to parse field [ports.ssl.notAfter] of type [date]')

[0xdade]

I just got a saved failure of this tonight, it appears to happen when nmap detects the ssl cert to have a notAfter date of 1-01-01T00:00:00 according to the xml. I'll have to write a special case for this probably. I manually connected to the port and checked the certificate in the browser and sure enough the period of validity section:

Begins On    January 2, 1
Expires On   January 2, 1

[0xdade]

I'm inclined to mark these fields with ignore_malformed in elastic. I can parse them to datetime with dateutil which will convert them automatically to 2001-01-01 00:00:00, but I believe this is an error considering what other parsers say the field is (the year 1).

I'd rather just ignore the bad data for these two particular date fields, since we can't control them and I'd rather them be empty than to be parsed incredibly incorrectly.

@ajacques thoughts on marking fields as ignore_malformed in the mapping? I know you have an open issue to cleanup our mappings.