Elasticsearch Instrumentation: datetime object not JSON serializable

[dougbyrne]

Describe the bug: Sampled ES queries that use a datetime object in the query body raise a TypeError: datetime.date() is not JSON serializable exception.

To Reproduce

1. In a sampled transaction, make a ES call that uses a datetime object in the query body.
2. Results in an exception inside of the APM instrumentation code.

Sample stack trace:

File "/srv/env/local/lib/python2.7/site-packages/elasticapm/instrumentation/packages/base.py", line 107, in call_if_sampling
  return self.call(module, method, wrapped, instance, args, kwargs)
File "/srv/env/local/lib/python2.7/site-packages/elasticapm/instrumentation/packages/elasticsearch.py", line 145, in call
  return wrapped(*args, **kwargs)
File "/srv/env/local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 71, in _wrapped
  return func(*args, params=params, **kwargs)
File "/srv/env/local/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 388, in search
  params=params, body=body)
File "/srv/env/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 276, in perform_request
  status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/srv/env/local/lib/python2.7/site-packages/elasticapm/instrumentation/packages/base.py", line 107, in call_if_sampling
  return self.call(module, method, wrapped, instance, args, kwargs)
File "/srv/env/local/lib/python2.7/site-packages/elasticapm/instrumentation/packages/elasticsearch.py", line 47, in call
  query.append(json.dumps(body["query"]))
File "/usr/lib/python2.7/json/__init__.py", line 244, in dumps
  return _default_encoder.encode(obj)
File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
  chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
  return _iterencode(o, 0)
File "/usr/lib/python2.7/json/encoder.py", line 184, in default
  raise TypeError(repr(o) + " is not JSON serializable")

TypeError: datetime.date(2018, 12, 1) is not JSON serializable

Expected behavior: ...

Environment (please complete the following information)

• OS: Ubuntu 14.04
• Python version: Python 2.7
• Framework and version [e.g. Django 2.1]: Django==1.10.8
• APM Server version: 6.4.0
• Agent version: elastic-apm==3.0.3

Additional context

I've created a fork and patch that we are using internally: shareablee@0099b66

I can submit that as a PR if that is the direction you want to go, and I manage to get the CLA signed. The CLA site doesn't appear to be working for me right now.

[beniwohli]

@dougbyrne great catch, thanks! Your patch makes sense, but we should probably use elasticapm.utils.compat.text_type instead of str to ensure compatibility with Python 2.7. A PR would be awesome!

[dougbyrne]

OK, I've changed the type and opened a PR #403

However, I still can't get the CLA site to work for me. I pick an option and it doesn't load an agreement to sign.

[beniwohli]

That's awesome! Thanks!

I notified the team that runs the CLA checker, they are looking into it. I'll let you know when they resolved the issue :)

[beniwohli]

@dougbyrne the CLA checker should be back in action again :)

[dougbyrne]

@beniwohli We've submitted the CLA for my company.

[beniwohli]

@dougbyrne sorry to bother you again with this. Unfortunately it seems like the CLA signature didn't make it into our record due to a configuration issue on our side :( could you try again? Very sorry for this hassle!

[dougbyrne]

@beniwohli Looking at the company CLA, the form doesn't have spots for the individual's email and github usernames.

[beniwohli]

@dougbyrne I just checked again, and it seems like the CLA check on #403 passed now. Very confusing, sorry about this

[dougbyrne]

I re-signed under my personal email too, as I'm not sure how the contribution would appear to the check.