Object of type datetime is not JSON serializable

[shivaprasad573]

When try to sync the date field am getting below error

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/mongo_connector/util.py", line 107, in wrapped
    func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/mongo_connector/oplog_manager.py", line 292, in run
    docman.upsert(doc, ns, timestamp)
  File "/usr/local/lib/python3.8/dist-packages/mongo_connector/util.py", line 33, in wrapped
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/solr_doc_manager-0.1.0-py3.8.egg/mongo_connector/doc_managers/solr_doc_manager.py", line 269, in upsert
    self.solr.add([self._clean_doc(doc, namespace, timestamp)],
  File "/home/shiva/.local/lib/python3.8/site-packages/pysolr.py", line 1018, in add
    m = json.dumps(message).encode("utf-8")
  File "/usr/lib/python3/dist-packages/simplejson/__init__.py", line 382, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3/dist-packages/simplejson/encoder.py", line 296, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3/dist-packages/simplejson/encoder.py", line 378, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3/dist-packages/simplejson/encoder.py", line 272, in default
    raise TypeError('Object of type %s is not JSON serializable' %
TypeError: Object of type datetime is not JSON serializable
2020-11-18 14:35:24,139 [ERROR] mongo_connector.connector:408 - MongoConnector: OplogThread <OplogThread(Thread-2, stopped 140231634654976)> unexpectedly stopped! Shutting down

Date field configuration in managed-schema

<fieldType name="date" class="solr.TrieDateField" precisionStep="0" positionIncrementGap="0"/>
<field name="createdDate" type="date" indexed="true" stored="true"/>

Versions

Python : 3.8
Solr : 8.7.0
MongoDB : 4.2.10

Am not sure if this connector work with latest versions or not. But as per MongoDB documentation i have to use this https://www.mongodb.com/blog/post/introducing-mongo-connector

[AndrewG10i]

Hi! I have fixed this issue in my forked repo: https://github.com/Andrew-Gr/solr-doc-manager/commit/59461ac814e2c9acf5413449468a63a285fe6be8

This issue is related to pysolr starting from v3.9.0. In v3.8.1 - all works fine.

p.s. and yes, it is related to the: https://github.com/yougov/solr-doc-manager

[shivaprasad573]

Thank you for responding @Andrew-gr ,

Please can you add some setup instructions to use your doc manager instead of https://github.com/yougov/solr-doc-manager)

[AndrewG10i]

@shivaprasad573 you may download only fixed source code of the solr-doc-manager which fixes this issue and install it in the following way:

1. Download: https://github.com/Andrew-Gr/solr-doc-manager/releases/tag/0.1.1

2. Unpack and install:
   cd /local/distr/solr-doc-manager-0.1.1
pip3 install .

Alternatively you may install both projects updated by me in the following way:

1. Download and extract these repos:
   https://github.com/Andrew-Gr/mongo-connector/releases/tag/3.1.2
   https://github.com/Andrew-Gr/solr-doc-manager/releases/tag/0.1.3

2. Install each by switching to the unpacked dir and executing command like:
   cd /local/distr/mongo-connector-3.1.2
pip3 install .
cd /local/distr/solr-doc-manager-0.1.3
pip3 install .

Or using python3 -m pip install . command

[shivaprasad573]

Thanks for the reply @Andrew-gr , With below commands i was fixed my issue.

pip install mongo-connector
pip install mongo-connector[solr]
pip install "pysolr==3.8.1```