Spotlight search malfunction after upgrade

[toropok]

Description

After upgrade_2521 customer noticed that spotlight search returns no results.

The reason is:

2023-12-13T10:24:53 INFO senaite.app.spotlight Spotlight query={'listing_searchable_text': u'\u041c\u0430\u043a\u0441\u0438\u043c\u043e*'} for catalog='senaite_catalog_sample'
------
2023-12-13T10:24:53 ERROR Zope.SiteErrorLog 1702437893.10.406990922902 https://---------------/@@API/spotlight/search
Traceback (innermost last):
  Module ZServer.ZPublisher.Publish, line 144, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module ZServer.ZPublisher.Publish, line 44, in call_object
  Module plone.jsonapi.core.browser.api, line 76, in __call__
  Module plone.jsonapi.core.browser.decorators, line 58, in decorator
  Module simplejson, line 395, in dumps
  Module simplejson.encoder, line 297, in encode
  Module simplejson.encoder, line 379, in iterencode
  Module simplejson.encoder, line 274, in default
TypeError: Object of type Missing is not JSON serializable
------
2023-12-13T10:24:53 ERROR Zope.SiteErrorLog 1702437893.110.412666511337 https://---------------/@@API/spotlight/search
Traceback (innermost last):
  Module ZServer.ZPublisher.Publish, line 144, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module ZServer.ZPublisher.Publish, line 44, in call_object
  Module plone.jsonapi.core.browser.api, line 76, in __call__
  Module plone.jsonapi.core.browser.decorators, line 58, in decorator
  Module simplejson, line 395, in dumps
  Module simplejson.encoder, line 297, in encode
  Module simplejson.encoder, line 379, in iterencode
  Module simplejson.encoder, line 274, in default
TypeError: Object of type Missing is not JSON serializable

This issue solves when you resave (open the sample page and click save button) the sample manually.

Reindexing samples through the ZMI page doesn't help. What's magical happens after "save" button to repeat it automatically?

[ramonski]

Hi Leo,
thanks for sharing the traceback.

The issue above happens when a metadata (aka column) field hass a missing value, which is an instance of class Missing and it is part of the listing_searchable_text index.

As far as I remember, you extended the search terms for this index with an adapter?

This was probably the changeset that caused your error:
https://github.com/senaite/senaite.core/pull/2273/files

However, with this change we no longer use get_searchable_text_tokens from senaite.core.catalog.utils, which in fact used these Metadata columns.

Therefore, if you extend the search, avoid using this function, because it might give you these Missing values back and cause the above mentioned error.

Besides that, if you simply reindex an index in the catalog, the Metadata is not recreated. But if you save an object, it will reindex all indexes + metadata.

Hope that sheds some light into your issue.

[toropok]

@ramonski That's totally makes sense and helps a lot! I absolutely forgot about this legacy adapter!

thank you very much!