refactor: could make searching code easier to understand

[eddotman]

sandbox-conversant-lib/conversant/search/local_searcher.py

Lines 46 to 63 in 50dab74

	self._measure_similarity(embedded_query, d.embedding)
	for d in self.documents
	]
	max_similarity = max(similarities)
	if max_similarity < threshold:
	logging.warning(
	f"Max search similarity {max_similarity} below threshold {threshold}; "
	"no document returned."
	)
	return None
	logging.info(f"Search result found for query: {query}")
	nearest_idx = similarities.index(max_similarity)
	nearest_doc = self.documents[nearest_idx]
	return nearest_doc

Right now there are some list comprehensions & reliance on list order preservation that make this hard to interpret.

The code could be made more explicit:

# Search for the most similar doc
most_similar_document = None
top_similarity_score = 0
for doc in documents:
    similarity_score = self._measure_similarity(embedded_query, doc.embedding)
    if similarity_score > top_similarity_score:
        most_similar_document = doc

# Make sure it's similar enough
if top_similarity_score < threshold:
    logging.warning(
        f"Max search similarity {max_similarity} below threshold {threshold}; "
        "no document returned."
    )
    return None

return most_similar_document

Shoutout to @mkozakov and Jacob-GK for flagging this!