AI-Assisted Duplicate Question Detection With Semantic Similarity Search

[krrish612]

We propose an AI-powered real-time duplicate question detection system that considers semantic similarity as the user is typing a new question.

Motivation:
Current Q&A workflows heavily depend on exact keyword search, which allows semantically duplicate questions to fill the platform with slightly different wording.

For example:

How does the React virtual DOM work?
Vs.
"How does React make DOM rendering faster?"

Capabilities proposed:

semantic duplicate detection in real time
similarity rating
Suggested questions first post
typo- or paraphrase-aware matching
clustering of duplicates
auto-linking related threads

Areas affected:

frontend/question-editor/
backend/search/
backend/ai

Suggested implementation:

semantic search based on embeddings
vector similarity search
popular question embeddings ( cached )
hybrid rank (BM25 + embedding)
live suggestion pipeline using debounce

Expected Impact

little duplicate content
knowledge graph cleaner
better search quality
better engagement on existing threads
better scaling of platform

[IshitaSingh0822]

Hii @krrish612
I want to work on this issue under GSSOC
please assign me
/assign

[amans2003]

Hii @krrish612 can you assign this issue to me !!