bugfix: fix searching network errors by Dynamite2003 · Pull Request #10 · Dynamite2003/InsightReading

Dynamite2003 · 2025-11-20T12:40:53Z

No description provided.

Dynamite2003 · 2025-11-20T12:41:31Z

Copilot · 2025-11-20T12:41:38Z

@Dynamite2003 I've opened a new pull request, #11, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot

Pull Request Overview

This PR enhances the academic search functionality by adding comprehensive error handling for network requests across multiple data sources. The changes introduce support for a new OpenAlex provider, improve timeout and SSL configuration handling, display source-specific errors to users, and refactor the UI to include floating filters.

Added network error handling and reporting for arXiv, Semantic Scholar, and OpenAlex data sources
Introduced configurable SSL, timeout, and retry settings for external API calls
Enhanced UI to display source-specific errors and allow multi-source selection
Refactored filter panel into a draggable floating component

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 22 comments.

Show a summary per file

File	Description
frontend/src/lib/api/academic.ts	Added source_errors and sources_used fields to SearchResponse interface
frontend/src/components/academic/FloatingFilters.tsx	New draggable floating filter component with position persistence
frontend/src/components/academic/FilterPanel.tsx	Refactored filter panel with improved styling and NaN handling for year input
frontend/src/components/Recommendations.tsx	Integrated multi-source selection and source error display
frontend/src/app/academic/page.tsx	Added comprehensive error handling, source selection UI, and floating filters
backend/app/services/academic/search_service.py	Added per-source error tracking, OpenAlex provider integration, and post-query filtering
backend/app/services/academic/providers/semantic_scholar.py	Added timeout handling, SSL configuration, and improved error messages
backend/app/services/academic/providers/openalex.py	New OpenAlex API provider with inverted index abstract reconstruction
backend/app/services/academic/providers/arxiv.py	Enhanced with retry logic, exponential backoff, SSL configuration, and improved query tokenization
backend/app/schemas/academic.py	Added sources_used and source_errors fields to response schema
backend/app/core/config.py	Added SSL, timeout, and API configuration settings for all providers
backend/app/api/v1/arxiv.py	Reduced MAX_TOTAL_CANDIDATES from 200 to 150

Comments suppressed due to low confidence (1)

backend/app/services/academic/providers/arxiv.py:443

This import of module re is redundant, as it was previously imported on line 9.

                    import re

Copilot · 2025-11-20T12:49:07Z

+  const handleToggleSource = (sourceValue: string) => {
+    setSelectedSources((prev) => {
+      if (prev.includes(sourceValue) && prev.length === 1) {
+        return prev;
+      }
+      const nextSources = prev.includes(sourceValue)
+        ? prev.filter((value) => value !== sourceValue)
+        : [...prev, sourceValue];
+
+      if (mode === "standard" && lastQuery) {
+        void runStandardSearch(lastQuery, appliedFilters, nextSources);
+      }
+
+      return nextSources;
+    });
+  };


[nitpick] Similar race condition issue exists here. The handleToggleSource function (lines 151-166) immediately triggers a search when sources change. If multiple rapid clicks occur or if the search is slow, this could result in multiple concurrent searches with different source configurations, and the results displayed may not match the currently selected sources. Consider tracking the active search and canceling previous ones, or debouncing the source toggle.

Copilot · 2025-11-20T12:49:07Z

+    @staticmethod
+    def _apply_request_filters(papers: List[Dict], request: PaperSearchRequest) -> List[Dict]:
+        """Apply year/venue filters not handled directly by the provider query."""
+        filtered = papers
+
+        if request.year:
+            filtered = [paper for paper in filtered if paper.get('year') == request.year]
+
+        if request.venue:
+            target = request.venue.lower().strip()
+            if target:
+                filtered = [
+                    paper for paper in filtered
+                    if AcademicSearchService._paper_matches_venue(paper, target)
+                ]
+
+        return filtered


Missing documentation for the _apply_request_filters method. Since this is a new static method that performs post-processing filtering, it should have a docstring explaining its purpose, parameters, and return value, especially since other methods in the class have docstrings.

Copilot · 2025-11-20T12:49:07Z

+    OPENALEX_PER_PAGE: int = 25
+    OPENALEX_MAX_PER_PAGE: int = 100
+    OPENALEX_TRUST_ENV: bool = True
+    OPENALEX_MAILTO: str = "insightreading@example.com"


The email address "insightreading@example.com" in the OPENALEX_MAILTO configuration appears to be a placeholder using the example.com domain. OpenAlex uses this email to identify API users and may throttle or block example.com addresses. This should be replaced with a real contact email address before deployment.

Suggested change

OPENALEX_MAILTO: str = "insightreading@example.com"

OPENALEX_MAILTO: str = "tech@insightreading.local" # TODO: Set to a real contact email in production via env var

Copilot · 2025-11-20T12:49:08Z

+                        e.preventDefault();
+                        const form = e.currentTarget.form;
+                        if (form) {
+                          handleSubmit({ preventDefault: () => {}, currentTarget: form } as FormEvent<HTMLFormElement>);


[nitpick] The keyboard shortcut handling in line 363-368 creates a new form submission event by manually constructing an object with preventDefault and currentTarget. This is a workaround that bypasses normal form validation and event handling. A better approach would be to call form.requestSubmit() if available, or to extract the submission logic into a separate function that can be called from both the form's onSubmit and the keyboard handler.

Suggested change

handleSubmit({ preventDefault: () => {}, currentTarget: form } as FormEvent<HTMLFormElement>);

if (typeof form.requestSubmit === "function") {

form.requestSubmit();

} else {

form.submit();

}

Copilot · 2025-11-20T12:49:08Z

    total: int
    query_interpretation: Optional[dict] = None
+    sources_used: Optional[List[str]] = None
+    source_errors: Optional[dict[str, str]] = None


[nitpick] Type annotation uses dict[str, str] (PEP 585 style) on line 41 but Dict from typing module is used elsewhere in the codebase (line 2, other files). While both work in Python 3.9+, mixing styles reduces consistency. Consider standardizing on one approach throughout the codebase.

Copilot · 2025-11-20T12:49:12Z

+    <div
+      ref={containerRef}
+      className={`fixed z-40 w-64 overflow-hidden rounded-2xl border border-white/40 bg-white/90 shadow-xl backdrop-blur-xl transition dark:border-slate-700/50 dark:bg-slate-900/80 ${
+        isDragging ? 'cursor-grabbing' : ''


The cursor style classes are inconsistent. Line 175 sets cursor-grab and cursor-grabbing for the collapsed state, but line 191 only sets cursor-grabbing for the expanded state, missing cursor-grab when not dragging. This creates an inconsistent UX where the expanded filter doesn't show the grab cursor on hover. Consider adding cursor-grab to line 191 when !isDragging.

Suggested change

isDragging ? 'cursor-grabbing' : ''

isDragging ? 'cursor-grabbing' : 'cursor-grab'

Copilot · 2025-11-20T12:49:12Z

+    >
+      <div
+        className="flex items-center justify-between border-b border-white/30 px-3 py-2.5 text-xs font-semibold text-slate-700 dark:border-slate-700/60 dark:text-slate-100"
+        onMouseDown={handleMouseDown}


[nitpick] The drag handle area on line 197 only covers the header, but users might expect to drag from anywhere in the filter panel. Consider making this more discoverable or extending the draggable area, especially since the entire collapsed button is draggable (line 177) but only the header is draggable when expanded.

Suggested change

>

<div

className="flex items-center justify-between border-b border-white/30 px-3 py-2.5 text-xs font-semibold text-slate-700 dark:border-slate-700/60 dark:text-slate-100"

onMouseDown={handleMouseDown}

onMouseDown={handleMouseDown}

>

<div

className="flex items-center justify-between border-b border-white/30 px-3 py-2.5 text-xs font-semibold text-slate-700 dark:border-slate-700/60 dark:text-slate-100"

Copilot · 2025-11-20T12:49:12Z

+    @staticmethod
+    def _paper_matches_venue(paper: Dict, target: str) -> bool:
+        """Check whether any venue-related field roughly matches the user input."""
+        candidate_fields = [
+            paper.get('venue'),
+            paper.get('comment'),
+            paper.get('journal_ref'),
+        ]
+        for field in candidate_fields:
+            if field and target in field.lower():
+                return True
+        return False


Missing documentation for the _paper_matches_venue method. A docstring would help explain what constitutes a "match" and which fields are checked.

Copilot · 2025-11-20T12:49:12Z

+        tokens = re.findall(r'"[^"]+"|\S+', query or '')
+        parsed: List[Tuple[str, bool]] = []
+        for token in tokens:
+            stripped = token.strip()
+            if not stripped:
+                continue
+            if stripped.startswith('"') and stripped.endswith('"') and len(stripped) >= 2:
+                parsed.append((stripped[1:-1].strip(), True))
+            else:
+                parsed.append((stripped, False))


[nitpick] The method _tokenize_terms uses a regex to split quoted phrases but doesn't handle escaped quotes within phrases (e.g., "term with \" inside"). While this may be acceptable for the current use case, it could lead to unexpected behavior if users try to include literal quotes in their search terms. Consider documenting this limitation or handling escaped quotes.

Suggested change

tokens = re.findall(r'"[^"]+"|\S+', query or '')

parsed: List[Tuple[str, bool]] = []

for token in tokens:

stripped = token.strip()

if not stripped:

continue

if stripped.startswith('"') and stripped.endswith('"') and len(stripped) >= 2:

parsed.append((stripped[1:-1].strip(), True))

else:

parsed.append((stripped, False))

"""

Tokenize the query string into (term, is_phrase) tuples.

Handles quoted phrases and escaped quotes within phrases.

"""

parsed: List[Tuple[str, bool]] = []

if not query:

return parsed

i = 0

n = len(query)

while i < n:

# Skip whitespace

while i < n and query[i].isspace():

i += 1

if i >= n:

break

if query[i] == '"':

# Start of quoted phrase

i += 1

phrase = []

in_escape = False

while i < n:

c = query[i]

if in_escape:

phrase.append(c)

in_escape = False

elif c == '\\':

in_escape = True

elif c == '"':

i += 1

break

else:

phrase.append(c)

i += 1

parsed.append((''.join(phrase).strip(), True))

else:

# Unquoted token

start = i

while i < n and not query[i].isspace():

if query[i] == '"':

break

i += 1

token = query[start:i].strip()

if token:

parsed.append((token, False))

Copilot · 2025-11-20T12:49:13Z

+    SEMANTIC_SCHOLAR_VERIFY_SSL: bool = True
+    SEMANTIC_SCHOLAR_TRUST_ENV: bool = True
+    SEMANTIC_SCHOLAR_TIMEOUT: int = 30
+    SEMANTIC_SCHOLAR_USER_AGENT: str = "InsightReading/1.0 (mailto:tech@insightreading.local)"


The config setting SEMANTIC_SCHOLAR_USER_AGENT uses "insightreading.local" as the domain in the mailto field. The .local TLD is reserved for local networks and won't work as a contact email. This should be changed to a valid domain (matching the OPENALEX_MAILTO issue) or made configurable via environment variables.

Dynamite2003 added 7 commits November 20, 2025 17:28

bugfix: fix searching network errors

c89cabe

bugfix: fix frontend search outcome display problems

274b1db

bugfix: fix filter frontend error

2ab0188

bugfix：filter size and position

c9ef5cb

feature: add semantic scholar api

378fcc3

feature: add openalex api

c842460

feature: add APIs for paper recommendation

3400c58

Copilot AI review requested due to automatic review settings November 20, 2025 12:40

Copilot started reviewing on behalf of Dynamite2003 November 20, 2025 12:41 View session

Copilot AI mentioned this pull request Nov 20, 2025

[WIP] Fix searching network errors #11

Closed

Copilot finished reviewing on behalf of Dynamite2003 November 20, 2025 12:45

Copilot AI reviewed Nov 20, 2025

View reviewed changes

Dynamite2003 and others added 3 commits November 25, 2025 11:52

bugfix: fix bug "can't select pdf content in Normal Mode"

6d1f6aa

bugfix: revise recommendation method

f390d69

Merge branch 'dev' into bugfix/search

3b1752c

Dynamite2003 merged commit 1173fcc into dev Nov 25, 2025
1 of 2 checks passed

Dynamite2003 deleted the bugfix/search branch November 27, 2025 04:10

	OPENALEX_MAILTO: str = "insightreading@example.com"
	OPENALEX_MAILTO: str = "tech@insightreading.local" # TODO: Set to a real contact email in production via env var

-                          handleSubmit({ preventDefault: () => {}, currentTarget: form } as FormEvent<HTMLFormElement>);
+                          if (typeof form.requestSubmit === "function") {
+                            form.requestSubmit();
+                          } else {
+                            form.submit();
+                          }

	isDragging ? 'cursor-grabbing' : ''
	isDragging ? 'cursor-grabbing' : 'cursor-grab'

-        tokens = re.findall(r'"[^"]+"|\S+', query or '')
-        parsed: List[Tuple[str, bool]] = []
-        for token in tokens:
-            stripped = token.strip()
-            if not stripped:
-                continue
-            if stripped.startswith('"') and stripped.endswith('"') and len(stripped) >= 2:
-                parsed.append((stripped[1:-1].strip(), True))
-            else:
-                parsed.append((stripped, False))
+        """
+        Tokenize the query string into (term, is_phrase) tuples.
+        Handles quoted phrases and escaped quotes within phrases.
+        """
+        parsed: List[Tuple[str, bool]] = []
+        if not query:
+            return parsed
+        i = 0
+        n = len(query)
+        while i < n:
+            # Skip whitespace
+            while i < n and query[i].isspace():
+                i += 1
+            if i >= n:
+                break
+            if query[i] == '"':
+                # Start of quoted phrase
+                i += 1
+                phrase = []
+                in_escape = False
+                while i < n:
+                    c = query[i]
+                    if in_escape:
+                        phrase.append(c)
+                        in_escape = False
+                    elif c == '\\':
+                        in_escape = True
+                    elif c == '"':
+                        i += 1
+                        break
+                    else:
+                        phrase.append(c)
+                    i += 1
+                parsed.append((''.join(phrase).strip(), True))
+            else:
+                # Unquoted token
+                start = i
+                while i < n and not query[i].isspace():
+                    if query[i] == '"':
+                        break
+                    i += 1
+                token = query[start:i].strip()
+                if token:
+                    parsed.append((token, False))

Conversation

Dynamite2003 commented Nov 20, 2025

Uh oh!

Dynamite2003 commented Nov 20, 2025

Uh oh!

Copilot AI commented Nov 20, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants