Skip to content

feat: Conversations tab — filters, sorting, scroll UX, file-type filter, column picker & Security navigation#80

Merged
NotYuSheng merged 21 commits intomainfrom
feature/conversations-enhancements
Mar 28, 2026
Merged

feat: Conversations tab — filters, sorting, scroll UX, file-type filter, column picker & Security navigation#80
NotYuSheng merged 21 commits intomainfrom
feature/conversations-enhancements

Conversation

@NotYuSheng
Copy link
Copy Markdown
Owner

Summary

  • Filter panel — IP/hostname search, protocol/app/category/file-type multi-select pills, security-risks-only toggle, active filter chips with individual dismiss
  • Column sorting — clickable sortable headers (Source, Destination, Packets, Bytes, Duration, Start Time) with asc/desc/off cycle; sort state in URL
  • Column visibility picker — inside filter panel, persisted to localStorage; Protocol hidden by default
  • File-type filter — backend EXISTS subquery against packets table; frontend pill filter for detected file types (e.g. PDF, PNG, ZIP)
  • CSV export — download button exports current filtered results via GET /api/conversations/:fileId/export
  • Flow risk chips — inline risk badges on Conversations table rows; chip click sets hasRisks=true filter
  • Dual scrollbars — phantom top scrollbar synced to table bottom scrollbar for wide tables
  • Middle-click pan — persistent omni-directional pan mode (velocity-based rAF loop, both axes within table)
  • Vertical scroll constrained to table — table container uses overflow: auto + max-height: 62vh
  • Pagination layout — "Showing X of Y" and "Items per page" grouped left; SGDS Pagination on right; no top margin
  • Security tab — "View in Conversations" button in packet stream modal navigates to Conversations filtered by IP + hasRisks=true
  • Bug fixes — infinite reload (unstable filters reference), double /api/api/ prefix, middle-click not firing, TypeScript compile errors

Test plan

  • Upload a multi-protocol PCAP
  • Open Conversations — filter panel toggle shows, pills populated from data
  • Click a protocol pill → table filters, URL updates to ?protocols=TCP
  • Click a sortable column header → rows sort, icon changes; click again → descending; again → clears
  • Toggle column visibility — Protocol off by default; refresh → preference persisted
  • File-type pills appear when packets contain detected file types; filter works
  • Export CSV button downloads correct filtered results
  • Middle-click on table → pan mode activates (dot indicator); move mouse to scroll all directions within table only
  • Security tab → open a risk conversation → click "View in Conversations" → lands on Conversations tab filtered by IP + risks
  • docker compose build && docker compose up -d — clean build, no errors

🤖 Generated with Claude Code

NotYuSheng and others added 18 commits March 28, 2026 14:55
Swap the hand-rolled pagination for @govtechsg/sgds-react Pagination.
The wrapper preserves the existing props interface (currentPage,
totalPages, totalItems, pageSize, onPageChange, onPageSizeChange) so
all callers remain unchanged. Page size selector and item count summary
are kept as-is alongside the SGDS component.

Also add CLAUDE.md with a note to always prefer SGDS components over
custom implementations.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend:
- GET /api/conversations/:fileId now accepts optional ?search= param
  which filters by appName, protocol, srcIp, dstIp, or hostname
  (case-insensitive contains) before paginating

Frontend:
- ConversationPage: search bar with Apply/clear, reads ?app= URL param
  to pre-populate (e.g. when navigating from Overview)
- AnalysisOverview: detected application badges are now buttons that
  navigate to /conversations?app=<name>, triggering the search filter

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend:
- ConversationFilterParams DTO: ip, protocols, apps, categories, hasRisks, sortBy, sortDir
- ConversationRepository extends JpaSpecificationExecutor; Specification factory
  handles IP contains, protocol/app/category IN lists, and hasRisks flag
- AnalysisService.getConversations uses DB-level filtering + sorting via Pageable
- New getConversationsForExport returns all matching rows without pagination
- ConversationsController: new structured params, backward-compat ?search= alias,
  new GET /:fileId/export endpoint returns text/csv
- PagedResponse: new of(content, total, page, pageSize) overload for DB pagination
- V16 migration: indexes on app_name, category, start_time, packet_count, total_bytes

Frontend:
- conversation/types/index.ts: SortField, SortDir, ConversationFilters
- useConversationFilters hook: all filter/sort/page state in URL params
- conversationService: accepts ConversationFilters, getExportUrl helper
- ConversationFilterPanel: collapsible with IP input, protocol/app/category
  multi-select pills, security risks toggle, active filter chips
- ConversationList: sortable column headers (cycle asc→desc→clear),
  conditional Risks column with clickable risk chips
- ConversationPage: uses hook, renders filter panel, sort, page-size selector,
  Export CSV button; legacy srcIp/peerIp/app params auto-migrated

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wrapped `filters` in useMemo([searchParams]) so the object reference is
stable between renders. The previous plain object literal was recreated
every render, causing useEffect([fileId, filters]) in ConversationPage
to fire on every render — an infinite reload loop.

Also redesigned setFilters to read current state from `prev` inside the
setSearchParams updater callback, removing `filters` from its closure
and making the callback stable with only [setSearchParams] as dependency.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ConversationList: replace table-responsive with a custom scroll container
that shows a persistent thin scrollbar and a right-edge fade gradient that
disappears when the user has scrolled to the end.

Pagination: move "Showing X to Y of Z items" and "Items per page" selector
into a shared left-hand group (.pagination-meta) so both info labels sit
together, with the page navigation buttons on the right.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… table

- Adds a mirrored top scrollbar div above the table (synced with the
  bottom scrollbar via JS) so users can scroll without reaching the bottom
- Middle-click (button 1) on the table activates free-pan mode: drag in
  any direction to scroll, cursor changes to all-scroll while active

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
File type filtering:
- ConversationFilterParams: add fileTypes field
- ConversationRepository: EXISTS subquery against packets.detected_file_type
  + findDistinctFileTypesByFileId JPQL query
- AnalysisService: getDistinctFileTypes() method
- ConversationsController: fileTypes param on list + export endpoints,
  new GET /{fileId}/file-types endpoint
- Frontend types/hook/service: carry fileTypes through URL params
- ConversationFilterPanel: File Types pill section + active chips
- ConversationPage: fetch file types on mount, pass to filter panel

Column visibility:
- ConversationList: COLUMN_DEFS with Protocol hidden by default,
  persisted to localStorage; "Columns" dropdown button with checkboxes;
  all thead/tbody cells gated on col() visibility helper

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Middle-click pan:
- Single middle-click enters persistent auto-scroll mode (like browser native)
- Mouse position relative to origin drives continuous scrolling — further
  from origin = faster scroll in that direction, with an 8px dead zone
- A crosshair indicator appears at the click origin while active
- Exit by: another click (any button), Escape, or second middle-click

Column picker:
- Removed standalone Columns button from above the table
- Column checkboxes now live inside the collapsible Filters panel
- State lifted to ConversationPage; passed as visibleColumns prop to
  ConversationList and onToggleColumn to ConversationFilterPanel
- Shared ColumnKey type and COLUMN_DEFS moved to constants.ts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rizontally

The conv-table-scroll container only has overflow-x, so el.scrollTop had no
effect. Use window.scrollBy() for the vertical axis instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Our custom implementation suppressed the browser's native auto-scroll
(via preventDefault on middle mousedown) and replaced it with a different
feel — wrong speed curve, split axes, custom indicator. Removing it lets
Firefox/Chrome native auto-scroll take over, which handles scrollable
containers correctly out of the box.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Firefox native auto-scroll skips our container because it only has
overflow-x (not both axes), so native never fires. Re-add custom pan:
- Middle-click enters persistent pan mode (small dot at origin)
- Horizontal distance scrolls the table container (scrollLeft)
- Vertical distance scrolls the page (window.scrollBy)
- 8px dead zone, velocity proportional to distance from origin
- Exit: second middle-click, any other click, or Escape

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tListener

Native addEventListener on the scroll container wasn't reliably receiving
middle-click events fired on child table cells. Switching to React's
onMouseDown prop guarantees the event is caught regardless of where inside
the table the user clicks. Pan state lives in refs so the rAF tick loop
always reads current values without stale closures.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Set overflow: auto + max-height: 62vh on conv-table-scroll so it becomes
vertically scrollable itself. Pan tick now uses el.scrollTop for both axes
instead of window.scrollBy for vertical.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a small button in the modal header that closes the modal and
navigates to the Conversations tab pre-filtered by the source IP of
the selected conversation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
apiClient already has /api as its baseURL, so the path should be
/conversations/:id/file-types not /api/conversations/:id/file-types.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Adds hasRisks=true to the navigation URL so the Conversations tab
  lands pre-filtered to risk conversations for the selected IP
- Changes btn-outline-primary (SGDS purple) to btn-primary (blue per
  the sgds-overrides.css tracepcap-primary override)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request significantly enhances the network conversations view by implementing structured filtering, sorting, and server-side pagination. Key additions include a JPA Specification-based filtering backend, a CSV export feature, and a refactored frontend using SGDS components and URL-synchronized state. The review feedback highlights a critical security concern regarding CSV injection in the export endpoint and suggests improving consistency between the listing and export logic. Additionally, there are recommendations for optimizing database performance on text-based searches using GIN indexes and minor refinements to frontend state management and error logging.

Comment thread backend/src/main/java/com/tracepcap/analysis/dto/ConversationFilterParams.java Outdated
Comment thread frontend/src/pages/Conversation/ConversationPage.tsx Outdated
NotYuSheng and others added 3 commits March 28, 2026 16:31
- Extract shared buildFilterParams() helper in ConversationsController;
  adds missing `search` legacy param to the export endpoint so list and
  export filters stay in sync
- Fix CSV injection in export: replace printf with RFC 4180 escapeCsv()
  that quotes fields containing commas, quotes, or newlines
- Fix sortBy Javadoc to document frontend field names (packets, bytes,
  duration) instead of internal entity names
- Add V17 migration: enable pg_trgm and GIN indexes on lower(src_ip),
  lower(dst_ip), lower(hostname) for LIKE-based IP/hostname searches
- Remove redundant useEffect in ConversationFilterPanel (isOpen already
  initialised from activeFilterCount > 0 in useState)
- Restore console.error in ConversationPage catch block for debugging

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Story page catch block now detects 500/connection errors and shows a
clear "LLM server not responding" message instead of a raw axios error
string. Filter generator already had equivalent detection.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Both pages now show:
"The LLM server is not responding. Make sure the LLM service is running
and reachable, then try again."

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@NotYuSheng NotYuSheng merged commit 9edd130 into main Mar 28, 2026
@NotYuSheng NotYuSheng deleted the feature/conversations-enhancements branch March 28, 2026 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant