Skip to content

RAS-1867 Re-implement My Messages to make query more performant#472

Merged
LJBabbage merged 6 commits into
mainfrom
refactor-internal-threads-queries
Jun 1, 2026
Merged

RAS-1867 Re-implement My Messages to make query more performant#472
LJBabbage merged 6 commits into
mainfrom
refactor-internal-threads-queries

Conversation

@LJBabbage
Copy link
Copy Markdown
Contributor

@LJBabbage LJBabbage commented May 18, 2026

What and why?

This PR improves performance for internal thread retrieval (My Messages), removes unused functionality, rationalises the query logic, and adds coverage for previously untested paths.

Performance

Before

When the My Messages tab was selected, we queried the secure_message table first and performed operations such as determining the latest message before joining to the status table, without index.

This resulted in unnecessary scanning and processing of large datasets before pagination was applied, causing poor performance for users with a high message volume.

Now

The query flow has been reversed for internal users using my_messages:

  • We query the status table first using actor-based filters
  • We then join to secure_message using the indexed msg_id
  • Latest message logic is then applied
  • LIMIT and OFFSET are used to paginate efficiently at the database level

Additional changes

  • Consolidated the 4 internal thread SQL queries into a single query builder to avoid future misalignment between count and result queries
  • Retained the existing pattern of:
    • querying total count first
    • then retrieving the paginated results
  • Left external thread retrieval unchanged to minimise risk in this PR (though it could be optimised similarly in future)
  • Removed redundant pagination code
  • Removed unused search criteria and associated tests
  • Extended test coverage for previously untested logic paths

How to test

Deploy this branch and test a variety of thread and filtering scenarios. Suggested checks:

  1. Create messages across 2 different surveys and confirm they appear in the correct inbox
  2. Verify that:
  • a new respondent message appears in the inbox
  • the thread is shown as unread/bold
  • once replied to, it moves into My Messages
  1. Verify messages sent initially via reporting units appear in My Messages
  2. Confirm business reference searching still works
  3. Confirm closed message filtering still works
  4. Verify pagination behaviour
  5. Regression test any additional thread filtering combinations

Jira

@LJBabbage LJBabbage requested a review from a team as a code owner May 18, 2026 14:31
Comment thread secure_message/common/utilities.py
Comment thread secure_message/common/utilities.py
Comment thread secure_message/repository/database.py
Comment thread secure_message/repository/retriever.py
Comment thread tests/app/test_retriever.py
Comment thread secure_message/repository/database.py
Comment thread secure_message/resources/threads.py
Comment thread tests/behavioural/features/steps/collection_case_field.py
Comment thread tests/behavioural/features/steps/collection_exercise_field.py
Comment thread tests/behavioural/features/threads_count.feature
Comment thread tests/behavioural/features/threads_get.feature
Comment thread secure_message/resources/threads.py
@anwilkie
Copy link
Copy Markdown
Contributor

/deploy wilkia

@ras-rm-pr-bot
Copy link
Copy Markdown
Collaborator

Deploying to dev cluster with following parameters:

  • namespace: wilkia

  • tag: pr-472

  • configBranch: main

  • paramKey: ``

  • paramValue: ``

Comment thread tests/app/test_retriever.py
Comment thread secure_message/repository/retriever.py Outdated
Copy link
Copy Markdown
Contributor

@matthew-robinson-ons matthew-robinson-ons left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested multiple message scenarios. Labels are consistently displayed as before, attaching to the correct actor. Messages move in and out of my messages as before based on which internal user replied most recently. It's hard to comment on the performance of the queries without having anything to compare it against, we only really have that information in PROD currently.

The pagination object is displaying the messages on the page in a different order, previously it was showing the most recent (last sent) messages first.

Copy link
Copy Markdown
Contributor

@matthew-robinson-ons matthew-robinson-ons left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy with this. All looks good to me.

@LJBabbage LJBabbage merged commit e2b0c4b into main Jun 1, 2026
4 checks passed
@LJBabbage LJBabbage deleted the refactor-internal-threads-queries branch June 1, 2026 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants