Skip to content

feat: add WHERE clause test coverage for MongoDB and Azure SQL connectors#22

Merged
MaximumTrainer merged 2 commits intomainfrom
copilot/add-where-clause-support
Apr 1, 2026
Merged

feat: add WHERE clause test coverage for MongoDB and Azure SQL connectors#22
MaximumTrainer merged 2 commits intomainfrom
copilot/add-where-clause-support

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 1, 2026

TableConfiguration.whereClause was already wired through the pipeline (JobServicefetchData → all four connectors), but MongoDBConnector and AzureSQLConnector had no test coverage for their fetchData WHERE clause and limit behaviour.

Changes

  • MongoDBConnectorTest — three new fetchData tests verifying the correct BSON filter document is passed to collection.find():

    • no whereClausefind(Document()) (empty filter)
    • whereClause provided → find(Document.parse(whereClause))
    • limit provided → findIterable.limit(n) is called

    Uses RETURNS_DEEP_STUBS to handle the MongoIterable.map(...).toList() chain without boilerplate.

    connector.fetchData("users", whereClause = """{"age": 18}""")
    verify(mockCollection).find(Document.parse("""{"age": 18}"""))
  • AzureSQLConnectorTest (new file) — full connector test suite using H2 in MODE=MSSQLServer (supports SELECT TOP n and [bracket] identifiers). Covers testConnection, fetchData with/without WHERE clause and with row limit, writeData, and truncateTable. Passes null credentials so the connector uses the simple DriverManager.getConnection(url) path, avoiding Azure-specific JDBC properties.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • example.com
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java -Dorg.gradle.internal.worker.tmpdir=/home/REDACTED/work/OpenDataMask/OpenDataMask/backend/build/tmp/test/work -Dorg.gradle.native=false -javaagent:/home/REDACTED/work/OpenDataMask/OpenDataMask/backend/build/tmp/.cache/expanded/zip_0bd0851504e6286cb5fc6f37a81e25cc/jacocoagent.jar=destfile=build/jacoco/test.exec,append=true,inclnolocationclasses=false,dumponexit=true,output=file,jmx=false @/home/REDACTED/.gradle/.tmp/gradle-worker-classpath10669771887708325242txt -Xmx512m -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -ea worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 2' (dns block)
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java -Dorg.gradle.internal.worker.tmpdir=/home/REDACTED/work/OpenDataMask/OpenDataMask/backend/build/tmp/test/work -Dorg.gradle.native=false -javaagent:/home/REDACTED/work/OpenDataMask/OpenDataMask/backend/build/tmp/.cache/expanded/zip_0bd0851504e6286cb5fc6f37a81e25cc/jacocoagent.jar=destfile=build/jacoco/test.exec,append=true,inclnolocationclasses=false,dumponexit=true,output=file,jmx=false @/home/REDACTED/.gradle/.tmp/gradle-worker-classpath2143947722485740049txt -Xmx512m -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -ea worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 5' (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI changed the title [WIP] Add WHERE clause support in DatabaseConnector fetchData method feat: add WHERE clause test coverage for MongoDB and Azure SQL connectors Apr 1, 2026
Copilot AI requested a review from MaximumTrainer April 1, 2026 16:48
@MaximumTrainer MaximumTrainer requested a review from Copilot April 1, 2026 17:31
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@MaximumTrainer MaximumTrainer marked this pull request as ready for review April 1, 2026 18:12
@MaximumTrainer MaximumTrainer merged commit 19b4965 into main Apr 1, 2026
10 checks passed
@MaximumTrainer MaximumTrainer deleted the copilot/add-where-clause-support branch April 1, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WHERE Clause Support in Connectors (Subsetting)

3 participants