Skip to content

WHERE Clause Support in Connectors (Subsetting) #13

@MaximumTrainer

Description

@MaximumTrainer

Gap

TableConfiguration stores a whereClause field (up to 4096 characters) and the frontend supports entering it, but:

  • The DatabaseConnector.fetchData() method signature does not accept a whereClause parameter.
  • None of the four connector implementations apply a WHERE clause.
  • JobService.processTable() for SUBSET mode only passes rowLimit to fetchData()whereClause is ignored.

Affected files:

  • backend/src/main/kotlin/com/opendatamask/connector/DatabaseConnector.kt
  • All connector implementations
  • backend/src/main/kotlin/com/opendatamask/model/TableConfiguration.kt

Required Work

  • Extend DatabaseConnector.fetchData() to accept an optional whereClause parameter (with proper SQL-injection prevention via parameterized queries).
  • Implement WHERE clause support in PostgreSQLConnector and AzureSQLConnector.
  • Implement equivalent filtering for MongoDBConnector and MongoDBCosmosConnector (translate WHERE to MongoDB query filters).
  • Wire TableConfiguration.whereClause into the JobService processing pipeline.

Metadata

Metadata

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions