Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WEBDEV-5583 Add filters param to handle facet/date-range constraints #23

Merged
merged 21 commits into from
Nov 21, 2022
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
bc82745
Don't send empty sort or fields params in PPS requests
latonv Nov 2, 2022
422f771
Add param for filter_map to external interface
latonv Nov 4, 2022
dad5a33
Rethink search service filters interface, and document more clearly
latonv Nov 4, 2022
9d90811
Further refine filter map interface/docs, and include it in URL
latonv Nov 5, 2022
f5df311
Add tests for filter_map
latonv Nov 7, 2022
0583afe
Add tests for empty fields/sort params
latonv Nov 7, 2022
d3a0963
Collapsible URL params in demo app + refactoring
latonv Nov 8, 2022
66a35af
Add FilterMapBuilder utility class
latonv Nov 9, 2022
9d77e88
Fix demo app alignment of radio/checkboxes
latonv Nov 9, 2022
80deb9f
Add filters to demo app
latonv Nov 9, 2022
8791c95
Add removeFilter method to filter map builder
latonv Nov 9, 2022
0c9ed48
Add tests for filter map builder and fix formatting
latonv Nov 9, 2022
4f2cf73
Clean up demo app queries
latonv Nov 9, 2022
bc0f49a
Fill in missing doc comment on builder
latonv Nov 9, 2022
b41482a
Add notes to gt/lt constraint docs (they are unsupported by FTS)
latonv Nov 10, 2022
a2d8b48
Further streamline demo app with common template for aggregation inputs
latonv Nov 10, 2022
c023aed
Make aggregation label param required
latonv Nov 10, 2022
907911e
Remove filter overlap handling -- PPS doesn't actually impl gt/lt so …
latonv Nov 11, 2022
e0422f1
Support filter constraints being arrays or single values
latonv Nov 19, 2022
5cedba4
Extract duplicate filter map builder logic into helper method
latonv Nov 21, 2022
563a895
Extract repeated filter map builder test prep into helper
latonv Nov 21, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
412 changes: 292 additions & 120 deletions demo/app-root.ts

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion demo/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@
<head>
<meta charset="utf-8">
<style>
body {
html {
background: #fff;
font-family: sans-serif;
font-size: 10px;
}
</style>
</head>
Expand Down
4 changes: 4 additions & 0 deletions index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,5 +46,9 @@ export {
SortDirection,
AggregateSearchParams,
AggregateSearchParam,
FilterMap,
FieldFilter,
FilterConstraint,
} from './src/search-params';
export { FilterMapBuilder } from './src/filter-map-builder';
export { SearchServiceError } from './src/search-service-error';
134 changes: 134 additions & 0 deletions src/filter-map-builder.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
import { FilterConstraint, FilterMap } from './search-params';

/**
* A utility class for building filter maps
*/
export class FilterMapBuilder {
private filterMap: FilterMap = {};

/**
* Adds a filter to the FilterMap under construction.
* If existing constraint(s) already exist for this field and value, then the old and new constraints
* will be joined into a single array.
* @param field The field to filter on (e.g., 'subject', 'year', ...)
* @param value The value of the field to filter on (e.g., 'Cicero', '1920', ...)
* @param constraint The constraint to apply to the `field`, with respect to the given `value`.
* Allowed values are the enum members of `FilterConstraint`.
*/
addFilter(field: string, value: string, constraint: FilterConstraint): this {
if (!this.filterMap[field]) {
this.filterMap[field] = {};
}

// If there are already constraints for this value, concat them into an array
if (this.filterMap[field][value]) {
const mergedConstraints = ([] as FilterConstraint[]).concat(
this.filterMap[field][value],
constraint
);

// Ensure there are no duplicate constraints in the array
this.filterMap[field][value] = Array.from(new Set(mergedConstraints));
} else {
// Otherwise just use the provided value
this.filterMap[field][value] = constraint;
}

return this;
}

/**
* Removes a single filter currently associated with the given field, value, and constraint type.
* @param field The field to remove a filter for
* @param value The value to remove the filter for
* @param constraint The constraint type to remove for this field and value
*/
removeSingleFilter(
field: string,
value: string,
constraint: FilterConstraint
): this {
if (!this.filterMap[field]?.[value]) return this;

const constraints = ([] as FilterConstraint[]).concat(
this.filterMap[field][value]
);
const constraintIndex = constraints.indexOf(constraint);
if (constraintIndex >= 0) {
constraints.splice(constraintIndex, 1);
}

// 2 or more constraints -> leave as array
// 1 constraint -> pull out single constraint
// 0 constraints -> delete the value entirely
this.filterMap[field][value] =
constraints.length === 1 ? constraints[0] : constraints;
if (constraints.length === 0) {
delete this.filterMap[field][value];
}

// If there are no remaining filters for this field, delete the whole field object.
if (Object.keys(this.filterMap[field]).length === 0) {
delete this.filterMap[field];
}

return this;
}

/**
* Removes any filters currently associated with the given field and value.
* @param field The field to remove a filter for
* @param value The value to remove the filter for
*/
removeFilters(field: string, value: string): this {
if (!this.filterMap[field]) return this;

delete this.filterMap[field][value];

// If there are no remaining filters for this field, delete the whole field object.
if (Object.keys(this.filterMap[field]).length === 0) {
latonv marked this conversation as resolved.
Show resolved Hide resolved
delete this.filterMap[field];
}

return this;
}

/**
* Initializes the filter map under construction to have filters exactly equal to the given one.
* This will overwrite *all* existing filters already added to the builder.
* @param map The FilterMap to set this builder's state to.
*/
setFilterMap(map: FilterMap): this {
this.filterMap = { ...map };
return this;
}

/**
* Adds all filters from an existing filter map to the one being built.
* Filters from the provided map may overwrite existing filters already added to the builder.
* @param map The FilterMap to merge into the one being built.
*/
mergeFilterMap(map: FilterMap): this {
for (const [field, filters] of Object.entries(map)) {
for (const [value, constraint] of Object.entries(filters)) {
// There may be either a single constraint or an array of them
if (Array.isArray(constraint)) {
for (const subConstraint of constraint) {
this.addFilter(field, value, subConstraint);
}
} else {
this.addFilter(field, value, constraint);
}
}
}
return this;
}

/**
* Produces a `FilterMap` including all the filters that have been applied to
* this builder.
*/
build(): FilterMap {
return this.filterMap;
}
}
18 changes: 16 additions & 2 deletions src/search-param-url-generator.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import {
AggregateSearchParams,
FilterMap,
SearchParams,
SortParam,
} from './search-params';
Expand Down Expand Up @@ -54,6 +55,11 @@ export class SearchParamURLGenerator {
return `${sortParams.field}:${sortParams.direction}`;
}

static filterParamsAsString(filters: FilterMap): string {
// FilterMap already has the correct shape, we just need to stringify it
return JSON.stringify(filters);
}

static generateURLSearchParams(searchParams: SearchParams): URLSearchParams {
const params: URLSearchParams = new URLSearchParams();
params.append('user_query', searchParams.query);
Expand All @@ -74,11 +80,19 @@ export class SearchParamURLGenerator {
params.append('page', String(searchParams.page));
}

if (searchParams.fields) {
if (searchParams.fields && searchParams.fields.length > 0) {
params.append('fields', searchParams.fields.join(','));
}

if (searchParams.sort) {
if (searchParams.filters && Object.keys(searchParams.filters).length > 0) {
const filterMapString = this.filterParamsAsString(searchParams.filters);
if (filterMapString && filterMapString !== '{}') {
// Don't send an empty map
params.append('filter_map', filterMapString);
}
}

if (searchParams.sort && searchParams.sort.length > 0) {
const sortStrings = searchParams.sort.map(sort =>
this.sortParamsAsString(sort)
);
Expand Down
122 changes: 122 additions & 0 deletions src/search-params.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,108 @@ export interface SortParam {
direction: SortDirection;
}

/**
* Enumerates the possible contraints that may be imposed on search results
* by filter params.
*/
export enum FilterConstraint {
/**
* Specifies that all results must include _at least one of_ the values constrained
* with INCLUDE for this field.
*
* For instance, `{ subject: { baseball: INCLUDE, basketball: INCLUDE } }` specifies
* that only results containing _either_ `baseball` _or_ `basketball` as a subject
* should be returned.
*/
INCLUDE = 'inc',

/**
* Specifies that all results must _not_ include the given value for this field.
*
* For instance, `{ subject: { baseball: EXCLUDE, basketball: EXCLUDE } }` specifies
* that only results containing _neither_ `baseball` _nor_ `basketball` as a subject
* should be returned.
*/
EXCLUDE = 'exc',

/**
* Imposes a strict lower bound on numeric values for the current field.
* All returned hits must have a value for this field that is greater than the one
* specified by this filter.
*
* This only makes sense for numeric fields like `year`.
* Note that `GREATER_THAN` is not supported by the FTS engine, for which it is
* coerced to `GREATER_OR_EQUAL`.
*/
GREATER_THAN = 'gt',

/**
* Imposes a non-strict lower bound on numeric values for the current field.
* All returned hits must have a value for this field that is greater than or equal
* to the one specified by this filter.
*
* This only makes sense for numeric fields like `year`.
*/
GREATER_OR_EQUAL = 'gte',

/**
* Imposes a strict upper bound on numeric values for the current field.
* All returned hits must have a value for this field that is less than the one
* specified by this filter.
*
* This only makes sense for numeric fields like `year`.
* Note that `LESS_THAN` is not supported by the FTS engine, for which it is
* coerced to `LESS_OR_EQUAL`.
*/
LESS_THAN = 'lt',

/**
* Imposes a non-strict upper bound on numeric values for the current field.
* All returned hits must have a value for this field that is less than or equal
* to the one specified by this filter.
*
* This only makes sense for numeric fields like `year`.
*/
LESS_OR_EQUAL = 'lte',
}

/**
* A filter mapping a field value to the type of constraint(s) that it should impose on results.
* Multiple constraints for the same value may be provided as an array.
*
* Some examples (where the property values are members of `FilterConstraint`):
* - `{ 'puppies': INCLUDE }`
* - `{ '1950': GREATER_OR_EQUAL, '1970': LESS_OR_EQUAL }`
* - `{ '1950': [ GREATER_OR_EQUAL, EXCLUDE ] }`
*/
export type FieldFilter = Record<string, FilterConstraint | FilterConstraint[]>;

/**
* A map of fields (e.g., 'year', 'subject', ...) to the filters that should be
* applied to them when retrieving search results.
*
* These filters may represent selected/hidden facets, value ranges (e.g., date picker),
* or other types of restrictions on the result set.
*
* An example of a valid FilterMap:
* ```
* {
* 'subject': {
* 'dogs': INCLUDE,
* 'puppies': EXCLUDE,
* },
* 'year': {
* '1990': GREATER_OR_EQUAL,
* '2010': LESS_OR_EQUAL,
* '2003': EXCLUDE,
* '2004': EXCLUDE,
* },
* // ...
* }
* ```
*/
export type FilterMap = Record<string, FieldFilter>;

/**
* SearchParams provides an encapsulation to all of the search parameters
* available for searching.
Expand Down Expand Up @@ -105,6 +207,26 @@ export interface SearchParams {
*/
fields?: string[];

/**
* A map from field names to filters that can be used to shape the result set.
* The keys identify what field to filter on (e.g., `'year'`, `'subject'`, etc.),
* and the values identify what filters to apply for that field.
*
* The constraints allowed are the members of `FilterContraint`:
* - `INCLUDE` (at least one of these values must be present)
* - `EXCLUDE` (none of these values may be present)
* - `GREATER_THAN` (result values must be strictly greater than the one specified)
* - `GREATER_OR_EQUAL` (result values must be greater than or equal to than the one specified)
* - `LESS_THAN` (result values must be strictly less than the one specified)
* - `LESS_OR_EQUAL` (result values must be less than or equal to the one specified)
*
* So filters like `{ creator: { 'Cicero': INCLUDE } }` will produce
* search results that all include `Cicero` as a creator, while filters like
* `{ year: { '2000': GREATER_THAN, '2005': LESS_THAN } }` will produce search results whose
* `year` field is between 2000 and 2005 (exclusive).
*/
filters?: FilterMap;

/**
* An object specifying which aggregation types should be returned with
* a search query.
Expand Down
Loading