Skip to content

FIR-4 (rich query updates) and FIR-7 (DX manifests) integration to onchain-logic branch #384

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from
Closed

FIR-4 (rich query updates) and FIR-7 (DX manifests) integration to onchain-logic branch #384

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
8c77392
Update query syntax for starts/ends with and tweak for consistency
peterbroadhurst Jan 6, 2022
2b013c5
Andrew confirmed we don't need escaping per RFC3986
peterbroadhurst Jan 6, 2022
aec5771
Merge with main
peterbroadhurst Jan 6, 2022
2a91130
Update code after merge
peterbroadhurst Jan 6, 2022
a59a5c1
Support partial ID queries, and null queries
peterbroadhurst Jan 7, 2022
4e0a334
Handle case sensitivity in SQLite
peterbroadhurst Jan 7, 2022
d97fed1
Avoid double reigstering SQLite driver
peterbroadhurst Jan 7, 2022
f59e686
Provide error messages in the case of bad combinations
peterbroadhurst Jan 7, 2022
bc09eea
Make bytables JSONAny, and serialize to string
peterbroadhurst Jan 7, 2022
4283dc7
Err tweaks
peterbroadhurst Jan 7, 2022
7b254c9
Move to TEXT fields
peterbroadhurst Jan 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion db/migrations/postgres/000002_create_data_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ CREATE TABLE data (
datatype_version VARCHAR(64) NOT NULL,
hash CHAR(64) NOT NULL,
created BIGINT NOT NULL,
value BYTEA NOT NULL,
value TEXT NOT NULL,
blobstore BOOLEAN NOT NULL
);
CREATE UNIQUE INDEX data_id ON data(id);
Expand Down
2 changes: 1 addition & 1 deletion db/migrations/postgres/000004_create_batches_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ CREATE TABLE batches (
group_hash CHAR(64),
hash CHAR(64),
created BIGINT NOT NULL,
payload BYTEA NOT NULL,
payload TEXT NOT NULL,
payload_ref CHAR(64),
confirmed BIGINT,
tx_type VARCHAR(64) NOT NULL,
Expand Down
2 changes: 1 addition & 1 deletion db/migrations/postgres/000005_create_transactions_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ CREATE TABLE transactions (
created BIGINT NOT NULL,
protocol_id VARCHAR(256),
status VARCHAR(64) NOT NULL,
info BYTEA
info TEXT
);

CREATE UNIQUE INDEX transactions_id ON data(id);
Expand Down
2 changes: 1 addition & 1 deletion db/migrations/postgres/000006_create_datatypes_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ CREATE TABLE datatypes (
version VARCHAR(64) NOT NULL,
hash CHAR(64) NOT NULL,
created BIGINT NOT NULL,
value BYTEA
value TEXT
);

CREATE UNIQUE INDEX datatypes_id ON data(id);
Expand Down
2 changes: 1 addition & 1 deletion db/migrations/postgres/000008_create_operations_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ CREATE TABLE operations (
created BIGINT NOT NULL,
updated BIGINT,
error VARCHAR NOT NULL,
info BYTEA
info TEXT
);

CREATE UNIQUE INDEX operations_id ON operations(id);
Expand Down
2 changes: 1 addition & 1 deletion db/migrations/postgres/000010_create_subscriptions_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ CREATE TABLE subscriptions (
filter_topics VARCHAR(256) NOT NULL,
filter_tag VARCHAR(256) NOT NULL,
filter_group VARCHAR(256) NOT NULL,
options BYTEA NOT NULL,
options TEXT NOT NULL,
created BIGINT NOT NULL
);

Expand Down
2 changes: 1 addition & 1 deletion db/migrations/postgres/000013_create_orgs_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ CREATE TABLE orgs (
parent VARCHAR(1024),
identity VARCHAR(1024) NOT NULL,
description VARCHAR(4096) NOT NULL,
profile BYTEA,
profile TEXT,
created BIGINT NOT NULL
);

Expand Down
2 changes: 1 addition & 1 deletion db/migrations/postgres/000014_create_nodes_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ CREATE TABLE nodes (
name VARCHAR(64) NOT NULL,
description VARCHAR(4096) NOT NULL,
dx_peer VARCHAR(256),
dx_endpoint BYTEA,
dx_endpoint TEXT,
created BIGINT NOT NULL
);

Expand Down
2 changes: 1 addition & 1 deletion db/migrations/postgres/000015_create_config_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ BEGIN;
CREATE TABLE config (
seq SERIAL PRIMARY KEY,
config_key VARCHAR(512) NOT NULL,
config_value BYTEA NOT NULL
config_value TEXT NOT NULL
);
CREATE UNIQUE INDEX config_sequence ON config(seq);
CREATE UNIQUE INDEX config_config_key ON config(config_key);
Expand Down
2 changes: 1 addition & 1 deletion db/migrations/postgres/000027_add_operations_input.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
BEGIN;
ALTER TABLE operations RENAME COLUMN info TO output;
ALTER TABLE operations ADD COLUMN input BYTEA;
ALTER TABLE operations ADD COLUMN input TEXT;
COMMIT;
2 changes: 1 addition & 1 deletion db/migrations/sqlite/000002_create_data_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ CREATE TABLE data (
datatype_version VARCHAR(64) NOT NULL,
hash CHAR(64) NOT NULL,
created BIGINT NOT NULL,
value BYTEA NOT NULL,
value TEXT NOT NULL,
blob_hash CHAR(64),
blob_public VARCHAR(1024)
);
Expand Down
2 changes: 1 addition & 1 deletion db/migrations/sqlite/000004_create_batches_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ CREATE TABLE batches (
group_hash CHAR(64),
hash CHAR(64),
created BIGINT NOT NULL,
payload BYTEA NOT NULL,
payload TEXT NOT NULL,
payload_ref VARCHAR(256),
confirmed BIGINT,
tx_type VARCHAR(64) NOT NULL,
Expand Down
2 changes: 1 addition & 1 deletion db/migrations/sqlite/000005_create_transactions_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ CREATE TABLE transactions (
created BIGINT NOT NULL,
protocol_id VARCHAR(256),
status VARCHAR(64) NOT NULL,
info BYTEA
info TEXT
);

CREATE UNIQUE INDEX transactions_id ON data(id);
Expand Down
2 changes: 1 addition & 1 deletion db/migrations/sqlite/000006_create_datatypes_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ CREATE TABLE datatypes (
version VARCHAR(64) NOT NULL,
hash CHAR(64) NOT NULL,
created BIGINT NOT NULL,
value BYTEA
value TEXT
);

CREATE UNIQUE INDEX datatypes_id ON data(id);
Expand Down
2 changes: 1 addition & 1 deletion db/migrations/sqlite/000008_create_operations_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ CREATE TABLE operations (
created BIGINT NOT NULL,
updated BIGINT,
error VARCHAR NOT NULL,
info BYTEA
info TEXT
);

CREATE UNIQUE INDEX operations_id ON operations(id);
Expand Down
2 changes: 1 addition & 1 deletion db/migrations/sqlite/000010_create_subscriptions_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ CREATE TABLE subscriptions (
filter_topics VARCHAR(256) NOT NULL,
filter_tag VARCHAR(256) NOT NULL,
filter_group VARCHAR(256) NOT NULL,
options BYTEA NOT NULL,
options TEXT NOT NULL,
created BIGINT NOT NULL
);

Expand Down
2 changes: 1 addition & 1 deletion db/migrations/sqlite/000013_create_orgs_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ CREATE TABLE orgs (
parent VARCHAR(1024),
identity VARCHAR(1024) NOT NULL,
description VARCHAR(4096) NOT NULL,
profile BYTEA,
profile TEXT,
created BIGINT NOT NULL
);

Expand Down
2 changes: 1 addition & 1 deletion db/migrations/sqlite/000014_create_nodes_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ CREATE TABLE nodes (
name VARCHAR(64) NOT NULL,
description VARCHAR(4096) NOT NULL,
dx_peer VARCHAR(256),
dx_endpoint BYTEA,
dx_endpoint TEXT,
created BIGINT NOT NULL
);

Expand Down
2 changes: 1 addition & 1 deletion db/migrations/sqlite/000015_create_config_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
CREATE TABLE config (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
config_key VARCHAR(512) NOT NULL,
config_value BYTEA NOT NULL
config_value TEXT NOT NULL
);
CREATE UNIQUE INDEX config_sequence ON config(seq);
CREATE UNIQUE INDEX config_config_key ON config(config_key);
Expand Down
2 changes: 1 addition & 1 deletion db/migrations/sqlite/000027_add_operations_input.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
ALTER TABLE operations RENAME COLUMN info TO output;
ALTER TABLE operations ADD COLUMN input BYTEA;
ALTER TABLE operations ADD COLUMN input TEXT;
3 changes: 3 additions & 0 deletions db/migrations/sqlite/000049_add_blobs_size_and_name.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,8 @@ ALTER TABLE blobs ADD size BIGINT;
ALTER TABLE data ADD blob_name VARCHAR(1024);
ALTER TABLE data ADD blob_size BIGINT;

UPDATE blobs SET size = 0;
UPDATE data SET blob_size = 0, blob_name = '';

CREATE INDEX data_blob_name ON data(blob_name);
CREATE INDEX data_blob_size ON data(blob_size);
62 changes: 50 additions & 12 deletions docs/reference/api_query_syntax.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ nav_order: 1

REST collections provide filter, `skip`, `limit` and `sort` support.
- The field in the message is used as the query parameter
- Syntax: `field=[modifiers][operator]match-string`
- When multiple query parameters are supplied these are combined with AND
- When the same query parameter is supplied multiple times, these are combined with OR

Expand All @@ -38,15 +39,52 @@ This states:

Table of filter operations, which must be the first character of the query string (after the `=` in the above URL path example)

| Operator | Description |
|----------|-----------------------------------|
| (none) | Equal |
| `!` | Not equal |
| `<` | Less than |
| `<=` | Less than or equal |
| `>` | Greater than |
| `>=` | Greater than or equal |
| `@` | Containing - case sensitive |
| `!@` | Not containing - case sensitive |
| `^` | Containing - case insensitive |
| `!^` | Not containing - case insensitive |
### Operators

Operators are a type of comparison operation to
perform against the match string.

| Operator | Description |
|----------|------------------------------------|
| `=` | Equal |
| (none) | Equal (shortcut) |
| `@` | Containing |
| `^` | Starts with |
| `$` | Ends with |
| `<<` | Less than |
| `<` | Less than (shortcut) |
| `<=` | Less than or equal |
| `>>` | Greater than |
| `>` | Greater than (shortcut) |
| `>=` | Greater than or equal |

> Shortcuts are only safe to use when your match
> string starts with `a-z`, `A-Z`, `0-9`, `-` or `_`.

### Modifiers

Modifiers can appear before the operator, to change its
behavior.

| Modifier | Description |
|----------|------------------------------------|
| `!` | Not - negates the match |
| `:` | Case insensitive |
| `?` | Treat empty match string as null |

## Detailed examples

| Example | Description |
|--------------|--------------------------------------------|
| `cat` | Equals "cat" |
| `=cat` | Equals "cat" (same) |
| `!=cat` | Not equal to "cat" |
| `:=cat` | Equal to "CAT", "cat", "CaT etc. |
| `!:cat` | Not equal to "CAT", "cat", "CaT etc. |
| `=!cat` | Equal to "!cat" (! is after operator) |
| `^cats/` | Starts with "cats/" |
| `$_cat` | Ends with with "_cat" |
| `!:^cats/` | Does not start with "cats/", "CATs/" etc. |
| `!$-cat` | Does not end with "-cat" |
| `?=` | Is null |
| `!?=` | Is not null |
19 changes: 5 additions & 14 deletions docs/swagger/swagger.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -479,7 +479,6 @@ paths:
validator:
type: string
value:
format: byte
type: string
type: object
type: array
Expand Down Expand Up @@ -633,7 +632,6 @@ paths:
validator:
type: string
value:
format: byte
type: string
type: object
type: array
Expand Down Expand Up @@ -744,7 +742,6 @@ paths:
- definition
type: string
value:
format: byte
type: string
version:
type: string
Expand Down Expand Up @@ -1070,6 +1067,11 @@ paths:
name: validator
schema:
type: string
- description: 'Data filter field. Prefixes supported: > >= < <= @ ^ ! !@ !^'
in: query
name: value
schema:
type: string
- description: Sort field. For multi-field sort use comma separated values (or
multiple query values) with '-' prefix for descending
in: query
Expand Down Expand Up @@ -1137,7 +1139,6 @@ paths:
validator:
type: string
value:
format: byte
type: string
type: object
type: array
Expand Down Expand Up @@ -1190,7 +1191,6 @@ paths:
validator:
type: string
value:
format: byte
type: string
type: object
multipart/form-data:
Expand Down Expand Up @@ -1247,7 +1247,6 @@ paths:
validator:
type: string
value:
format: byte
type: string
type: object
description: Success
Expand Down Expand Up @@ -1310,7 +1309,6 @@ paths:
validator:
type: string
value:
format: byte
type: string
type: object
description: Success
Expand Down Expand Up @@ -1787,7 +1785,6 @@ paths:
- definition
type: string
value:
format: byte
type: string
version:
type: string
Expand Down Expand Up @@ -1834,7 +1831,6 @@ paths:
- definition
type: string
value:
format: byte
type: string
version:
type: string
Expand All @@ -1860,7 +1856,6 @@ paths:
- definition
type: string
value:
format: byte
type: string
version:
type: string
Expand All @@ -1886,7 +1881,6 @@ paths:
- definition
type: string
value:
format: byte
type: string
version:
type: string
Expand Down Expand Up @@ -1941,7 +1935,6 @@ paths:
- definition
type: string
value:
format: byte
type: string
version:
type: string
Expand Down Expand Up @@ -2001,7 +1994,6 @@ paths:
- definition
type: string
value:
format: byte
type: string
version:
type: string
Expand Down Expand Up @@ -2726,7 +2718,6 @@ paths:
validator:
type: string
value:
format: byte
type: string
type: object
type: array
Expand Down
Loading