The snapstream cli tool allows you to inspect any Topic
or Cache
:
snapstream --help
When providing an action; topic or cache, the configurations can be saved to a config file; ~/.snapstreamcfg, which may look like this:
[ { "type": "Topic", "name": "demo", "conf": { "bootstrap.servers": "localhost:29091", "sasl.username": "\\$ConnectionString", "sasl.password": "$MY_PASSWORD" }, "schema_path": "schemas/my-schema.avsc", "secrets_base_path": "/etc/secrets" } ]
Environment variables may be referenced by prepending the $
character. Which can be escaped by prepending \\
in front of $
.
Starting off with a cache that has a single entry:
from snapstream import Cache c = Cache('db') c['123'] = {'timestamp': 123, 'value': 'A story about jack and james.'}
There are several ways to query the cache:
snapstream cache db # show all values in cache snapstream cache db -k "123" # values where key is "123" snapstream cache db -k "^\d" # values where key starts with a digit
Regular expressions also work on values:
snapstream cache db -v "^(?=.*?\bjack\b).*?\bjames\b.*"
>>> key: 123 {'timestamp': 123, 'value': 'A story about jack and james.'}
Fields can be filtered when a dictionary is returned:
snapstream cache db -c "value," # column names are comma separated
>>> key: 123 {'value': 'A story about jack and james.'}
To see the live files of the cache and the total size, pass the --stats
flag:
snapstream cache db --stats
Statistics: [ { "name": "/000009.sst", "size": 1059, "level": 0, "start_key": "123", "end_key": "123", "num_entries": 1, "num_deletions": 0 } ] Folder size: 0.034 mb
The same filtering logic applies for topics.
Pass an avro schema filepath to deserialize avro messages:
snapstream topic my-topic --schema schemas/my-schema.avsc
>>> timestamp: 2023-05-20T12:00:00.000000+00:00 >>> offset: 0 >>> key: '123' {'value': 'A story about jack and james.'}
Start reading from a given offset:
snapstream topic my-topic -o -1 # read from latest (default) snapstream topic my-topic -o -2 # read from start snapstream topic my-topic -o 33 # read from specific offset
>>> timestamp: 2023-05-20T12:00:00.000000+00:00 >>> offset: 33 >>> key: '456' {'timestamp': 156, 'value': 'They discovered two pillars and a stairway.'}
The --stats
flag is not available for topics.