Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/publish_package_client.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
with:
node-version: "20"
- run: npm ci
- run: npx lerna run build
- run: npx lerna run build --scope '{@map-colonies/detiler-common,@map-colonies/detiler-client}'
- uses: JS-DevTools/npm-publish@v3
with:
token: ${{ secrets.NPM_TOKEN }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish_package_common.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
with:
node-version: "20"
- run: npm ci
- run: npx lerna run build
- run: npx lerna run build --scope=@map-colonies/detiler-common
- uses: JS-DevTools/npm-publish@v3
with:
token: ${{ secrets.NPM_TOKEN }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/pull_request.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:

strategy:
matrix:
node: [18.x, 20.x]
node: [20.x]

steps:
- name: Check out TS Project Git repository
Expand Down Expand Up @@ -85,7 +85,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
node: [18.x, 20.x]
node: [20.x]

steps:
- name: Check out Git repository
Expand Down
2 changes: 1 addition & 1 deletion .redocly.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ lint:

rules:
boolean-parameter-prefixes:
severity: error
severity: warn
prefixes: ['should', 'is', 'has']
no-unused-components:
severity: error
Expand Down
58 changes: 35 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ data is colored in relation to some metric (state, update count, skip count, cur
"updateCount":13,
"renderCount":11,
"skipCount":2,
"coolCount":0,
"geoshape":"POLYGON ((34.453125 31.541748046875, 34.453125 31.53076171875, 34.464111328125 31.53076171875, 34.464111328125 31.541748046875, 34.453125 31.541748046875))",
"coordinates":"34.458618, 31.536255"
}
Expand All @@ -55,36 +56,47 @@ to achieve runtime variables we inject for each variable it's value in runtime i
see [.env.production](/packages/frontend/config/.env.production) and [env.sh](/packages/frontend/env.sh).

## Redis
### redis search index creation:
### redis search tile details index creation:
```
FT.CREATE tileDetailsIdx ON JSON PREFIX 1 tile:
SCHEMA $.kit AS kit TEXT $.updatedAt AS updatedAt NUMERIC $.renderedAt AS renderedAt NUMERIC $.createdAt AS createdAt NUMERIC $.updateCount AS updateCount NUMERIC $.renderCount AS renderCount NUMERIC $.skipCount AS skipCount NUMERIC $.coordiantes AS coordinates GEO $.geoshape AS geoshape GEOSHAPE SPHERICAL $.state AS state NUMERIC $.states[*] AS states NUMERIC $.z AS z NUMERIC $.x AS x NUMERIC $.y AS y NUMERIC
FT.CREATE tileDetailsIdx ON JSON PREFIX 1 tile: SCHEMA $.kit AS kit TEXT $.updatedAt AS updatedAt NUMERIC $.renderedAt AS renderedAt NUMERIC $.createdAt AS createdAt NUMERIC $.geoshape AS geoshape GEOSHAPE SPHERICAL $.state AS state NUMERIC $.states[*] AS states NUMERIC $.z AS z NUMERIC $.x AS x NUMERIC $.y AS y NUMERIC
```

### redis post processing:
preferably we would use the `Redis Gears` module, but in the meantime the deprecated `Triggers and Functions` module is in use.
### redis search cooldowns index creation:
```
FT.CREATE cooldownIdx ON JSON PREFIX 1 cooldown: SCHEMA $.kits[*] AS kits TAG $.minZoom AS minZoom NUMERIC $.maxZoom AS maxZoom NUMERIC $.enabled AS enabled TAG $.geoshape AS geoshape GEOSHAPE SPHERICAL
```

the following post processing function is being used to maintain additional metadata for each kit - it's `maxState` and `maxUpdatedAt`
### redis post processing:
using `Redis Gears` the following post processing function is being used to maintain additional metadata for each kit - it's `maxState` and `maxUpdatedAt` see [maintain_kit_metadata.py](/packages/backend/gears/maintain_kit_metadata.py) for full documented python code

execute with `redis-cli`
```
TFUNCTION LOAD REPLACE "#!js name=LibName api_version=1.0\n
function getMaximum(client, data) {
currentKit = client.call('json.get', data.key, 'kit');
currentState = client.call('json.get', data.key, 'state');
currentUpdatedAt = client.call('json.get', data.key, 'updatedAt');
key = 'kit:' + currentKit.slice(1, currentKit.length - 1);
maxState = client.call('hget', key, 'maxState');
if(maxState < currentState) {
client.call('hset', key, 'maxState', currentState);
}
maxUpdatedAt = client.call('hget', key, 'maxUpdatedAt');
if(maxUpdatedAt < currentUpdatedAt) {
client.call('hset', key, 'maxUpdatedAt', currentUpdatedAt);
}
}

redis.registerKeySpaceTrigger('KitLibrary', 'tile:', getMaximum);"
RG.PYEXECUTE "
def extract_data(record):\n
data_key = record['key']\n

kit = execute('JSON.GET', data_key, 'kit')\n
state = execute('JSON.GET', data_key, 'state')\n
updated_at = execute('JSON.GET', data_key, 'updatedAt')\n
return { 'kit': kit[1:-1], 'state': int(state), 'updated_at': int(updated_at) }\n

def update_maximums(data):\n
kit_key = 'kit:' + data['kit']\n

max_state = execute('HGET', kit_key, 'maxState')\n
max_state = int(max_state) if max_state else 0\n
if data['state'] > max_state:\n
execute('HSET', kit_key, 'maxState', data['state'])\n

max_updated_at = execute('HGET', kit_key, 'maxUpdatedAt')\n
max_updated_at = int(max_updated_at) if max_updated_at else 0\n
if data['updated_at'] > max_updated_at:\n
execute('HSET', kit_key, 'maxUpdatedAt', data['updated_at'])\n

gb = GearsBuilder()\n
gb.map(extract_data)\n
gb.foreach(update_maximums)\n
gb.register('tile:*')"
```

## Development
Expand Down
Loading