Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle cache misses and RPC methods that are not in the config file #16

Merged
merged 33 commits into from Oct 3, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
2510fc7
Updates to settings to work for caching Metaplex info
TomLisankie Aug 26, 2021
894726b
Remove unnecessary exports
TomLisankie Aug 26, 2021
bd9f804
Fix type error
TomLisankie Aug 31, 2021
cd16e0a
Change RPC endpoint used
TomLisankie Aug 31, 2021
a7c63a5
Fix issue of variable values not being used as property values by usi…
TomLisankie Sep 1, 2021
56f905c
Deanonimize precaching function
TomLisankie Sep 6, 2021
817a4fd
Create handleCacheMiss function
TomLisankie Sep 6, 2021
f23a38b
Had forgotten to actually call the async function in app.post
TomLisankie Sep 6, 2021
d7a07c1
Start writing code for handling cache misses
TomLisankie Sep 6, 2021
613e7ab
Add ability for server to add new handler methods when the config did…
TomLisankie Sep 7, 2021
3caf2d9
Update condition for whether RPC method is found to reflect changes f…
TomLisankie Sep 7, 2021
397bf29
Remove unused variable
TomLisankie Sep 7, 2021
10f12b6
Update if condition to use a Set since the previous condition was usi…
TomLisankie Sep 7, 2021
83c1aa2
Add a TODO
TomLisankie Sep 7, 2021
91ba7ab
Reform flow for incoming JSON-RPC call
TomLisankie Sep 13, 2021
1a067d6
Implement function for asking Writer for a given value during a cache…
TomLisankie Sep 13, 2021
ac5fd38
Remove unnecessary code in the Writer
TomLisankie Sep 13, 2021
02697ea
Write makeSolanaRPCRequest function
TomLisankie Sep 13, 2021
f6656e1
Write populateCacheWithResults function
TomLisankie Sep 13, 2021
3e7c280
Write getCachedValue function
TomLisankie Sep 14, 2021
f6d3363
Catch errors when getting response from writer
TomLisankie Sep 14, 2021
f0088e9
Use name writerClient instead of client
TomLisankie Sep 14, 2021
bf1ef0b
Dockerization
TomLisankie Sep 15, 2021
bdfe6ad
Update build script
TomLisankie Sep 15, 2021
db11a4f
Add rewrite of reader
TomLisankie Oct 1, 2021
1c83d8c
Add rewrite of writer
TomLisankie Oct 1, 2021
e8a624c
Remove old versions of the reader and writer
TomLisankie Oct 1, 2021
acb33b4
Add jest and supertest dependencies
TomLisankie Oct 1, 2021
14e478f
Remove test-client.js because we have automated test suites now
TomLisankie Oct 1, 2021
6d35843
Update Docker files and various config files
TomLisankie Oct 1, 2021
07f864f
Update .env.default
TomLisankie Oct 1, 2021
d23bfb3
Cast bodyParser.json() as a RequestHandler
TomLisankie Oct 1, 2021
d4dfc9e
Merge branch 'main' into handle-cache-miss-and-other-rewrites
TomLisankie Oct 1, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
9 changes: 1 addition & 8 deletions .env.default
@@ -1,9 +1,2 @@
ENV=local
REDIS_SERVER_PRIMARY_URL=localhost
REDIS_SERVER_READ_URL=localhost
REDIS_SERVER_PORT=6379
READER_PORT=3000
WRITER_PORT=3001
READER_CONTAINER_IMAGE_REPO_URL=none
WRITER_CONTAINER_IMAGE_REPO_URL=none
AWS_REGION=none
WRITER_URL=http://localhost:3001
12 changes: 7 additions & 5 deletions rpc-cache-writer/Dockerfile → Dockerfile.reader
@@ -1,12 +1,14 @@
FROM node:15-buster
FROM node:16-buster

ARG port=3000

COPY . .
ARG port=2999
WORKDIR /app
COPY ./package.json /app/

RUN npm install

COPY . /app/

EXPOSE $port

# Command can be overwritten by providing a different command in the template directly.
CMD ["node", "dist/lib/rpc-cache-writer/src/writer.js"]
CMD ["node", "dist/reader/index.js"]
15 changes: 15 additions & 0 deletions Dockerfile.writer
@@ -0,0 +1,15 @@
FROM node:16-buster

ARG port=3001

WORKDIR /app
COPY ./package.json /app/

RUN npm install

COPY . /app/

EXPOSE $port

# Command can be overwritten by providing a different command in the template directly.
CMD ["node", "dist/writer/index.js"]
6 changes: 6 additions & 0 deletions babel.config.js
@@ -0,0 +1,6 @@
module.exports = {
presets: [
["@babel/preset-env", { targets: { node: "current" } }],
"@babel/preset-typescript",
],
};
9 changes: 0 additions & 9 deletions babel.config.json

This file was deleted.

9 changes: 2 additions & 7 deletions build.sh
Expand Up @@ -3,10 +3,5 @@
echo "Compiling TypeScript..."
tsc

# Build and deploy Reader function along with API Gateway endpoint using AWS SAM CLI
echo "Building Reader image..."
docker build --build-arg port=$READER_PORT -t rpc-cache-reader -f rpc-cache-reader/Dockerfile .

# Build and deploy Writer Docker image to ECS
echo "Building Writer image..."
docker build --build-arg port=$READER_PORT -t rpc-cache-writer -f rpc-cache-writer/Dockerfile .;
echo "Building Docker Compose images"
sudo docker-compose build --pull
23 changes: 23 additions & 0 deletions docker-compose.yml
@@ -0,0 +1,23 @@
version: "3.9"
services:
reader:
build:
context: .
dockerfile: Dockerfile.reader
ports:
- "2999:2999"
depends_on:
- redis
- writer
writer:
build:
context: .
dockerfile: Dockerfile.writer
ports:
- "3001:3001"
depends_on:
- redis
redis:
image: "redis"
ports:
- "6379:6379"
94 changes: 27 additions & 67 deletions package.json
@@ -1,77 +1,37 @@
{
"name": "rpc-cache-server",
"version": "0.0.3",
"description": "A server to cache results of (expensive) RPC requests made to the Solana JSON RPC API",
"author": "Monadical",
"license": "APACHE",
"private": false,
"repository": {
"type": "git",
"url": "https://github.com/Monadical-SAS/rpc-cache-server.git"
},
"bugs": {
"url": "https://github.com/Monadical-SAS/rpc-cache-server/issues"
},
"homepage": "https://github.com/Monadical-SAS/rpc-cache-server#readme",
"keywords": [
"rpc",
"solana",
"cache"
],
"browser": {
"./dist/lib/rpc-cache-connection/src/index.js": "./dist.browser/lib/index.browser.esm.js"
},
"main": "./dist/lib/rpc-cache-connection/src/index.js",
"name": "new-rpc-caching",
"version": "1.0.0",
"main": "dist/index.js",
"author": "Thomas Lisankie",
"license": "MIT",
"dependencies": {
"@solana/web3.js": "^1.10.1",
"aws-sdk": "^2.905.0",
"axios": "^0.21.1",
"body-parser": "^1.19.0",
"bs58": "^4.0.1",
"@solana/web3.js": "^1.29.1",
"axios": "^0.21.4",
"cors": "^2.8.5",
"dotenv": "^9.0.2",
"dotenv": "^10.0.0",
"express": "^4.17.1",
"json-rpc-2.0": "^0.2.16",
"node-fetch": "^2.6.1",
"redis": "^3.1.2",
"typescript": "^4.2.4"
"jayson": "^3.6.4",
"json-rpc-2.0": "^0.2.19",
"redis": "^3.1.2"
},
"files": [
"dist/lib",
"dist.browser/lib"
],
"scripts": {
"lint:fix": "eslint --fix '*/{src,test}/**/*.ts' && prettier --write '*/{src,test}/**/*.ts'",
"build": "tsc && cross-env NODE_ENV=production rollup -c"
"test": "jest"
},
"devDependencies": {
"@babel/core": "^7.14.3",
"@babel/plugin-proposal-class-properties": "^7.13.0",
"@babel/plugin-transform-runtime": "^7.14.3",
"@babel/preset-env": "^7.14.2",
"@babel/preset-typescript": "^7.13.0",
"@babel/register": "^7.13.16",
"@rollup/plugin-babel": "^5.3.0",
"@rollup/plugin-commonjs": "^19.0.0",
"@rollup/plugin-json": "^4.1.0",
"@rollup/plugin-node-resolve": "^13.0.0",
"@rollup/plugin-replace": "^2.4.2",
"@types/aws-lambda": "^8.10.76",
"@types/bs58": "^4.0.1",
"@types/cors": "^2.8.10",
"@types/express": "^4.17.11",
"@types/node": "^15.0.2",
"@types/node-fetch": "^2.5.10",
"@types/redis": "^2.8.28",
"@typescript-eslint/eslint-plugin": "^4.23.0",
"@typescript-eslint/parser": "^4.23.0",
"babel-plugin-root-import": "^6.6.0",
"cross-env": "^7.0.3",
"eslint": "^7.26.0",
"eslint-config-prettier": "^8.3.0",
"prettier": "^2.3.0",
"rollup": "^2.48.0",
"rollup-plugin-node-polyfills": "^0.2.1",
"typescript": "^4.2.4"
"@babel/core": "^7.15.5",
"@babel/preset-env": "^7.15.6",
"@babel/preset-typescript": "^7.15.0",
"@types/cors": "^2.8.12",
"@types/express": "^4.17.13",
"@types/jest": "^27.0.2",
"@types/node": "^16.10.1",
"@types/redis": "^2.8.32",
"@types/redis-mock": "^0.17.0",
"@types/supertest": "^2.0.11",
"babel-jest": "^27.2.2",
"jest": "^27.2.2",
"redis-mock": "^0.56.3",
"supertest": "^6.1.6",
"typescript": "^4.4.3"
}
}
13 changes: 0 additions & 13 deletions rpc-cache-reader/.aws-sam/build.toml

This file was deleted.

12 changes: 0 additions & 12 deletions rpc-cache-reader/Dockerfile

This file was deleted.

13 changes: 0 additions & 13 deletions rpc-cache-reader/build.toml

This file was deleted.

11 changes: 0 additions & 11 deletions rpc-cache-reader/samconfig.toml

This file was deleted.

94 changes: 0 additions & 94 deletions rpc-cache-reader/src/reader-aws-lambda.ts

This file was deleted.