This repository goes hand-in-hand with our YouTube video walk-through on how to run Ceramic Recon (Rust implementation) locally.
The major themes we will cover are:
- Guided Ceramic node setup (using Ceramic-One, i.e. our Rust implementation)
- ComposeDB data modeling directly relevant to fulfill the spec
- Data interactions (querying, mutations, filtering)
We will be using this repository for part of our workshop, so let's first clone the codebase so we don't have to worry about it later (you will need npm installed):
git clone https://github.com/ceramicstudio/recon-starter && cd recon-starter && npm installYou can open this repository in your code editor of choice for the second half of this workshop. This implementation mimics the Kubo RPC API and relies on https://github.com/ceramicnetwork/js-ceramic for the remaining logic.
This workshop will use our rust-ceramic implementation
You can find our external Rust-Ceramic instructions here.
Assuming we will all be operating on macs, open a new terminal and:
- Install nvm (if not already installed)
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash- Select node v20:
nvm install v20
nvm use 20- Install js-ceramic (with nighly builds):
npm install --location=global @ceramicnetwork/cli@nightly- Download Rust-Ceramic from binary distribution:
curl -LO https://github.com/ceramicnetwork/rust-ceramic/releases/download/v0.13.0/ceramic-one_aarch64-apple-darwin.tar.gz
tar zxvf ceramic-one_aarch64-apple-darwin.tar.gzOpen Finder, double click the ceramic-one.pkg file to start the install
After installation, copy the binary:
sudo cp /Applications/ceramic-one /usr/local/bin/curl -LO https://github.com/ceramicnetwork/rust-ceramic/releases/download/v0.13.0/ceramic-one_x86_64-unknown-linux-gnu.tar.gz
tar zxvf ceramic-one_x86_64-unknown-linux-gnu.tar.gz
dpkg -i ceramic-one.deb- Start rust-ceramic (must be started first as js-ceramic needs the endpoint on startup):
ceramic-one daemon- In a new terminal, start js-ceramic with
CERAMIC_RECON_MODEenabled and pointing to the rust-ceramic instance:
nvm use 20
CERAMIC_RECON_MODE="true" ceramic daemon --ipfs-api http://localhost:5001We now have our Ceramic node running on port 7007. In the following steps, we will deploy our composite onto our node.
Troubleshooting
If you received error messages, please first ensure that you're running node v20 in the terminals you're running the above commands in. Next, if you've previously ran Ceramic locally, the legacy data from those sessions might also be causing issues. Go into ~/.ceramic from your machine's root directory (if exists) and delete the statestore subdirectory and the indexing.sqlite file before trying the commands again.
Back in your code editor's terminal (or a terminal opened to the software-trust-workshop directory), configure an admin DID our node will use in order to deploy our composites:
nvm use 20
npm install -g @composedb/clicomposedb did:generate-private-keyThis will print out a unique and random private key. We need to get the public key associated with this private key.
Store the private in an environment variable.
read -s DID_PRIVATE_KEY #paste in the key from the previous command and hit enter
export DID_PRIVATE_KEYNow generate the corresponding public key from the private key:
composedb did:from-private-keyOn your machine, go into the following file off your machine's root: ~/.ceramic/daemon.config.json. In your daemon.config.json file, add the did you just added into the admin-dids array. It should look something like this:
{
"anchor": {},
"http-api": {
"cors-allowed-origins": [
".*"
],
"admin-dids": ["did:key:<your new public key>"]
},
"ipfs": {
"mode": "bundled"
},
"logger": {
"log-level": 2,
"log-to-files": false
},
"network": {
"name": "testnet-clay"
},
"node": {},
"state-store": {
"mode": "fs",
"local-directory": "/Users/<you>/.ceramic/statestore"
},
"indexing": {
"db": "sqlite:///Users/<you>/.ceramic/indexing.sqlite",
"allow-queries-before-historical-sync": true,
"models": []
}
}Restart the js-ceramic process for the new config to take effect - go back to the original two terminals you began with, cancel the processes that are running, and re-run steps 6 above (which restarts the js-ceramic process - you can keep the rust-ceramic process running).
Back in the root directory of this repository you should now be able to deploy the existing composite we previously created previously by running:
composedb composite:deploy ./src/__generated__/definition.json"@context": ["https://www.w3.org/2018/credentials/v1"],
"type": ["VerifiableCredential", "PeerTrustCredential"],
"issuanceDate": "2024-02-15T07:05:56.273Z",
"issuer": "did:pkh:eip155:1:0x44dc4E3309B80eF7aBf41C7D0a68F0337a88F044",
"credentialSubject":
{
"id": "did:pkh:eip155:1:0xfA045B2F2A25ad0B7365010eaf9AC2Dd9905895c",
"trustworthiness":
[
{
"scope": "Honesty",
"level": 0.5,
"reason": ["Alumnus"]
},
{
"scope": "Software development",
"level": 1,
"reason": ["Software engineer", "Ethereum core developer"]
},
{
"scope": "Software security",
"level": 0.5,
"reason": ["White Hat", "Smart Contract Auditor"]
}
]
},
"credentialSchema": [{
"id": "ipfs://QmcwYEnLysTyepjjtJw19oTDwuiopbCDbEcCuprCBiL7gl",
"type": "JsonSchema"
},
"proof": {}The data model above is in JWT verifiable credential format. Here's how we are thinking about how this translates to ComposeDB:
################## Account Trust Credentials
type AccountTrustSignal
@createModel(accountRelation: SET, accountRelationFields: ["recipient"], description: "An account trust signal")
@createIndex(fields: [{ path: "recipient" }])
@createIndex(fields: [{ path: "issuanceDate" }]) {
issuer: DID! @documentAccount
recipient: DID! @accountReference
issuanceDate: DateTime!
trustWorthiness: [AccountTrustTypes!]! @list(maxLength: 1000)
proof: String! @string(maxLength: 10000)
}
type AccountTrustTypes {
scope: String! @string(maxLength: 1000)
level: Float!
reason: [String] @string(maxLength: 1000) @list(maxLength: 100)
}Use of SET
In ComposeDB, you can define relations between a schema model instance and the Ceramic account that controls that instance in 3 ways - SINGLE, LIST, AND SET. SINGLE requires there only ever be 1 model instance for that schema per account. LIST allows there to be an unlimited number of instances associated with that account. Conversely, SET allows developers to enforce a unique list of instances associated with an account based on a certain subfield (or set of subfields).
In the schema above, by indicating SET with the accountRelationFields array set to "recipient", we are ensuring that user A can only create 1 instance that points to user B.
For more information, read our SET RFC.
Use of @createIndex
The @createIndex directive instructs your ComposeDB node to build an index on the defined field, allowing developers to perform filters and ordering based on those indexes. Note that this does NOT work for embedded objects or enums.
Issuer
Any field definition marked as DID! @documentAccount will be automatically filled based on the authenticated account creating or updating a model instance (meaning it does not need to be manually inputted).
Proof
The plaintext fields in the AccountTrustSignal model provides all the information we need to read from directly. However, we've left in the "proof" field to accommodate any portable verified data object - for example, a stringified VC signed by the authenticated user.
Original JSON representation example:
"id": "ipfs://QmPTqvH3vm6qcZSGqAUsq78MQa9Ctb56afRZg1WJ5sKLiu",
"type": ["VerifiableCredential", "SecurityReportCredential"],
"issuanceDate": "2024-02-15T07:05:56.273Z",
"issuer": "did:pkh:eth:0x44dc4E3309B80eF7aBf41C7D0a68F0337a88F044",
"credentialSubject":
{
"id": "snap://CLwZocaUEbDErtQAsybaudZDJq65a8AwlEFgkGUpmAQ=",
"securityStatus": "Unsecured",
"securityFindings": [
{
"criticality": 1,
"type": "Key leak",
"description": "`snap_getBip44Entropy` makes the parent key accessible"
"lang": "en"
},
{
"criticality": 0.5,
"type": "Buffer Overflow"
},
{
"criticality": 0.25,
"type": "Phishing"
},
{
"criticality": 0,
"type": "Data leak",
"description": "API can communicate data to a centralized server"
},
]
},
"proof": {}ComposeDB interpretation:
type SecurityAudit
@createModel(accountRelation: SET, accountRelationFields: ["subjectId"], description: "A security audit")
@createIndex(fields: [{ path: "subjectId" }])
@createIndex(fields: [{ path: "issuanceDate" }])
@createIndex(fields: [{ path: "securityStatus" }]) {
issuer: DID! @documentAccount
subjectId: String! @string(maxLength: 1000)
issuanceDate: DateTime!
securityStatus: Boolean!
securityFindings: [Findings!]! @list(maxLength: 1000)
proof: String! @immutable @string(maxLength: 10000)
reviews: [AuditReview] @relationFrom(model: "AuditReview", property: "auditId")
}
type Findings {
criticality: Float!
type: String! @string(maxLength: 1000)
description: String @string(maxLength: 1000)
lang: String! @string(maxLength: 2)
}Use of SET
It's highly relevant here for us to limit 1 security audit an account can issue against any 1 software verion. In this case, we've represented the identifier for a snap as subjectId (for example, "snap://CLwZocaUEbDErtQAsybaudZDJq65a8AwlEFgkGUpmAQ=").
Use of @immutable
Since our system will allow users to generate reviews of audits, we want to ensure that the values of those audits don't change over time (thus nullifying reviews that point to those audits). It's unreasonable to expect users to constantly keep watch across audits they've left reviews for to ensure they don't change over time, so we want some assurances that they won't change.
In this case, since we're assuming that the proof subfield would contain a stringified signed object (like a VC), we've identified this field as immutable, which means it cannot change after it's been initially set. Given that there's a "!" following the scalar type (String), GraphQL will not allow any model instance creation query to go through without this field filled in.
This implementation would require some client-side work to verify that the VC matches the values in the ComposeDB fields. Another solution could enforce locking of the entire document, if desired.
Relation to AuditReview
In the next section we will observe the AuditReview schema definition (which can only ever be created in relation to a specific SecurityAudit instance). This means that developers will be able to query based on the relationship between an AuditReview and the SecurityAudit it references, in addition to the opposite direction (AuditReviews associated with a specific SecurityAudit).
Original JSON representation example:
"type": ["VerifiableCredential", "ReviewCredential"],
"issuanceDate": "2024-02-15T07:05:56.273Z",
"issuer": "did:pkh:eth:0x44dc4E3309B80eF7aBf41C7D0a68F0337a88F044",
"credentialSubject":
{
"id": "ipfs://QmPTqvH3vm6qcZSGqAUsq78MQa9Ctb56afRZg1WJ5sKLiu",
"currentStatus": "Disputed",
"reason": ["Missed Vulnerability"],
},
"proof": {}ComposeDB interpretation:
type AuditReview
@createModel(accountRelation: SET, accountRelationFields: ["auditId"], description: "An audit review")
@createIndex(fields: [{ path: "auditId" }])
@createIndex(fields: [{ path: "endorsedStatus" }]) {
issuer: DID! @documentAccount
issuanceDate: DateTime!
auditId: StreamID! @documentReference(model: "SecurityAudit")
audit: SecurityAudit! @relationDocument(property: "auditId")
endorsedStatus: Boolean!
reason: [String!]! @string(maxLength: 1000) @list(maxLength: 1000)
proof: String! @immutable @string(maxLength: 10000)
}Use of SET
We want to make sure an account can only ever create 1 review per specific audit.
Relation Back to SecurityAudit
We've defined a relationship between the AuditReview and SecurityAudit schemas such that the auditId subfield must intake a Stream ID scalar that is a model instance document of a SecurityAudit. The audit subfield below it exposes that node for querying based on an AuditReview as the entrypoint. This is also what allows us to expose querying to all AuditReview instances from within a SecurityAudit.
We've set up this repository so you can run some queries to an existing production node that's indexing the models found in /composites, as well as running queries to your local node running on localhost:7007 (which you'll be able to toggle manually).
From the root of this repository (/software-trust-system) start your application in developer mode:
npm run devYou can now open your browser to http://localhost:3000/.
Under the "ComposeDB Endpoint" you will see that an external cloud node endpoint has already been provided to you. Go ahead and fill in the fields below. Once you've added at least 1 trustworthiness item, you should be able to issue a credential.
Go ahead and issue a credential with the default node endpoint given to you.
Navigate to http://localhost:3000/reads where we can verify that it synced to our node. With your local node still running, replace the value under the "ComposeDB Endpoint" text with http://localhost:7007. Next, if you run the default query within the GetAccountTrustSignals tab, you should be able to see the document you just created within the set of returned documents.
Finally, you can navigate to http://localhost:3000/mutations to explore additional pre-made queries we've constructed across the other schemas.