Skip to content

Commit

Permalink
fix: 🐛 properly decode tuple values in call/event inputs
Browse files Browse the repository at this point in the history 
Restored the ability to decode tuple values in inputs of contract calls
and events. This change also adds an additional flag to the ABI
configuration that will cause ethlogger to attempt to reconcile the
shape of structs from tuples if the information in the ABI definition is
sufficient to do so (this is disabled by default).
  • Loading branch information
@ziegfried
ziegfried committed Oct 13, 2020
1 parent b20d4bf commit 28edb0b
Show file tree
Hide file tree
Showing 16 changed files with 137 additions and 37 deletions.
4 changes: 4 additions & 0 deletions config.schema.json
View file
Expand Up @@ -20,6 +20,10 @@
"description": "If enabled, the ABI repository will creates hashes of all function and event signatures of an ABI\n(the hash is the fingerprint) and match it against the EVM bytecode obtained from live smart contracts\nwe encounter.",
"type": "boolean"
},
"reconcileStructShapeFromTuples": {
"description": "If enabled, ethlogger will attempt to reconcile the shape of struct definitions from decoded tuples\ndata if the ABI definition contains names for each of the tuple components. This basically turns\nthe decoded array into an key-value map, where the keys are the names from the ABI definition.",
"type": "boolean"
},
"requireContractMatch": {
"description": "If enabled, signature matches will be treated as anonymous (parameter names will be omitted from\nthe output) if a contract cannot be tied to an ABI definition via either a fingerprint match,\nor a contract address match (when the ABI file includes the address of the deployed contract).\nEnabled by default. Setting this to `false` will output parameter names for any matching signature.",
"type": "boolean"
Expand Down
1 change: 1 addition & 0 deletions defaults.ethlogger.yaml
View file
Expand Up @@ -65,6 +65,7 @@ abi:
fingerprintContracts: true
requireContractMatch: true
decodeAnonymous: true
reconcileStructShapeFromTuples: false
contractInfo:
maxCacheEntries: 25000
blockWatcher:
Expand Down
17 changes: 9 additions & 8 deletions docs/configuration.md
View file
Expand Up @@ -212,14 +212,15 @@ The checkpoint is where ethlogger keeps track of its state, which blocks have al

The ABI repository is used to decode ABI information from smart contract calls and event logs. It generates and adds some additional information in transactions and events, including smart contract method call parameter names, values and data types, as well as smart contract names associated with a particular contract address.

| Name | Type | Description |
| ---------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `directory` | `string` | If specified, the ABI repository will recursively search this directory for ABI files |
| `searchRecursive` | `boolean` | `true` to search ABI directory recursively for ABI files |
| `abiFileExtension` | `string` | Set to `.json` by default as the file extension for ABIs |
| `fingerprintContracts` | `boolean` | If enabled, the ABI repository will creates hashes of all function and event signatures of an ABI (the hash is the fingerprint) and match it against the EVM bytecode obtained from live smart contracts we encounter. |
| `requireContractMatch` | `boolean` | If enabled, signature matches will be treated as anonymous (parameter names will be omitted from the output) if a contract cannot be tied to an ABI definition via either a fingerprint match, or a contract address match (when the ABI file includes the address of the deployed contract). Enabled by default. Setting this to `false` will output parameter names for any matching signature. |
| `decodeAnonymous` | `boolean` | If enabled, ethlogger will attempt to decode function calls and event logs using a set of common signatures as a fallback if no match against any supplied ABI definition was found. |
| Name | Type | Description |
| -------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `directory` | `string` | If specified, the ABI repository will recursively search this directory for ABI files |
| `searchRecursive` | `boolean` | `true` to search ABI directory recursively for ABI files |
| `abiFileExtension` | `string` | Set to `.json` by default as the file extension for ABIs |
| `fingerprintContracts` | `boolean` | If enabled, the ABI repository will creates hashes of all function and event signatures of an ABI (the hash is the fingerprint) and match it against the EVM bytecode obtained from live smart contracts we encounter. |
| `requireContractMatch` | `boolean` | If enabled, signature matches will be treated as anonymous (parameter names will be omitted from the output) if a contract cannot be tied to an ABI definition via either a fingerprint match, or a contract address match (when the ABI file includes the address of the deployed contract). Enabled by default. Setting this to `false` will output parameter names for any matching signature. |
| `decodeAnonymous` | `boolean` | If enabled, ethlogger will attempt to decode function calls and event logs using a set of common signatures as a fallback if no match against any supplied ABI definition was found. |
| `reconcileStructShapeFromTuples` | `boolean` | If enabled, ethlogger will attempt to reconcile the shape of struct definitions from decoded tuples data if the ABI definition contains names for each of the tuple components. This basically turns the decoded array into an key-value map, where the keys are the names from the ABI definition. |

### ContractInfo

Expand Down
116 changes: 93 additions & 23 deletions src/abi/decode.ts
View file
@@ -1,6 +1,6 @@
import { createModuleDebug, TRACE_ENABLED } from '../utils/debug';
import { DataSize, getDataSize, isArrayType } from './datatypes';
import { AbiItemDefinition } from './item';
import { AbiItemDefinition, AbiInput } from './item';
import { computeSignature, encodeParam } from './signature';
import { abiDecodeParameters, Value } from './wasm';

Expand All @@ -9,21 +9,21 @@ const { trace } = createModuleDebug('abi:decode');
export interface DecodedParameter {
name?: string;
type: string;
value: Value;
value: Value | DecodedStruct;
}

export interface DecodedFunctionCall {
name: string;
signature: string;
params: DecodedParameter[];
args?: { [name: string]: Value };
args?: { [name: string]: Value | DecodedStruct };
}

export interface DecodedLogEvent {
name: string;
signature: string;
params: DecodedParameter[];
args?: { [name: string]: Value };
args?: { [name: string]: Value | DecodedStruct };
}

export function getInputSize(abi: AbiItemDefinition): DataSize {
Expand All @@ -39,19 +39,75 @@ export function getInputSize(abi: AbiItemDefinition): DataSize {
}
}

export const isTuple = (inputDef: AbiInput): boolean => inputDef.type === 'tuple';

export const encodeInputType = (inputDef: AbiInput): string =>
// eslint-disable-next-line @typescript-eslint/no-use-before-define
isTuple(inputDef) ? encodeTupleInputType(inputDef) : inputDef.type;

export function encodeTupleInputType(itemDef: AbiInput): string {
if (!itemDef.components?.length) {
// invalid tuple definition without component types
return 'tuple';
}
const serializedComponentTypes = itemDef.components.map(c => encodeInputType(c)).join(',');
return `(${serializedComponentTypes})`;
}

export type DecodedStruct = { [k: string]: Value | DecodedStruct };

export function reconcileStructFromDecodedTuple(decodedTuple: Value[], inputDef: AbiInput): DecodedStruct | null {
if (!inputDef.components?.length || !Array.isArray(decodedTuple)) {
return null;
}
const result: DecodedStruct = {};
for (const [i, component] of inputDef.components.entries()) {
if (!component.name) {
// no name provided for at least one component of the tuple - skipping struct reconciliation
return null;
}
if (isTuple(component)) {
const value = reconcileStructFromDecodedTuple(decodedTuple[i] as Value[], component);
if (value == null) {
result[component.name] = decodedTuple[i];
} else {
result[component.name] = value;
}
} else {
result[component.name] = decodedTuple[i];
}
}

return result;
}

export const reconcileStructs = (decodedValues: Value[], inputDefs: AbiInput[]): Array<Value | DecodedStruct> =>
decodedValues.map((val, i) => {
const input = inputDefs[i];
if (isTuple(input)) {
const structData = reconcileStructFromDecodedTuple(val as Value[], input);
if (structData != null) {
return structData;
}
}
return val;
});

export function decodeFunctionCall(
data: string,
abi: AbiItemDefinition,
signature: string,
anonymous: boolean
anonymous: boolean,
shouldReconcileStructs: boolean
): DecodedFunctionCall {
const inputs = abi.inputs ?? [];
const decodedParams = abiDecodeParameters(
data.slice(10),
inputs.map(i => i.type)
);
let decodedParams: Array<Value | DecodedStruct> = abiDecodeParameters(data.slice(10), inputs.map(encodeInputType));
if (shouldReconcileStructs) {
decodedParams = reconcileStructs(decodedParams as Value[], inputs);
}

const params: DecodedParameter[] = [];
const args: { [name: string]: string | number | boolean | Array<string | number | boolean> } = {};
const args: { [name: string]: Value | DecodedStruct } = {};

for (let i = 0; i < inputs.length; i++) {
const input = inputs[i];
Expand All @@ -77,11 +133,12 @@ export function decodeFunctionCall(
export function decodeBestMatchingFunctionCall(
data: string,
abis: AbiItemDefinition[],
anonymous: boolean
anonymous: boolean,
reconcileStructShapeFromTuples: boolean
): DecodedFunctionCall {
if (abis.length === 1) {
// short-circut most common case
return decodeFunctionCall(data, abis[0], computeSignature(abis[0]), anonymous);
return decodeFunctionCall(data, abis[0], computeSignature(abis[0]), anonymous, reconcileStructShapeFromTuples);
}
const abisWithSize = abis.map(abi => [abi, getInputSize(abi)] as const);
const dataLength = (data.length - 10) / 2;
Expand All @@ -90,7 +147,7 @@ export function decodeBestMatchingFunctionCall(
for (const [abi, { length, exact }] of abisWithSize) {
if (dataLength === length && exact) {
try {
return decodeFunctionCall(data, abi, computeSignature(abi), anonymous);
return decodeFunctionCall(data, abi, computeSignature(abi), anonymous, reconcileStructShapeFromTuples);
} catch (e) {
lastError = e;
if (TRACE_ENABLED) {
Expand All @@ -103,11 +160,11 @@ export function decodeBestMatchingFunctionCall(
}
}
}
// Consider dynamaic data types
// Consider dynamic data types
for (const [abi, { length, exact }] of abisWithSize) {
if (dataLength >= length && !exact) {
try {
return decodeFunctionCall(data, abi, computeSignature(abi), anonymous);
return decodeFunctionCall(data, abi, computeSignature(abi), anonymous, reconcileStructShapeFromTuples);
} catch (e) {
lastError = e;
if (TRACE_ENABLED) {
Expand All @@ -124,7 +181,7 @@ export function decodeBestMatchingFunctionCall(
// Brute-force try all ABI signatures, use the first one that doesn't throw on decode
for (const abi of abis) {
try {
return decodeFunctionCall(data, abi, computeSignature(abi), anonymous);
return decodeFunctionCall(data, abi, computeSignature(abi), anonymous, reconcileStructShapeFromTuples);
} catch (e) {
lastError = e;
}
Expand All @@ -138,13 +195,18 @@ export function decodeLogEvent(
topics: string[],
abi: AbiItemDefinition,
signature: string,
anonymous: boolean
anonymous: boolean,
shouldReconcileStructs: boolean = false
): DecodedLogEvent {
const nonIndexedTypes = abi.inputs.filter(i => !i.indexed).map(i => i.type);
const decodedData = abiDecodeParameters(data.slice(2), nonIndexedTypes);
const nonIndexedInputs = abi.inputs.filter(i => !i.indexed);
const nonIndexedTypes = nonIndexedInputs.map(encodeInputType);
let decodedData: Array<Value | DecodedStruct> = abiDecodeParameters(data.slice(2), nonIndexedTypes);
if (shouldReconcileStructs) {
decodedData = reconcileStructs(decodedData as Value[], nonIndexedInputs);
}
let topicIndex = 1;
let dataIndex = 0;
const args: { [k: string]: Value } = {};
const args: { [k: string]: Value | DecodedStruct } = {};
const params = abi.inputs.map(input => {
let value;
if (input.indexed) {
Expand All @@ -159,8 +221,15 @@ export function decodeLogEvent(
`Expected data in topic index=${topicIndex - 1}, but topics length is ${topics.length}`
);
}
const [decoded] = abiDecodeParameters(rawValue.slice(2), [input.type]);

const [decoded] = abiDecodeParameters(rawValue.slice(2), [encodeInputType(input)]);
value = decoded;
if (shouldReconcileStructs && isTuple(input)) {
const reconciled = reconcileStructFromDecodedTuple(decoded as Value[], input);
if (reconciled != null) {
value = reconciled;
}
}
}
} else {
value = decodedData[dataIndex++];
Expand All @@ -185,14 +254,15 @@ export function decodeBestMatchingLogEvent(
data: string,
topics: string[],
abis: AbiItemDefinition[],
anonymous: boolean
anonymous: boolean,
reconcileStructShapeFromTuples: boolean
): DecodedFunctionCall {
// No need to prioritize and check event logs for hash collisions since with the longer hash
// collisions are very unlikely
let lastError: Error | undefined;
for (const abi of abis) {
try {
return decodeLogEvent(data, topics, abi, computeSignature(abi), anonymous);
return decodeLogEvent(data, topics, abi, computeSignature(abi), anonymous, reconcileStructShapeFromTuples);
} catch (e) {
lastError = e;
if (TRACE_ENABLED) {
Expand Down
8 changes: 6 additions & 2 deletions src/abi/repo.ts
View file
Expand Up @@ -255,7 +255,9 @@ export class AbiRepository implements ManagedResource {

public decodeFunctionCall(data: string, matchParams: AbiMatchParams): DecodedFunctionCall | undefined {
const sigHash = data.slice(2, 10);
return this.abiDecode(sigHash, matchParams, (abi, anon) => decodeBestMatchingFunctionCall(data, abi, anon));
return this.abiDecode(sigHash, matchParams, (abi, anon) =>
decodeBestMatchingFunctionCall(data, abi, anon, this.config.reconcileStructShapeFromTuples)
);
}

public decodeLogEvent(logEvent: RawLogResponse, matchParams: AbiMatchParams): DecodedLogEvent | undefined {
Expand All @@ -272,7 +274,9 @@ export class AbiRepository implements ManagedResource {
const sigHash = logEvent.topics[0].slice(2);
const { data, topics } = logEvent;

return this.abiDecode(sigHash, matchParams, (abi, anon) => decodeBestMatchingLogEvent(data, topics, abi, anon));
return this.abiDecode(sigHash, matchParams, (abi, anon) =>
decodeBestMatchingLogEvent(data, topics, abi, anon, this.config.reconcileStructShapeFromTuples)
);
}

public async shutdown() {
Expand Down

0 comments on commit 28edb0b

Please sign in to comment.