Skip to content
This repository has been archived by the owner on Jul 9, 2021. It is now read-only.

Commit

Permalink
abi-gen/Py: fix incorrect method return types and other small issues (#…
Browse files Browse the repository at this point in the history 
…2345)

* .gitignore gen'd Python staking contract wrappers

* abi-gen/test-cli: check Python type hints in lint

* sra_client.py: Update doc for replicating examples

* abi-gen/Py: fix call() return type incl. tx hash

Previously, generated wrappers for contract methods were including type
hints that suggested that a call() (as opposed to a send_transaction())
might return either the underlying return type or a transaction hash.
This doesn't make sense because a call() will never return a TX hash.
Now, the type hint just has the return type of the underlying method.

* abi-gen: fix test_cli:lint checking wrong code

test_cli:lint is meant to be a rudimentary test of the code generated by
abi-gen.  However, previously, this script was incorporated into `yarn
lint`, and in CircleCI `static-tests` runs independently of `build`.
Consequently, the runs of test_cli:lint were checking the OLD code,
which was previously generated and checked in to git, NOT the code
generated with the version of abi-gen represented by the git repo.  Now,
test_cli:lint happens during `yarn test` rather than `yarn lint`,
because `yarn test` IS dependent on `yarn build`.

* contract_wrappers.py: fix misplaced doc

Previously, the routines `order_to_jsdict()` and `jsdict_to_order()`
were moved from contract_wrappers.exchange.types to
contract_wrappers.order_conversions.  However, the module-level
docstring describing those routines was accidentally left behind in
exchange.types.

* abi-gen/Py: stop documenting return types for TXs

Previously the send_transaction() interface included docstring
documentation for the return types of the contract method, but that
doesn't make any sense because send_transaction() returns a transaction
hash rather than any actual return values.

* abi-gen/Py: stop gen'ing send_tx for const methods

* abi-gen/Py: add build_tx to contract methods

* abi-gen/Py: fix incorrect method return types

Fixes #2298 .

* abi-gen/Py: rm validator arg to no-input methods

* abi-gen: mv Py Handlebars helpers to own module

Move all existing Python-related Handlebars helpers to the newly created
python_handlebars_helpers module.

* abi-gen: refactor internal interface

No functionality is changed.  Sole purpose of this commit is to
facilitate an upcoming commit.

* abi-gen: refactor internal interface

No functionality is changed.  Sole purpose of this commit is to
facilitate an upcoming commit.

* abi-gen/Py: name tuples w/internalType, not hash

Use the new `internalType` field on the `DataItem`s in the contract
artifact to give generated tuple classes a better name than just hashing
their component field names.

* Fix CI errors

* abi-gen/Py/wrapper: make internal member private

* Update CHANGELOGs
  • Loading branch information
@feuGeneA
feuGeneA committed Nov 15, 2019
1 parent 9e3cc37 commit df97b20
Show file tree
Hide file tree
Showing 31 changed files with 679 additions and 847 deletions.
2 changes: 2 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,8 @@ python-packages/contract_wrappers/src/zero_ex/contract_wrappers/i_validator/__in
python-packages/contract_wrappers/src/zero_ex/contract_wrappers/i_wallet/__init__.py
python-packages/contract_wrappers/src/zero_ex/contract_wrappers/multi_asset_proxy/__init__.py
python-packages/contract_wrappers/src/zero_ex/contract_wrappers/order_validator/__init__.py
python-packages/contract_wrappers/src/zero_ex/contract_wrappers/staking/__init__.py
python-packages/contract_wrappers/src/zero_ex/contract_wrappers/staking_proxy/__init__.py
python-packages/contract_wrappers/src/zero_ex/contract_wrappers/static_call_proxy/__init__.py
python-packages/contract_wrappers/src/zero_ex/contract_wrappers/weth9/__init__.py
python-packages/contract_wrappers/src/zero_ex/contract_wrappers/zrx_token/__init__.py
Expand Down
20 changes: 20 additions & 0 deletions packages/abi-gen/CHANGELOG.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,26 @@
{
"note": "Refactored TS wrapper templates to result in a more succint interface. See https://github.com/0xProject/0x-monorepo/pull/2325 for details.",
"pr": 2284
},
{
"note": "Python: Corrected return types and values for call() interface to generated method wrappers. (Fixes #2298)",
"pr": 2345
},
{
"note": "Python: Stopped generating send_transaction() interface for constant (view/pure) methods",
"pr": 2345
},
{
"note": "Python: Added a build_transaction() interface to contract method classes",
"pr": 2345
},
{
"note": "Python: Removed `validator` argument to contract method classes for methods that don't have any inputs",
"pr": 2345
},
{
"note": "Python: Changed the names of generated tuples to use the `internalType` field in the ABI, if it's present, resulting in human readable struct names rather than hashes of component field names.",
"pr": 2345
}
]
},
Expand Down
7 changes: 4 additions & 3 deletions packages/abi-gen/package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"main": "lib/src/index.js",
"types": "lib/src/index.d.ts",
"scripts": {
"lint": "tslint --format stylish --project . && yarn test_cli:lint",
"lint": "tslint --format stylish --project .",
"fix": "tslint --fix --format stylish --project . && yarn lint-contracts",
"clean": "shx rm -rf lib && yarn test_cli:clean",
"build": "tsc -b && yarn generate_contract_wrappers && yarn prettier_contract_wrappers && yarn test_cli:build",
Expand All @@ -18,15 +18,16 @@
"run_mocha": "(uname -s | grep -q Darwin && echo 'HACK! skipping mocha run due to https://github.com/0xProject/0x-monorepo/issues/2000') || mocha --require source-map-support/register --require make-promises-safe lib/test/*_test.js --timeout 100000 --bail --exit",
"test:coverage": "nyc npm run test --all && yarn coverage:report:lcov",
"coverage:report:lcov": "nyc report --reporter=text-lcov > coverage/lcov.info",
"test_cli": "run-s test_cli:test_typescript diff_contract_wrappers",
"test_cli": "run-p test_cli:test_typescript diff_contract_wrappers test_cli:lint",
"test_cli:clean": "rm -rf test-cli/test_typescript/lib",
"test_cli:build": "tsc --project test-cli/tsconfig.json",
"test_cli:test_typescript": "mocha --require source-map-support/register --require make-promises-safe test-cli/test_typescript/lib/**/*_test.js --timeout 100000 --bail --exit",
"test_cli:lint": "run-p test_cli:lint_solidity test_cli:lint_python # test_cli:lint_typescript # HACK: typescript lint disabled because prettier fails",
"test_cli:lint_solidity": "solhint -c ../../contracts/.solhint.json test-cli/fixtures/contracts/*.sol",
"test_cli:lint_typescript": "prettier --check ./test-cli/output/typescript/* --config ../../.prettierrc",
"test_cli:lint_python": "pip install -r test-cli/fixtures/python-requirements.txt && run-p test_cli:lint_python:black test_cli:lint_python:pylint",
"test_cli:lint_python": "pip install -r test-cli/fixtures/python-requirements.txt && run-p test_cli:lint_python:black test_cli:lint_python:mypy test_cli:lint_python:pylint",
"test_cli:lint_python:black": "black --line-length=79 --check ./test-cli/output/python/*",
"test_cli:lint_python:mypy": "MYPYPATH=./stubs mypy ./test-cli/output/python",
"test_cli:lint_python:pylint": "PYTHONPATH=../../python-packages/contract_wrappers/src pylint --rcfile=test-cli/fixtures/pylintrc ./test-cli/output/python/*",
"rebuild_and_test": "run-s build test",
"test:profiler": "SOLIDITY_PROFILER=true run-s build run_mocha profiler:report:html",
Expand Down
160 changes: 2 additions & 158 deletions packages/abi-gen/src/index.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,14 @@ import chalk from 'chalk';
import * as changeCase from 'change-case';
import { execSync } from 'child_process';
import * as cliFormat from 'cli-format';
import {
AbiDefinition,
ConstructorAbi,
ContractAbi,
DataItem,
DevdocOutput,
EventAbi,
MethodAbi,
} from 'ethereum-types';
import { AbiDefinition, ConstructorAbi, ContractAbi, DevdocOutput, EventAbi, MethodAbi } from 'ethereum-types';
import { sync as globSync } from 'glob';
import * as Handlebars from 'handlebars';
import * as _ from 'lodash';
import * as mkdirp from 'mkdirp';
import toposort = require('toposort');
import * as yargs from 'yargs';

import { registerPythonHelpers } from './python_handlebars_helpers';
import { ContextData, ContractsBackend, ParamKind } from './types';
import { utils } from './utils';

Expand Down Expand Up @@ -146,154 +138,6 @@ function registerTypeScriptHelpers(): void {
);
}

function registerPythonHelpers(): void {
Handlebars.registerHelper('equal', (lhs: any, rhs: any) => {
return lhs === rhs;
});
Handlebars.registerHelper('safeString', (str: string) => new Handlebars.SafeString(str));
Handlebars.registerHelper('parameterType', utils.solTypeToPyType.bind(utils));
Handlebars.registerHelper('returnType', utils.solTypeToPyType.bind(utils));
Handlebars.registerHelper('toPythonIdentifier', utils.toPythonIdentifier.bind(utils));
Handlebars.registerHelper('sanitizeDevdocDetails', (_methodName: string, devdocDetails: string, indent: number) => {
// wrap to 80 columns, assuming given indent, so that generated
// docstrings can pass pycodestyle checks. also, replace repeated
// spaces, likely caused by leading indents in the Solidity, because
// they cause repeated spaces in the output, and in particular they may
// cause repeated spaces at the beginning of a line in the docstring,
// which leads to "unexpected indent" errors when generating
// documentation.
if (devdocDetails === undefined || devdocDetails.length === 0) {
return '';
}
const columnsPerRow = 80;
return new Handlebars.SafeString(
`\n${cliFormat.wrap(devdocDetails.replace(/ +/g, ' ') || '', {
paddingLeft: ' '.repeat(indent),
width: columnsPerRow,
ansi: false,
})}\n`,
);
});
Handlebars.registerHelper('makeParameterDocstringRole', (name: string, description: string, indent: number) => {
let docstring = `:param ${name}:`;
if (description && description.length > 0) {
docstring = `${docstring} ${description}`;
}
return new Handlebars.SafeString(utils.wrapPythonDocstringRole(docstring, indent));
});
Handlebars.registerHelper(
'makeReturnDocstringRole',
(description: string, indent: number) =>
new Handlebars.SafeString(
utils.wrapPythonDocstringRole(`:returns: ${description.replace(/ +/g, ' ')}`, indent),
),
);
Handlebars.registerHelper(
'makeEventParameterDocstringRole',
(eventName: string, indent: number) =>
new Handlebars.SafeString(
utils.wrapPythonDocstringRole(
`:param tx_hash: hash of transaction emitting ${eventName} event`,
indent,
),
),
);
Handlebars.registerHelper('tupleDefinitions', (abisJSON: string) => {
const abis: AbiDefinition[] = JSON.parse(abisJSON);
// build an array of objects, each of which has one key, the Python
// name of a tuple, with a string value holding the body of a Python
// class representing that tuple. Using a key-value object conveniently
// filters duplicate references to the same tuple.
const tupleBodies: { [pythonTupleName: string]: string } = {};
// build an array of tuple dependencies, whose format conforms to the
// expected input to toposort, a function to do a topological sort,
// which will help us declare tuples in the proper order, avoiding
// references to tuples that haven't been declared yet.
const tupleDependencies: Array<[string, string]> = [];
for (const abi of abis) {
let parameters: DataItem[] = [];
if (abi.hasOwnProperty('inputs')) {
// HACK(feuGeneA): using "as MethodAbi" below, but abi
// could just as well be ConstructorAbi, EventAbi, etc. We
// just need to tell the TypeScript compiler that it's NOT
// FallbackAbi, or else it would complain, "Property
// 'inputs' does not exist on type 'AbiDefinition'.
// Property 'inputs' does not exist on type
// 'FallbackAbi'.", despite the enclosing if statement.
// tslint:disable:no-unnecessary-type-assertion
parameters = parameters.concat((abi as MethodAbi).inputs);
}
if (abi.hasOwnProperty('outputs')) {
// HACK(feuGeneA): same as described above, except here we
// KNOW that it's a MethodAbi, given the enclosing if
// statement, because that's the only AbiDefinition subtype
// that actually has an outputs field.
parameters = parameters.concat((abi as MethodAbi).outputs);
}
for (const parameter of parameters) {
utils.extractTuples(parameter, tupleBodies, tupleDependencies);
}
}
// build up a list of tuples to declare. the order they're pushed into
// this array is the order they will be declared.
const tuplesToDeclare = [];
// first push the ones that have dependencies
tuplesToDeclare.push(...toposort(tupleDependencies));
// then push any remaining bodies (the ones that DON'T have
// dependencies)
for (const pythonTupleName in tupleBodies) {
if (!tuplesToDeclare.includes(pythonTupleName)) {
tuplesToDeclare.push(pythonTupleName);
}
}
// now iterate over those ordered tuples-to-declare, and prefix the
// corresponding class bodies with their class headers, to form full
// class declarations.
const tupleDeclarations = [];
for (const pythonTupleName of tuplesToDeclare) {
if (tupleBodies[pythonTupleName]) {
tupleDeclarations.push(
`class ${pythonTupleName}(TypedDict):\n """Python representation of a tuple or struct.\n\n Solidity compiler output does not include the names of structs that appear\n in method definitions. A tuple found in an ABI may have been written in\n Solidity as a literal, anonymous tuple, or it may have been written as a\n named \`struct\`:code:, but there is no way to tell from the compiler\n output. This class represents a tuple that appeared in a method\n definition. Its name is derived from a hash of that tuple's field names,\n and every method whose ABI refers to a tuple with that same list of field\n names will have a generated wrapper method that refers to this class.\n\n Any members of type \`bytes\`:code: should be encoded as UTF-8, which can be\n accomplished via \`str.encode("utf_8")\`:code:\n """${
tupleBodies[pythonTupleName]
}`,
);
}
}
// finally, join the class declarations together for the output file
return new Handlebars.SafeString(tupleDeclarations.join('\n\n\n'));
});
Handlebars.registerHelper('docBytesIfNecessary', (abisJSON: string) => {
const abis: AbiDefinition[] = JSON.parse(abisJSON);
// see if any ABIs accept params of type bytes, and if so then emit
// explanatory documentation string.
for (const abi of abis) {
if (abi.hasOwnProperty('inputs')) {
// HACK(feuGeneA): using "as MethodAbi" below, but abi
// could just as well be ConstructorAbi, EventAbi, etc. We
// just need to tell the TypeScript compiler that it's NOT
// FallbackAbi, or else it would complain, "Property
// 'inputs' does not exist on type 'AbiDefinition'.
// Property 'inputs' does not exist on type
// 'FallbackAbi'.", despite the enclosing if statement.
// tslint:disable:no-unnecessary-type-assertion
if ((abi as MethodAbi).inputs) {
for (const input of (abi as MethodAbi).inputs) {
if (input.type === 'bytes') {
return new Handlebars.SafeString(
'\n\n All method parameters of type `bytes`:code: should be encoded as UTF-8,\n which can be accomplished via `str.encode("utf_8")`:code:.\n ',
);
}
}
}
}
}
return '';
});
Handlebars.registerHelper(
'toPythonClassname',
(sourceName: string) => new Handlebars.SafeString(changeCase.pascal(sourceName)),
);
}
if (args.language === 'TypeScript') {
registerTypeScriptHelpers();
} else if (args.language === 'Python') {
Expand Down
Loading

0 comments on commit df97b20

Please sign in to comment.