Rest server #839

cschuchardt88 · 2023-11-08T08:17:41Z

RestServer

In this section you will learn about RestServer plugin and how it works.

Dependencies

Microsoft.AspNetCore.JsonPatch.dll Required
Microsoft.AspNetCore.Mvc.NewtonsoftJson.dll Required
Microsoft.AspNetCore.Mvc.Versioning.ApiExplorer.dll Required
Microsoft.AspNetCore.Mvc.Versioning.dll Required
Microsoft.OpenApi.dll Required
Newtonsoft.Json.Bson.dll Required
Newtonsoft.Json.dll Required
System.ServiceProcess.ServiceController.dll linux maybe
Microsoft.AspNetCore.Mvc.Versioning.dll Required
Microsoft.AspNetCore.Mvc.Versioning.dll Required
Microsoft.AspNetCore.Mvc.Versioning.dll Required
Microsoft.OpenApi.dll Swagger (optional)
Swashbuckle.AspNetCore.Swagger.dll Swagger (optional)
Swashbuckle.AspNetCore.SwaggerGen.dll Swagger (optional)
Swashbuckle.AspNetCore.SwaggerUI.dll Swagger (optional)
Swashbuckle.AspNetCore.Newtonsoft.dll Swagger (optional)
RestServer.xml Swagger UI (optional)

In Docker

RestServer
|   |   |-- Microsoft.AspNetCore.JsonPatch.dll
|   |   |-- Microsoft.AspNetCore.Mvc.NewtonsoftJson.dll
|   |   |-- Microsoft.AspNetCore.Mvc.Versioning.ApiExplorer.dll
|   |   |-- Microsoft.AspNetCore.Mvc.Versioning.dll
|   |   |-- Microsoft.OpenApi.dll
|   |   |-- Neo.ConsoleService.dll
|   |   |-- Newtonsoft.Json.Bson.dll
|   |   |-- Newtonsoft.Json.dll
|   |   |-- RestServer.deps.json
|   |   |-- RestServer.dll
|   |   |-- RestServer.pdb
|   |   |-- RestServer.xml
|   |   |-- Swashbuckle.AspNetCore.Newtonsoft.dll
|   |   |-- Swashbuckle.AspNetCore.Swagger.dll
|   |   |-- Swashbuckle.AspNetCore.SwaggerGen.dll
|   |   |-- Swashbuckle.AspNetCore.SwaggerUI.dll
|   |   |-- System.ServiceProcess.ServiceController.dll
|   |   |-- config.json
|   |   `-- runtimes
|   |       `-- win
|   |           `-- lib
|   |               `-- net6.0
|   |                   `-- System.ServiceProcess.ServiceController.dll

These files go in the same directory as the RestServer.dll. In neo-cli
plugins/RestServer/ folder.

Response Headers

Name	Value(s)	Description
server	neo-cli/3.6.0 RestServer/3.6.0	`neo-cli` and `RestServer` version.

JSON Serializer

RestServer uses custom Newtonsoft Json Converters to serialize controller action
responses and route parameters.

One Way Binding - Write only.

Neo.SmartContract.ContractState
Neo.SmartContract.NefFile
Neo.SmartContract.MethodToken
Neo.SmartContract.Native.TrimmedBlock
Neo.SmartContract.Manifest.ContractAbi
Neo.SmartContract.Manifest.ContractGroup
Neo.SmartContract.Manifest.ContractManifest
Neo.SmartContract.Manifest.ContractPermission
Neo.SmartContract.Manifest.ContractPermissionDescriptor
Neo.Network.P2P.Payloads.Block
Neo.Network.P2P.Payloads.Header
Neo.Network.P2P.Payloads.Signer
Neo.Network.P2P.Payloads.TransactionAttribute
Neo.Network.P2P.Payloads.Transaction
Neo.Network.P2P.Payloads.Witness

Two Way Binding - Read & Write

System.Guid
System.ReadOnlyMemory<T>
Neo.BigDecimal
Neo.UInt160
Neo.UInt256
Neo.Cryptography.ECC.ECPoint
Neo.VM.Types.Array
Neo.VM.Types.Boolean
Neo.VM.Types.Buffer
Neo.VM.Types.ByteString
Neo.VM.Types.Integer
Neo.VM.Types.InteropInterface
Neo.VM.Types.Null
Neo.VM.Types.Map
Neo.VM.Types.Pointer
Neo.VM.Types.StackItem
Neo.VM.Types.Struct

Remote Endpoints

Parametes {hash} can be any Neo N3 address or scripthash; {address} can be any Neo N3 address only; {number} and {index} can be any uint32.

Parameter Examples

{hash} - 0xef4073a0f2b305a38ec4050e4d3d28bc40ea63f5 or NiHURyS83nX2mpxtA7xq84cGxVbHojj5Wc
{address} - NiHURyS83nX2mpxtA7xq84cGxVbHojj5Wc
{number} - 1
{index} - 2500000

Paths

Utils
- [GET] /api/v1/utils/{hash}/address
- [GET] /api/v1/utils/{address}/scripthash
- [GET] /api/v1/utils/{hash}/{address}/validate
Node
- [GET] /api/v1/node
- [GET] /api/v1/node/peers
- [GET] /api/v1/node/plugins
- [GET] /api/v1/node/settings
Ledger
- [GET] /api/v1/ledger/neo/accounts
- [GET] /api/v1/ledger/gas/accounts
- [GET] /api/v1/ledger/blocks?page={number}&size={number}
- [GET] /api/v1/ledger/blocks/height
- [GET] /api/v1/ledger/blocks/{index}
- [GET] /api/v1/ledger/blocks/{index}/header
- [GET] /api/v1/ledger/blocks/{index}/witness
- [GET] /api/v1/ledger/blocks/{index}/transactions?page={number}&size={number}
- [GET] /api/v1/ledger/transactions/{hash}
- [GET] /api/v1/ledger/transactions/{hash}/witnesses
- [GET] /api/v1/ledger/transactions/{hash}/signers
- [GET] /api/v1/ledger/transactions/{hash}/attributes
- [GET] /api/v1/ledger/memorypool?page={number}&size={number}
- [GET] /api/v1/ledger/memorypool/verified?page={number}&size={number}
- [GET] /api/v1/ledger/memorypool/unverified?page={number}&size={number}
- [GET] /api/v1/ledger/memorypool/count
Tokens
- [GET] /api/v1/tokens/balanceof/{address}
- NFTs
  - [GET] /api/v1/tokens/nep-11?page={number}&size={number}
  - [GET] /api/v1/tokens/nep-11/count
  - [GET] /api/v1/tokens/nep-11/{hash}/balanceof/{address}
- NEP-17
  - [GET] /api/v1/tokens/nep-17?page={number}&size={number}
  - [GET] /api/v1/tokens/nep-17/count
  - [GET] /api/v1/tokens/nep-17/{hash}/balanceof/{address}
Contracts
- [GET] /api/v1/contracts?page={number}&size={number}
- [GET] /api/v1/contracts/count
- [GET] /api/v1/contracts/{hash}
- [GET] /api/v1/contracts/{hash}/abi
- [GET] /api/v1/contracts/{hash}/manifest
- [GET] /api/v1/contracts/{hash}/nef
- [GET] /api/v1/contracts/{hash}/storage
Wallet
- [POST] /api/v1/wallet/open
- [POST] /api/v1/wallet/create
- [POST] /api/v1/wallet/{session}/address/create
- [GET] /api/v1/wallet/{session}/address/list
- [GET] /api/v1/wallet/{session}/asset/list
- [GET] /api/v1/wallet/{session}/balance/list
- [POST] /api/v1/wallet/{session}/changepassword
- [GET] /api/v1/wallet/{session}/close
- [GET] /api/v1/wallet/{session}/delete/{address}
- [GET] /api/v1/wallet/{session}/export/{address}
- [GET] /api/v1/wallet/{session}/export
- [GET] /api/v1/wallet/{session}/gas/unclaimed
- [GET] /api/v1/wallet/{session}/key/list
- [POST] /api/v1/wallet/{session}/import
- [POST] /api/v1/wallet/{session}/import/multisigaddress
- [POST] /api/v1/wallet/{session}/transfer

…njection.

Added HTTP Basic Auth. Changed most classes to internal that didnt need to be public. Added Rest Server Middleware. Added the ability to block urls paths (for module rest server plugins). Added the ability to enable CORS and whitelist of urls. Added HTTP user and pass configurable. Added hot load of Controllers from plugin assemblies Changed Controllers to be "ApiController" class

…abled with no origins,.

Added MaxTransactionFee to config Added Anything for Kestrel to be configurable

Added Nep11 token controller. Changed namespaces and file names around.

Added peers route for node controller Changed RemoteNodeModel and TokenBalanceModel namespaces

Added Neo/Gas controller Added send raw transactions relay

Fixed formating Added blacklist/disable controllers Removed DisableRoute list from config (wasn't being used)

…rom end user

… format Added InvokeScript to helper class Added ConvertToScriptHash for address to scripthash

Changed all controller parameters for json converters Added custom error handling Fixed wallet sessions Added wallet session manager removed development mode (not needed) Added session timeout to config file Added max allow sessions to config file

Added swagger configurable in config.json Changed controllers to reflect swagger Changed wallet controller to reset expiring for wallets per request

…ules into RestServer

Added new json converter for block, transaction and contract removed the model associated with blocks, transactions, and contracts changed json settings for new json converters that were added

Added Console Commands for wallet sessions

Fixed UInt160, UInt256 json converters value not being set right in certain situations. Fixed wallet transfers for null signers

Fixed Wallet transfers with invalid data or output messages

Changed wallet route paths around to be suit the means Fixed ECPoint json converter for reading Fixed UInt160 json converter Changed error out for invalid serialization of json on route parameters Fixed with dotnet format

Changed restricted wallet path to Environment.CurrentDirectory

Changed NEP-11 Models to IReadOnlyDictionary and IEnumerable.

Fixed with dotnet format

Fixed Guid json converter with values.

src/RestServer/Controllers/v1/WalletController.cs

roman-khimov · 2024-01-10T20:01:35Z

src/RestServer/Controllers/v1/LedgerController.cs

+        [ProducesResponseType(StatusCodes.Status200OK, Type = typeof(AccountDetails[]))]
+        public IActionResult ShowGasAccounts()
+        {
+            var accounts = NativeContract.GAS.ListAccounts(_neosystem.StoreView, _neosystem.Settings);


Consider mainnet and some public server providing this API. Won't it be easy to DOS it by doing these accounts requests?

It would be something to test. I am using the same logic as core; that uses the storage API. But i do see your point.

We can't have these methods unrestricted.

roman-khimov · 2024-01-10T20:05:21Z

src/RestServer/Controllers/v1/LedgerController.cs

+        /// <returns>An array of the Transaction object.</returns>
+        /// <response code="200">Successful</response>
+        /// <response code="400">An error occurred. See Response for details.</response>
+        [HttpGet("memorypool", Name = "GetMemoryPoolTransactions")]


Why memorypool is a part of ledger? I'd say it's a part of node, there is nothing here that is a part of the ledger yet.

Also, ledger can be associated with the respective native contract.

True. Good point. I was thinking more that, node would be settings, peers, connections. However for ledger that would be, more like transactions and blocks; the actual data related the ledger as in; the stuff that keeps track of the accounts and data.

roman-khimov · 2024-01-10T20:06:35Z

src/RestServer/Controllers/v1/LedgerController.cs

+        {
+            if (skip < 0 || take < 0 || take > RestServerSettings.Current.MaxPageSize)
+                throw new InvalidParameterRangeException();
+            return Ok(_neosystem.MemPool.Skip((skip - 1) * take).Take(take));


Memory pool can change from request to request, I'm not sure paging really fits here.

Yes i thought about that too. But if we were as busy as etherium and blocks can have 10,000 transactions in it. I figure paging would be good.

Vote up for paging removal, no practical usage for this option.

roman-khimov · 2024-01-10T20:11:27Z

src/RestServer/Exceptions/RestErrorCodes.cs

+{
+    internal static class RestErrorCodes
+    {
+        //=========================Rest Codes=========================


The API is different, but neo-project/proposals#156 is likely still applicable, just two codes are not really usable.

Ok thx. I was wondering about the codes. Is there a list somewhere?

Right in the proposal. We should have some unification between REST and RPC codes.

cschuchardt88 · 2024-02-12T13:51:18Z

@shargon @Jim8y
This already been tested and reviewed by @superboyiii. However @vncoelho caused a seen, that ended up being false.

shargon · 2024-02-13T14:55:36Z

I will review it likely the next week (working of bug fixes now).

shargon · 2024-02-14T11:40:57Z

src/RestServer/Newtonsoft/Json/BigDecimalJsonConverter.cs

+using Newtonsoft.Json.Linq;
+using System.Numerics;
+
+namespace Neo.Plugins.RestServer.Newtonsoft.Json


Cal we move all of these converters to Neo.Json project?

More organized.

Reusable.

Two PR, easy to review.

I am using NewtonSoft.Json library. However would be nice to have in a library.

Orderd a new machine, will arrive two days later. My current machine is too weak.

BTW, how hard is it for you to build a Restful Mock testing system?

Well you have to mock the asp.net core to access controllers. Which can be a pain. After that it's pretty easy.

Let's move it to Neo.Json, easier to approve by chunks

@shargon

Let's move it to Neo.Json, easier to approve by chunks

Won't be able to do because of circular dependencies.

cschuchardt88 · 2024-03-30T15:48:19Z

Ready for merge #839 (comment)

AnnaShaleva · 2024-04-01T19:06:43Z

docs/RestServer/Addons.md

@@ -0,0 +1,108 @@
+## RestServer Plugin
+In this section of you will learn how to make a `neo-cli` plugin that integrates with `RestServer`
+plugin. Lets take a look at [Example Plugin](/examples/RestServerPlugin).


Do we still have an example? Because currently an example plugin link navigates to nowhere.

I think this is a start https://github.com/cschuchardt88/neo-modules/blob/RestServer/docs/RestServer/Addons.md however looks like some stuff is missing.

AnnaShaleva · 2024-04-01T19:09:36Z

docs/RestServer/Addons.md

+  - **Type** - _Must have a base class of [error](#error-class)._
+
+## Error Class
+Needs to be the same as `RestServer` of else there will be some inconsistencies


s/of else/otherwise

AnnaShaleva · 2024-04-01T19:11:57Z

docs/RestServer/Addons.md

+Properties `Code`, `Name` and `Message` values can be whatever you desire.
+
+**Model**
+```csharp
+public class ErrorModel
+{
+    public int Code { get; set; };
+    public string Name { get; set; };
+    public string Message { get; set; };
+}


The case is they can't. We have neo-project/proposals#156 for RPC errors, and if we're merging this PR, then I think we need some unified set of error codes for REST as far.

AnnaShaleva · 2024-04-01T19:16:36Z

docs/RestServer/Addons.md

+```
+Notice that the _above_ example also returns with HTTP status code of `204 No Content`.
+This action `route` also extends the `contracts` API. Adding method `sayHello`. Routes
+can be what you like as well. But if you want to extend on any existing controller you


s/extend on/extend

AnnaShaleva · 2024-04-01T19:22:51Z

src/RestServer/Binder/UInt160Binder.cs

+
+namespace Neo.Plugins.RestServer.Binder
+{
+    internal class UInt160Binder : IModelBinder


How about the same provider for UInt256?

There is no alternative formatting for UInt256. Unlike UInt160 can be scripthash or 160HASH format.

AnnaShaleva · 2024-04-01T19:36:12Z

src/RestServer/Controllers/v1/LedgerController.cs

+        {
+            if (skip < 0 || take < 0 || take > RestServerSettings.Current.MaxPageSize)
+                throw new InvalidParameterRangeException();
+            return Ok(_neosystem.MemPool.Skip((skip - 1) * take).Take(take));


Vote up for paging removal, no practical usage for this option.

AnnaShaleva · 2024-04-01T19:37:28Z

src/RestServer/Controllers/v1/NodeController.cs

+        [HttpGet("settings", Name = "GetNodeProtocolSettings")]
+        [ProducesResponseType(StatusCodes.Status200OK, Type = typeof(ProtocolSettingsModel))]
+        public IActionResult GetSettings() =>
+            Ok(_neosystem.Settings.ToModel());
+    }


Can be extended to be similar to getversion response.

AnnaShaleva · 2024-04-01T19:38:22Z

src/RestServer/Controllers/v1/TokensController.cs

+            var tokenList = NativeContract.ContractManagement.ListContracts(_neosystem.StoreView);
+            var vaildContracts = tokenList
+                .Where(ContractHelper.IsNep17Supported)
+                .OrderBy(o => o.Manifest.Name)


Ditto, sorting by name is a bad idea.

AnnaShaleva · 2024-04-01T19:41:48Z

src/RestServer/Exceptions/RestErrorCodes.cs

+{
+    internal static class RestErrorCodes
+    {
+        //=========================Rest Codes=========================


Right in the proposal. We should have some unification between REST and RPC codes.

src/RestServer/config.json

Christopher R. Schuchardt and others added 30 commits August 16, 2023 17:55

Add new project for restserver

26da86d

Added Utils, Node and contract controllers

b47eb05

updated license for each file.

f8b28c3

Added plugin support for plugins to add dependencies via dependency i…

a4f0793

…njection.

Update to allow MaxPageSize

ff95dda

Added CORS misconfiguation message to the user if auth and cors is en…

28a64b6

…abled with no origins,.

Added Wallet API (untested)

8ab3ce5

Added MaxTransactionFee to config Added Anything for Kestrel to be configurable

Added Nep17 tokens controller.

fac1217

Added Nep11 token controller. Changed namespaces and file names around.

Added invoke gas max amount

9a04a55

Added peers route for node controller Changed RemoteNodeModel and TokenBalanceModel namespaces

Added blockchain controller

bfbecd6

Added Neo/Gas controller Added send raw transactions relay

Changed controllers to show error if is invalid node protocol network

bb4f4da

Fixed formating Added blacklist/disable controllers Removed DisableRoute list from config (wasn't being used)

Fixed RestServerMiddleware crashing if malformed basic auth is sent f…

89ffcd8

…rom end user

Changed json converter for UInt160 to validate Address and scripthash…

ea8b90d

… format Added InvokeScript to helper class Added ConvertToScriptHash for address to scripthash

Changes Namespaces

28c8bc3

Fixed Wallet Transfers

0a26c15

Error Codes

3e60b80

Added Custom Exceptions

ee36f48

Changed all controller parameters for json converters Added custom error handling Fixed wallet sessions Added wallet session manager removed development mode (not needed) Added session timeout to config file Added max allow sessions to config file

Merge branch 'master' into RestServer

33b010f

Added swagger support

6f8eefc

Added swagger configurable in config.json Changed controllers to reflect swagger Changed wallet controller to reset expiring for wallets per request

Merge branch 'RestServer' of https://github.com/cschuchardt88/neo-mod…

ae89734

…ules into RestServer

Made RestServer more generic and plugin friendly

0382984

Added new json converter for block, transaction and contract removed the model associated with blocks, transactions, and contracts changed json settings for new json converters that were added

Changed now when wallets expire they will be terminated on time.

5a97e6b

Added Console Commands for wallet sessions

Fixed Block, transaction json converters for null values.

21bc035

Fixed UInt160, UInt256 json converters value not being set right in certain situations. Fixed wallet transfers for null signers

Fixed UInt160 and UInt256 with null values

8745fd7

Fixed Wallet transfers with invalid data or output messages

Added full wallet functionality

92e01d3

Changed wallet route paths around to be suit the means Fixed ECPoint json converter for reading Fixed UInt160 json converter Changed error out for invalid serialization of json on route parameters Fixed with dotnet format

Added Transactions with scripts

a2853ed

Changed restricted wallet path to Environment.CurrentDirectory

Fixed typos with json properties in json converters.

c5382ff

Changed NEP-11 Models to IReadOnlyDictionary and IEnumerable.

Changed Nep11 and Nep17 token classes to be more sounds on logic

f5c97ee

Fixed with dotnet format

Added swagger server discriptions for wallet controller.

459e7cb

Fixed Guid json converter with values.

shargon added 3 commits January 9, 2024 21:11

Nullable and some fixes

e915ad0

format

10de333

clean using added by error

87949e7

shargon previously approved these changes Jan 9, 2024

View reviewed changes

github-advanced-security bot found potential problems Jan 9, 2024

View reviewed changes

src/RestServer/Controllers/v1/WalletController.cs Fixed Show resolved Hide resolved

shargon dismissed their stale review via 16c669e January 9, 2024 20:21

fix LFI

16c669e

github-advanced-security bot found potential problems Jan 9, 2024

View reviewed changes

src/RestServer/Controllers/v1/WalletController.cs Fixed Show resolved Hide resolved

Merge branch 'master' into RestServer

35ff01b

roman-khimov reviewed Jan 10, 2024

View reviewed changes

cschuchardt88 added 4 commits January 10, 2024 15:49

Removed wallets

3971f5f

Code Format

a8d378a

Remove more wallet stuff

2beb17e

Merge branch 'master' into RestServer

aded6ed

Jim8y added the waiting for review label Feb 12, 2024

shargon reviewed Feb 14, 2024

View reviewed changes

Merge branch 'master' into RestServer

26d0b17

Jim8y added Need Update This pr needs to be updated. and removed waiting for review labels Feb 15, 2024

cschuchardt88 added 2 commits March 19, 2024 04:02

Merge branch 'master' into RestServer

724b10b

Merge branch 'master' into RestServer

6cbea40

cschuchardt88 requested review from shargon, Jim8y, superboyiii and vncoelho March 30, 2024 15:48

AnnaShaleva reviewed Apr 1, 2024

View reviewed changes

Merge branch 'master' into RestServer

0c96908

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rest server #839

Rest server #839

cschuchardt88 commented Nov 8, 2023 •

edited

roman-khimov Jan 10, 2024

cschuchardt88 Jan 10, 2024

AnnaShaleva Apr 1, 2024

roman-khimov Jan 10, 2024

cschuchardt88 Jan 10, 2024

roman-khimov Jan 10, 2024

cschuchardt88 Jan 10, 2024

AnnaShaleva Apr 1, 2024

roman-khimov Jan 10, 2024

cschuchardt88 Jan 10, 2024

AnnaShaleva Apr 1, 2024

cschuchardt88 commented Feb 12, 2024

shargon commented Feb 13, 2024

shargon Feb 14, 2024

cschuchardt88 Feb 14, 2024 •

edited

Jim8y Feb 14, 2024

cschuchardt88 Feb 14, 2024 •

edited

shargon Feb 14, 2024

cschuchardt88 Mar 19, 2024 •

edited

cschuchardt88 commented Mar 30, 2024

AnnaShaleva Apr 1, 2024

cschuchardt88 Apr 11, 2024

AnnaShaleva Apr 1, 2024

AnnaShaleva Apr 1, 2024

AnnaShaleva Apr 1, 2024

AnnaShaleva Apr 1, 2024

cschuchardt88 Apr 11, 2024

AnnaShaleva Apr 1, 2024

AnnaShaleva Apr 1, 2024

AnnaShaleva Apr 1, 2024

AnnaShaleva Apr 1, 2024

Rest server #839

Are you sure you want to change the base?

Rest server #839

Conversation

cschuchardt88 commented Nov 8, 2023 • edited

RestServer

Dependencies

In Docker

Response Headers

JSON Serializer

Remote Endpoints

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cschuchardt88 commented Feb 12, 2024

shargon commented Feb 13, 2024

Choose a reason for hiding this comment

cschuchardt88 Feb 14, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cschuchardt88 Feb 14, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cschuchardt88 Mar 19, 2024 • edited

Choose a reason for hiding this comment

cschuchardt88 commented Mar 30, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cschuchardt88 commented Nov 8, 2023 •

edited

cschuchardt88 Feb 14, 2024 •

edited

cschuchardt88 Feb 14, 2024 •

edited

cschuchardt88 Mar 19, 2024 •

edited