Tonutils-JS

tonutils is a comprehensive toolkit for working with the TON (The Open Network) protocols. This package provides an easy-to-use, TypeScript-based interface for building and interacting with applications in the TON ecosystem.

Overview

The TON has its own unique peer-to-peer network protocols, which are utilized for a range of functions, including the propagation of new blocks, the sending and collection of transaction candidates, and more. While the networking demands of single-blockchain projects like Bitcoin or Ethereum can be met with a simple peer-to-peer overlay network, the multi-blockchain nature of TON requires a more sophisticated approach.

In TON, it's possible to subscribe to updates from specific shardchains, rather than having to subscribe to all of them. This offers a level of flexibility and potential that is unique to the TON ecosystem.

Additionally, these protocols are used to power a range of TON Ecosystem services, such as TON Proxy, TON Sites, and TON Storage. Once the more advanced network protocols required to support TON Blockchain are implemented, they can be used for a variety of purposes that go beyond the immediate demands of the blockchain itself.

tonutils is a set of tools that make working with these protocols easy and intuitive. It provides a way to interact with the TON network using a TypeScript interface, bringing the power and flexibility of TON's unique network protocols to developers in a user-friendly package.

For more information about TON Networking, please visit the official TON Networking documentation. For a deeper understanding of ADNL, you can refer to the official ADNL documentation and the advanced ADNL documentation.

Installation

Each package can be installed individually via npm:

npm install tonutils

Packages

• tonutils/adnl: This package provides the implementation of the Abstract Datagram Network Layer (ADNL), used for peer-to-peer communication in the TON network.

• tonutils/config: A Node.js library for loading and parsing the global configuration file from the TON network (https://ton.org/global-config.json). It provides a typed interface for easy access to the configuration properties, simplifying the development of decentralized applications by offering a convenient way to access TON network settings.

• tonutils/dht: This package facilitates interaction with the Distributed Hash Table (DHT) used by the TON network.

• (not implemented yet) tonutils/fec: The Forward Error Correction (FEC) package provides mechanisms for error detection and correction in the data transmission process.

• (not implemented yet) tonutils/http: A utility package for HTTP-based communication with the TON network.

• tonutils/keyring: This package provides functionalities for managing and manipulating keys in the TON network.

• tonutils/keys: A utility package providing mechanisms for key generation and handling in the TON network.

• (not implemented yet) tonutils/overlay: This package provides functionalities related to the TON's Overlay Network.

• (not implemented yet) tonutils/raptorq: Implements the RaptorQ error correction code, which is used to ensure reliable data transmission.

• (not implemented yet) tonutils/rldp: The Reliable Linked Data Protocol (RLDP) package implements the data transfer protocol used in the TON network.

• (not implemented yet) tonutils/rldp2: An advanced and efficient RLDP2 client for Node.js, designed to provide high-speed data transfer with improved reliability in peer-to-peer networks.

• (not implemented yet) tonutils/storage: This package provides functionalities for managing TON Storage.

• tonutils/tl: A utility package for working with the Type Language (TL) scheme used by the TON network.

• (not implemented yet) tonutils/tlb: A utility package for working with the TL-B scheme used by the TON blockchain.

Examples

This repository includes a couple of examples to help you get started:

• ADNL Chat Example: This example demonstrates how to set up a basic chat using the ADNL layer.

• Lookup DHT Nodes Example: This example shows how to lookup DHT nodes.

Feel free to explore these examples to get a better understanding of how to use tonutils.

Usage

Refer to the documentation of each package for more specific usage details.

ADNL (Abstract Datagram Network Layer)

The tonutils/adnl package provides the implementation of the Abstract Datagram Network Layer (ADNL), used for peer-to-peer communication in the TON network.

Below is an example of connecting to the DHT using the adnl package:

import {loadConfig} from 'tonutils/config';
import {
  AdnlNode,
  AdnlNodeIdFull,
  AdnlNodeIdShort,
  NewPeerContext,
  NodeOptions,
  QueryConsumingResult,
  QueryConsumingResultType,
  SocketAddrV4,
  SubscriberContext
} from "tonutils/adnl";
import {KeyringDBNoop, KeyringImpl} from "tonutils/keyring";
import {PrivateKey, PublicKey} from "tonutils/keys";

// We use the tonutils/tl package for working with the TL scheme, because the `ton-tl` package is not yet compatible with the `ton-api.tl` scheme.
import {TLWriteBuffer, TLReadBuffer, Codecs, dht_Node, Functions} from "tonutils/tl";

// Node options
const options = NodeOptions.default();

// Keyring for managing keys
const keyring = new KeyringImpl(new KeyringDBNoop());

// Generate a private key
const privateKey = new PrivateKey(PrivateKey.Ed25519.random().tl());

// Compute a short ID for the local node
const localId = new AdnlNodeIdShort(privateKey.computeShortId());

// Add the private key to the keyring
await keyring.addKey(privateKey);

// Create a socket address
const addr = new SocketAddrV4('udp', '0.0.0.0', 0);

// Initialize an ADNL node
const adnlNode = new AdnlNode(addr, keyring, options);

// Load global TON configuration
const TON_CONFIG = 'https://ton.org/global-config.json';
const config = await loadConfig(TON_CONFIG);

// Check if the configuration contains DHT nodes
if (!config?.dht || !config.dht.staticNodes || !config.dht.staticNodes.nodes) {
  throw new Error(`No static nodes found in ${TON_CONFIG}`);
}

// Convert the DHT nodes to a more convenient format
const dhtNodes = config.dht.staticNodes.nodes.map((node: dht_Node) => {
  const addr = SocketAddrV4.createFromAddressList(node.addrList);
  const peerFullId = new AdnlNodeIdFull(new PublicKey(node.id));
  const peerId = peerFullId.computeShortId();

  return {addr, peerId, peerFullId};
});

// Add the DHT nodes to the ADNL node
for (const dhtNode of dhtNodes) {
  adnlNode.addPeer(NewPeerContext.Dht, localId, dhtNode.peerId, dhtNode.addr, dhtNode.peerFullId);
}

// Start the ADNL node
await adnlNode.start();

// Query the DHT nodes
const results = await Promise.all(dhtNodes.map(async (dhtNode) => {
  try {
    // Create a request to get the list of signed addresses
    const request = new TLWriteBuffer();
    Functions.dht_getSignedAddressList.encodeRequest({
      kind: 'dht.getSignedAddressList',
    }, request);

    // Send the request to the DHT node and wait for the response
    const response = await adnlNode.query(localId, dhtNode.peerId, request.build());
    if (!response) {
      throw new Error(`No response from ${dhtNode.addr.toString()}`);
    }

    // Decode the response as a list of signed addresses
    const answer = Functions.dht_getSignedAddressList.decodeResponse(new TLReadBuffer(response));

    return { answer };
  } catch (e) {
    return null;
  }
}));

// Print the results
console.log(`Connected to ${results.filter(p => p).length} DHT nodes`);

See the full documentation for the adnl package here.

tonutils/dht

This package provides the implementation of the Distributed Hash Table (DHT) used by the TON network.

Usage

1. Install the package.

npm install tonutils

2. Use the package in your code.

import {AdnlNode, AdnlNodeIdShort, AdnlNodeOptions, SocketAddrV4} from 'tonutils/adnl';
import {DhtNode, DhtNodeOptions} from 'tonutils/dht';
import {KeyringDBNoop, KeyringImpl} from 'tonutils/keyring';
import {PrivateKey} from 'tonutils/keys';
import {loadConfig} from "tonutils/config";
import {Peer} from "@tonutils/adnl";

// initialize ADNL node
const adnlOptions = new AdnlNodeOptions({
  queryMinTimeoutMs: 1000,
  queryDefaultTimeoutMs: 1000,
  transferTimeoutSec: 1,
});
const keyring = new KeyringImpl(new KeyringDBNoop());
const privateKey = new PrivateKey(PrivateKey.Ed25519.random().tl());
const localId = new AdnlNodeIdShort(privateKey.computeShortId());
await keyring.addKey(privateKey);
// replace '0.0.0.0' with your external IP address, if you want to receive packets from other nodes in the network
const addr = new SocketAddrV4('udp', '0.0.0.0', 0);
const adnlNode = new AdnlNode(addr, keyring, adnlOptions);

// start ADNL node
await adnlNode.start();

// initialize DHT node
const dhtOptions = new DhtNodeOptions({
  queryTimeoutMs: 1000,
});
const dhtNode = await DhtNode.create(adnlNode, localId, dhtOptions);

// loading ton-global.config.json
const TON_CONFIG = 'https://ton.org/global-config.json';
const config = await loadConfig(TON_CONFIG);

if (!config?.dht || !config.dht.staticNodes || !config.dht.staticNodes.nodes) {
  throw new Error(`No static nodes found in ${TON_CONFIG}`);
}

const staticNodes = config.dht.staticNodes.nodes;

for (const node of staticNodes) {
  const peerId = await dhtNode.addDhtPeer(node);


  if (!peerId) {
    const addr = SocketAddrV4.createFromAddressList(node.addrList);
    console.warn(`Failed to add peer ${addr.ip}:${addr.port}`);
    continue;
  }

  const addr = SocketAddrV4.createFromAddressList(node.addrList);
  console.info(`Added peer ${addr.ip}:${addr.port} with id ${peerId.serialize().toString('hex')}`);
}

await dhtNode.findMoreDhtNodes();

const knownNodes = dhtNode.state.knownPeers.size;
console.log(`Known ${knownNodes} nodes`);

// ping all known nodes
const tasks = [];
for (const peer of dhtNode.state.knownPeers.values()) {
  tasks.push(dhtNode.ping(peer.id).then((result): [boolean, Peer] => [result, peer]));
}

for (const [result, peer] of await Promise.all(tasks)) {
  if (result) {
    console.info(`Ping to ${peer.addr.ip}:${peer.addr.port} success`);
  } else {
    console.warn(`Ping to ${peer.addr.ip}:${peer.addr.port} failed`);
  }
}

console.log(`Shutdown DHT node`);
await dhtNode.shutdown();

console.log(`Shutdown ADNL node`);
await adnlNode.shutdown();

console.log(`Showcase finished`);

See the full documentation for the adnl package here.

tonutils/config

This package provides functions for loading and parsing the global configuration file from the TON network. It provides a typed interface for easy access to the configuration properties.

Usage

1. Install the package.

npm install tonutils

2. Use the package in your code.

import { loadConfig } from "tonutils/config";

async function main() {
  const TON_CONFIG = "https://ton.org/global-config.json";
  const config = await loadConfig(TON_CONFIG);

  if (!config) {
    throw new Error(`Failed to load configuration from ${TON_CONFIG}`);
  }

  console.log(config);
}

main();

See the full documentation for the config package here.

tonutils/tl

This package provides utilities for working with the Type Language (TL) scheme used by the TON network.

Usage

1. Install the package.

npm install tonutils

2. Use the package in your code.

import { TLWriteBuffer, TLReadBuffer, Functions } from "tonutils/tl";

function main() {
  const buffer = new TLWriteBuffer();
  Functions.dht_getSignedAddressList.encodeRequest({
    kind: "dht.getSignedAddressList",
  }, buffer);

  const data = buffer.build();
  const readBuffer = new TLReadBuffer(data);

  const decoded = Functions.dht_getSignedAddressList.decodeResponse(readBuffer);
  console.log(decoded);
}

main();

See the full documentation for the tl package here.

tonutils/keys

This package provides utilities for key generation and handling in the TON network.

Usage

1. Install the package.

npm install tonutils

2. Use the package in your code.

import { PrivateKey } from "tonutils/keys";

function main() {
  const privateKey = PrivateKey.Ed25519.random();
  console.log(privateKey.raw());
}

main();

See the full documentation for the keys package here.

tonutils/keyring

This package provides functionalities for managing and manipulating keys in the TON network.

Usage

1. Install the package.

npm install tonutils

2. Use the package in your code.

import { KeyringImpl, KeyringDBNoop } from "tonutils/keyring";
import { PrivateKey } from "tonutils/adnl";

async function main() {
  const keyring = new KeyringImpl(new KeyringDBNoop());
  const privateKey = new PrivateKey(PrivateKey.Ed25519.random().tl());

  await keyring.addKey(privateKey);
  console.log(`Key added to the keyring: ${privateKey.raw()}`);
}

main();

See the full documentation for the keyring package here.

Other packages

The following packages are still under development and their usage guides will be updated once they are ready:

• tonutils/http
• tonutils/overlay
• tonutils/storage
• tonutils/tlb

Please refer to the individual package directories for more details on their usage and APIs.

Contribution

Contributions are always welcome! Please feel free to open issues and create pull requests.

License

Tonutils is licensed under the MIT License.