diff --git a/docs/en/controllers.md b/docs/en/controllers.md new file mode 100644 index 00000000..46b12c8a --- /dev/null +++ b/docs/en/controllers.md @@ -0,0 +1,94 @@ +# Controllers + +## Launching a Validator in Controller Mode + +1. Prepare the hardware for the validator - 32 virtual cores, 64GB of memory, 1TB SSD, fixed IP address, and 1Gb/s internet speed. + + To maintain network stability, it is recommended to place validators in different locations around the world, rather than concentrating them in a single data center. You can use [this site](https://status.toncenter.com/) to determine the load on various locations. According to the map, there is a high load on data centers in Europe, especially in Finland, Germany, and Paris. Therefore, using providers such as Hetzner and OVH is not recommended. + + > Ensure your hardware meets or exceeds the specified configuration. Running the validator on inappropriate hardware can harm the network and result in penalties. + > Since May 2021, Hetzner has prohibited mining on its servers. This rule currently applies to both PoW and PoS algorithms. Even installing a regular node will be considered a breach of contract. + +2. Install and synchronize **mytonctrl** according to the description in [this instruction](https://github.com/ton-blockchain/mytonctrl/blob/master/docs/en/manual-ubuntu.md) — follow **only** paragraphs 1, 2, and 3. + + You can also refer to this [Video Tutorial](https://docs.ton.org/participate/run-nodes/full-node#installation) for additional help. + +3. Transfer 1 TON to the validator wallet address, which is displayed in the `wl` list. + +4. Use the `aw` command to activate the validator's wallet. + +5. Transfer enough TON to the validator wallet address. + +6. Enable the ton-http-api service: + ``` + mytonctrl -> installer -> enable THA + ``` + Exit installer mode with `Ctrl+D` + +7. Set the liquid pool address, which will lend TON for validation: + ``` + set liquid_pool_addr + ``` + +8. Set the lending parameters that acceptable to you: + ``` + set min_loan 41000 + set max_loan 43000 + set max_interest_percent 1.5 + ``` + + where +* `41000` is the minimum loan amount we are willing to receive from the liquid pool, +* `43000` is the maximum loan amount we are willing to receive from the liquid pool, +* `1.5` 1.5 is the maximum interest rate value for the liquid pool per validation cycle, which we have agreed upon. + +9. Display the annual percentage of profit from the liquid pool: + ``` + calculate_annual_controller_percentage + ``` + +10. Create two controllers with a single command: + + ``` + new_controllers + ``` + +11. Enter `controllers_list` to display the controller addresses: + + ``` + controllers_list + Address Status Balance + kf89KYOeRPRRDyjt_3bPsz92cKSghRqw64efNr8mT1eeHDaS active 0.704345 + kf_lT8QNykLh5PN5ictNX22maiyIf9iK787fXg6nJ_zB-jbN active 0.720866 + ``` + +12. Make a validator deposit in each controller: + + +``` +deposit_to_controller kf89KYOeRPRRDyjt_3bPsz92cKSghRqw64efNr8mT1eeHDaS 10000 +deposit_to_controller kf_lT8QNykLh5PN5ictNX22maiyIf9iK787fXg6nJ_zB-jbN 10000 +``` + + +where `10000` TON is the deposit amount. + +13. Get approval for the controllers. Each pool may have its own approval issuance policy, check with the operator. + +14. Set controller mode: + + ```bash + set useController true + set stake null + ``` + +> (!) If you were previously using nominator pools, do not forget to disable them using the `set usePool false` command. + + +## Switching a Regular Validator to Controller Operating Mode + +1. Enter `set stake 0` to stop participating in elections. + +2. Wait until both of your deposits have been returned from the Elector. + +3. Follow the instructions under "Launching a Validator in Controller Mode", beginning with **Step 6**. \ No newline at end of file diff --git a/docs/ru/controllers.md b/docs/ru/controllers.md new file mode 100644 index 00000000..d3944549 --- /dev/null +++ b/docs/ru/controllers.md @@ -0,0 +1,93 @@ +# Контроллеры + +## Запуск валидатора в режиме работы контроллеров + +1. Подготовьте аппаратное обеспечение для валидатора - 32 виртуальных ядер, 64GB памяти, 1TB SSD, фиксированный IP-адрес, скорость интернета 1Gb/s. + + Для поддержания стабильности сети рекомендуется разместить валидаторы в различных местах по всему миру, а не концентрировать их в одном дата-центре. Вы можете использовать [этот сайт](https://status.toncenter.com/) для определения нагрузки на различные места. Согласно карте, высокая нагрузка на дата-центры в Европе, особенно в Финляндии, Германии и Париже. Поэтому использование провайдеров, таких как Hetzner и OVH, не рекомендуется. + + > Ваше оборудование должно соответствовать указанной конфигурации или быть выше. Запуск валидатора на слабом оборудовании негативно влияет на сеть и может привести к штрафам. + + > С мая 2021 года Hetzner запретил майнинг на своих серверах, в настоящее время под это правило попадают алгоритмы PoW и PoS. Установка даже обычного узла будет считаться нарушением условий договора. + +2. Установите и синхронизируйте **mytonctrl** в соответствии с описанием в [этой инструкции](https://github.com/ton-blockchain/mytonctrl/blob/master/docs/ru/manual-ubuntu.md) — следуйте **только** пунктам 1, 2 и 3. + + Вы также можете обратиться к этой [Видеоинструкции](https://docs.ton.org/participate/run-nodes/full-node#installation) для дополнительной помощи. + +3. Переведите 1 TON на адрес кошелька валидатора, который отображается в списке `wl`. + +4. Используйте команду `aw` для активации кошелька валидатора. + +5. Переведите достаточно TON на адрес кошелька валидатора. + +6. Включите ton-http-api службу: + ``` + mytonctrl -> installer -> enable THA + ``` + Выйдите из режима установщика сочетанием клавиш `Ctrl+D` + +7. Задайте адрес ликвидного пула, который будет одалживать TON для валидации: + ``` + set liquid_pool_addr + ``` + +8. Задайте параметры кредитования, которые вам подходят: + ``` + set min_loan 41000 + set max_loan 43000 + set max_interest_percent 1.5 + ``` + + где +* `41000` - это минимальная сумма кредита который мы готовы получить у ликвидного пула, +* `43000` - это максимальная сумма кредита который мы готовы получить у ликвидного пула, +* `1.5` - это максимальная процентная ставка ликвидного пула за один цикл валидации на которую мы согласны. + +9. Отобразите годовой процент прибыли от ликвидного пула: + ``` + calculate_annual_controller_percentage + ``` + +10. Создайте два контроллера одной командой: + + ``` + new_controllers + ``` + +11. Введите `controllers_list` чтобы отобразить адреса контроллеров: + + ``` + controllers_list + Address Status Balance + kf89KYOeRPRRDyjt_3bPsz92cKSghRqw64efNr8mT1eeHDaS active 0.704345 + kf_lT8QNykLh5PN5ictNX22maiyIf9iK787fXg6nJ_zB-jbN active 0.720866 + ``` + +12. Совершите депозит валидатора в каждый контроллер: + + ``` + deposit_to_controller kf89KYOeRPRRDyjt_3bPsz92cKSghRqw64efNr8mT1eeHDaS 10000 + deposit_to_controller kf_lT8QNykLh5PN5ictNX22maiyIf9iK787fXg6nJ_zB-jbN 10000 + ``` + + где `10000` TON - это сумма депозита. + +13. Получите аппрувал контроллеров. У каждого пула может быть своя политика выдачи аппруволов, уточняйте у оператора. + +14. Активируйте режим работы контроллеров: + + ```bash + set useController true + set stake null + ``` + + > (!) Если до этого вы использовали номинатор-пулы, не забудьте отключить их использование командой `set usePool false`. + + +## Переключение обычного валидатора в режим работы контроллеров + +1. Введите `set stake 0`, чтобы отключить участие в выборах. + +2. Дождитесь, когда оба ваших депозита вернутся от электора. + +3. Следуйте инструкциям "Запуск валидатора в режиме работы контроллеров", начиная с **6-го шага**. diff --git a/functions/session_stats.py b/functions/session_stats.py new file mode 100644 index 00000000..ba08b06e --- /dev/null +++ b/functions/session_stats.py @@ -0,0 +1,136 @@ +#!/usr/bin/env python3 +# -*- coding: utf_8 -*- + +import os +import sys +import time +import json +sys.path.append("/usr/src/mytonctrl/") +from mypylib.mypylib import Dict + +def read_session_stats(need_time_period): + #print(f"read_session_stats: {need_time_period}") + file_path = "/var/ton-work/log.session-stats" + average_block_time = calculate_average_block_time(file_path) + need_blocks = int(need_time_period/average_block_time) + + lines = read_last_lines(file_path, need_lines=need_blocks) + #lines = read_all_lines(file_path) + data = lines2data(lines) + + result = Dict() + result.my_blocks = 0 + result.my_need_blocks = 0 + result.all_master_blocks = 0 + result.all_blocks = len(data) + for buff in data: + if buff.id.workchain == -1: + result.all_master_blocks += 1 + first_producer = buff.rounds[0].producers[0] + if buff.self != first_producer.id: + continue + result.my_need_blocks += 1 + if buff.self == buff.creator: + result.my_blocks += 1 + #end for + return result +#end define + +def calculate_average_block_time(file_path): + blocks = 100 + lines = read_last_lines(file_path, need_lines=blocks) + data = lines2data(lines) + first_block, last_block = get_first_last_block(data) + + diff = int(last_block.timestamp) - int(first_block.timestamp) + average_block_time = round(diff/blocks, 2) + #print("average_block_time:", average_block_time) + return average_block_time +#end define + +def get_first_last_block(data, last_index=-1): + first_block = data[0] + last_block = data[last_index] + if first_block.id.workchain == last_block.id.workchain: + blocks = int(last_block.id.seqno) - int(first_block.id.seqno) + else: + first_block, last_block = get_first_last_block(data, last_index=last_index-1) + return first_block, last_block +#end define + +def lines2data(lines): + #start = time.time() + data = list() + for line in lines: + try: + buff = json.loads(line) + data.append(Dict(buff)) + except json.decoder.JSONDecodeError: + pass + #end for + + #end = time.time() + #diff = round(end - start, 2) + #print(f"lines2data completed in {diff} seconds") + return data +#end define + +def read_all_lines(file_path): + #start = time.time() + with open(file_path, 'rt') as file: + text = file.read() + #end with + lines = text.split('\n') + + #end = time.time() + #diff = round(end - start, 2) + #print(f"read_all_lines completed in {diff} seconds") + return lines +#end define + +def read_last_lines(file_path, need_lines): + lines = list() + max_lines = 30000 + if need_lines < 1: + return lines + elif need_lines > max_lines: + need_lines = max_lines + #print(f"read_last_lines: {need_lines}") + #start = time.time() + with open(file_path, 'rb') as file: + find_last_lines(file, need_lines) + for i in range(need_lines): + line = file.readline().decode() + lines.append(line) + #end with + + + #end = time.time() + #diff = round(end - start, 2) + #print(f"read_last_lines {len(lines)} completed in {diff} seconds") + return lines +#end define + +def find_last_lines(file, need_lines): + # catch OSError in case of a one line file + try: + find_lines = 0 + #file.seek(-2, os.SEEK_END) + #while find_lines != need_lines: + # if file.read(1) == b'\n': + # find_lines += 1 + # file.seek(-2, os.SEEK_CUR) + + file.seek(-100, os.SEEK_END) + while find_lines != need_lines: + if b'\n' in file.read(100): + find_lines += 1 + file.seek(-200, os.SEEK_CUR) + except OSError: + file.seek(0) +#end define + + + + + diff --git a/mytoncore/mytoncore.py b/mytoncore/mytoncore.py index 6f90be3f..8b79278a 100644 --- a/mytoncore/mytoncore.py +++ b/mytoncore/mytoncore.py @@ -534,6 +534,14 @@ def GetVersionFromCodeHash(self, inputHash): arr["pool"] = "399838da9489139680e90fd237382e96ba771fdf6ea27eb7d513965b355038b4" arr["spool"] = "fc2ae44bcaedfa357d0091769aabbac824e1c28f14cc180c0b52a57d83d29054" arr["spool_r2"] = "42bea8fea43bf803c652411976eb2981b9bdb10da84eb788a63ea7a01f2a044d" + arr["liquid_pool_r1"] = "82bc5760719c34395f80df76c42dc5d287f08f6562c643601ebed6944302dcc2" + arr["liquid_pool_r2"] = "95abec0a66ac63b0fbcf28466eb8240ddcd88f97300691511d9c9975d5521e4a" + arr["liquid_pool_r3"] = "22a023bc75b649ff2b5b183cd0d34cd413e6e27ee6d6ad0787f75ad39787ed4e" + arr["liquid_pool_r4"] = "77282b45fd7cfc72ca68fe97af33ad10078730ceaf55e20534c9526c48d602d2" + arr["controller_r1"] = "0949cf92963dd27bb1e6bf76487807f20409131b6110acbc18b7fbb90280ccf0" + arr["controller_r2"] = "01118b9553151fb9bc81704a4b3e0fc7b899871a527d44435a51574806863e2c" + arr["controller_r3"] = "e4d8ce8ff7b4b60c76b135eb8702ce3c86dc133fcee7d19c7aa18f71d9d91438" + arr["controller_r4"] = "dec125a4850c4ba24668d84252b04c6ad40abf5c9d413a429b56bfff09ea25d4" for version, hash in arr.items(): if hash == inputHash: return version @@ -1284,8 +1292,10 @@ def ProcessRecoverStake(self): def GetStake(self, account, args=None): stake = self.local.db.get("stake") usePool = self.local.db.get("usePool") + useController = self.local.db.get("useController") stakePercent = self.local.db.get("stakePercent", 99) vconfig = self.GetValidatorConfig() + validators = vconfig.get("validators") config17 = self.GetConfig17() # Check if optional arguments have been passed to us @@ -1312,6 +1322,8 @@ def GetStake(self, account, args=None): if stake is None and usePool and not is_single_nominator: stake = account.balance - 20 + if stake is None and useController: + stake = account.balance - 50 if stake is None: sp = stakePercent / 100 if sp > 1 or sp < 0: @@ -1397,17 +1409,13 @@ def GetValidatorWallet(self, mode="stake"): def ElectionEntry(self, args=None): usePool = self.local.db.get("usePool") + useController = self.local.db.get("useController") wallet = self.GetValidatorWallet() addrB64 = wallet.addrB64 if wallet is None: raise Exception("Validator wallet not found") #end if - if usePool: - pool = self.GetPool(mode="stake") - addrB64 = pool.addrB64 - #end if - self.local.add_log("start ElectionEntry function", "debug") # Check if validator is not synchronized validatorStatus = self.GetValidatorStatus() @@ -1456,6 +1464,16 @@ def ElectionEntry(self, args=None): return #end if + if usePool: + pool = self.GetPool(mode="stake") + addrB64 = pool.addrB64 + elif useController: + controllerAddr = self.GetController(mode="stake") + self.CheckController(controllerAddr) + self.CreateLoanRequest(controllerAddr) + addrB64 = controllerAddr + #end if + # Calculate stake account = self.GetAccount(addrB64) stake = self.GetStake(account, args) @@ -1483,6 +1501,14 @@ def ElectionEntry(self, args=None): # Send boc file to TON resultFilePath = self.SignBocWithWallet(wallet, resultFilePath, pool.addrB64, 1.3) self.SendFile(resultFilePath, wallet) + elif useController: + var1 = self.CreateElectionRequest(controllerAddr, startWorkTime, adnl_addr, maxFactor) + validatorSignature = self.GetValidatorSignature(validatorKey, var1) + validatorPubkey, resultFilePath = self.SignElectionRequestWithController(controllerAddr, startWorkTime, adnl_addr, validatorPubkey_b64, validatorSignature, maxFactor, stake) + + # Send boc file to TON + resultFilePath = self.SignBocWithWallet(wallet, resultFilePath, controllerAddr, 1.03) + self.SendFile(resultFilePath, wallet) else: var1 = self.CreateElectionRequest(wallet, startWorkTime, adnl_addr, maxFactor) validatorSignature = self.GetValidatorSignature(validatorKey, var1) @@ -1579,7 +1605,7 @@ def PoolUpdateValidatorSet(self, poolAddr, wallet): poolData = self.GetPoolData(poolAddr) if poolData is None: return - #en if + #end if timeNow = int(time.time()) config34 = self.GetConfig34() @@ -3034,6 +3060,8 @@ def GetDestinationAddr(self, destination): #end define def AddrFull2AddrB64(self, addrFull, bounceable=True): + if addrFull is None or "None" in addrFull: + return testnet = self.IsTestnet() buff = addrFull.split(':') workchain = int(buff[0]) @@ -3664,6 +3692,400 @@ def GetPoolData(self, addrB64): return poolData #end define + def GetLiquidPoolAddr(self): + liquid_pool_addr = self.local.db.get("liquid_pool_addr") + if liquid_pool_addr is None: + raise Exception("GetLiquidPoolAddr error: liquid_pool_addr not set") + return liquid_pool_addr + #end define + + def CreateControllers(self): + new_controllers = self.GetControllers() + old_controllers = self.local.db.get("using_controllers", list()) + if new_controllers == old_controllers: + return + #end if + + self.local.add_log("start CreateControllers function", "debug") + wallet = self.GetValidatorWallet() + liquid_pool_addr = self.GetLiquidPoolAddr() + contractPath = self.contractsDir + "jetton_pool/" + if not os.path.isdir(contractPath): + self.DownloadContract("https://github.com/igroman787/jetton_pool") + #end if + + fileName0 = contractPath + "fift-scripts/deploy_controller0.boc" + fileName1 = contractPath + "fift-scripts/deploy_controller1.boc" + resultFilePath0 = self.SignBocWithWallet(wallet, fileName0, liquid_pool_addr, 1) + self.SendFile(resultFilePath0, wallet) + time.sleep(10) + resultFilePath1 = self.SignBocWithWallet(wallet, fileName1, liquid_pool_addr, 1) + self.SendFile(resultFilePath1, wallet) + + # Сохранить новые контроллеры + self.local.db["old_controllers"] = old_controllers + self.local.db["using_controllers"] = new_controllers + self.local.save() + #end define + + def GetControllerAddress(self, controller_id): + wallet = self.GetValidatorWallet() + addr_hash = HexAddr2Dec(wallet.addr) + liquid_pool_addr = self.GetLiquidPoolAddr() + cmd = f"runmethodfull {liquid_pool_addr} get_controller_address_legacy {controller_id} {wallet.workchain} {addr_hash}" + result = self.liteClient.Run(cmd) + buff = self.Result2List(result) + wc = buff[0] + addr_hash = Dec2HexAddr(buff[1]) + addrFull = f"{wc}:{addr_hash}" + controllerAddr = self.AddrFull2AddrB64(addrFull) + return controllerAddr + #end define + + def CheckController(self, controllerAddr): + self.local.add_log("start CheckController function", "debug") + controllerData = self.GetControllerData(controllerAddr) + using_controllers = self.local.db.get("using_controllers", list()) + if controllerData is None: + raise Exception(f"CheckController error: controller not initialized. Use new_controllers") + if controllerData["approved"] != -1: + raise Exception(f"CheckController error: controller not approved: {controllerAddr}") + if controllerAddr not in using_controllers: + raise Exception("CheckController error: controller is not up to date. Use new_controllers") + #end define + + def GetControllers(self): + self.local.add_log("start GetControllers function", "debug") + controller0 = self.GetControllerAddress(controller_id=0) + controller1 = self.GetControllerAddress(controller_id=1) + controllers = [controller0, controller1] + return controllers + #end define + + def GetController(self, mode): + controllers = self.GetControllers() + for controllerAddr in controllers: + if mode == "stake" and self.IsControllerReadyToStake(controllerAddr): + return controllerAddr + if mode == "vote" and self.IsControllerReadyToVote(controllerAddr): + return controllerAddr + raise Exception("Validator controller not found or not ready") + #end define + + def GetControllerRequiredBalanceForLoan(self, controllerAddr, credit, interest): + cmd = f"runmethodfull {controllerAddr} required_balance_for_loan {credit} {interest}" + result = self.liteClient.Run(cmd) + data = self.Result2List(result) + if data is None: + return + min_amount = data[0] + validator_amount = data[1] + return min_amount, validator_amount + #end define + + def IsControllerReadyToStake(self, addrB64): + stop_controllers_list = self.local.db.get("stop_controllers_list") + if stop_controllers_list is not None and addrB64 in stop_controllers_list: + return False + now = get_timestamp() + config15 = self.GetConfig15() + controllerData = self.GetControllerData(addrB64) + if controllerData is None: + raise Exception(f"IsControllerReadyToStake error: controller not initialized. Use new_controllers") + lastSentStakeTime = controllerData["stake_at"] + stakeFreezeDelay = config15["validatorsElectedFor"] + config15["stakeHeldFor"] + result = lastSentStakeTime + stakeFreezeDelay < now + print(f"{addrB64}: {result}. {lastSentStakeTime}, {stakeFreezeDelay}, {now}") + return result + #end define + + def IsControllerReadyToVote(self, addrB64): + vwl = self.GetValidatorsWalletsList() + result = addrB64 in vwl + return result + #end define + + def GetControllerData(self, controllerAddr): + cmd = f"runmethodfull {controllerAddr} get_validator_controller_data" + result = self.liteClient.Run(cmd) + data = self.Result2List(result) + if data is None: + return + result_vars = ["state", "halted", "approved", "stake_amount_sent", "stake_at", "saved_validator_set_hash", "validator_set_changes_count", "validator_set_change_time", "stake_held_for", "borrowed_amount", "borrowing_time"] + controllerData = dict() + for name in result_vars: + controllerData[name] = data.pop(0) + return controllerData + #end define + + def CreateLoanRequest(self, controllerAddr): + self.local.add_log("start CreateLoanRequest function", "debug") + min_loan = self.local.db.get("min_loan", 41000) + max_loan = self.local.db.get("max_loan", 43000) + max_interest_percent = self.local.db.get("max_interest_percent", 1.5) + max_interest = int(max_interest_percent/100*16777216) + + # Проверить наличие действующего кредита + controllerData = self.GetControllerData(controllerAddr) + if controllerData["borrowed_amount"] > 0: + self.local.add_log("CreateLoanRequest warning: past loan found", "warning") + return + #end define + + # Проверить наличие средств у ликвидного пула + if self.CalculateLoanAmount(min_loan, max_loan, max_interest) == '-0x1': + raise Exception("CreateLoanRequest error: The liquid pool cannot issue the required amount of credit") + #end if + + # Проверить хватает ли ставки валидатора + min_amount, validator_amount = self.GetControllerRequiredBalanceForLoan(controllerAddr, max_loan, max_interest) + if min_amount > validator_amount: + raise Exception("CreateLoanRequest error: Validator stake is too low. Use deposit_to_controller") + #end if + + wallet = self.GetValidatorWallet() + fiftScript = self.contractsDir + "jetton_pool/fift-scripts/generate-loan-request.fif" + resultFilePath = self.tempDir + self.nodeName + wallet.name + "_loan_request.boc" + args = [fiftScript, min_loan, max_loan, max_interest, resultFilePath] + result = self.fift.Run(args) + resultFilePath = self.SignBocWithWallet(wallet, resultFilePath, controllerAddr, 1.01) + self.SendFile(resultFilePath, wallet) + self.WaitLoan(controllerAddr) + #end define + + def CalculateLoanAmount(self, min_loan, max_loan, max_interest): + data = dict() + data["address"] = self.GetLiquidPoolAddr() + data["method"] = "calculate_loan_amount" + data["stack"] = [ + ["num", min_loan*10**9], + ["num", max_loan*10**9], + ["num", max_interest], + ] + print(f"CalculateLoanAmount data: {data}") + + url = "http://127.0.0.1:8801/runGetMethod" + res = requests.post(url, json=data) + res_data = res.json() + if res_data.get("ok") is False: + error = res_data.get("error") + raise Exception(error) + result = res_data.get("result").get("stack").pop().pop() + return result + #end define + + def CalculateLoanAmount_test(self): + min_loan = self.local.db.get("min_loan", 41000) + max_loan = self.local.db.get("max_loan", 43000) + max_interest_percent = self.local.db.get("max_interest_percent", 1.5) + max_interest = int(max_interest_percent/100*16777216) + return self.CalculateLoanAmount(min_loan, max_loan, max_interest) + #end define + + def WaitLoan(self, controllerAddr): + self.local.add_log("start WaitLoan function", "debug") + for i in range(10): + time.sleep(3) + controllerData = self.GetControllerData(controllerAddr) + if controllerData["borrowed_amount"] != 0: + return + raise Exception("WaitLoan error: time out") + #end define + + def ReturnUnusedLoan(self, controllerAddr): + self.local.add_log("start ReturnUnusedLoan function", "debug") + wallet = self.GetValidatorWallet() + fileName = self.contractsDir + "jetton_pool/fift-scripts/return_unused_loan.boc" + resultFilePath = self.SignBocWithWallet(wallet, fileName, controllerAddr, 1.05) + self.SendFile(resultFilePath, wallet) + #end define + + def DepositToController(self, controllerAddr, amount): + self.local.add_log("start DepositToController function", "debug") + wallet = self.GetValidatorWallet() + fileName = self.contractsDir + "jetton_pool/fift-scripts/top-up.boc" + resultFilePath = self.SignBocWithWallet(wallet, fileName, controllerAddr, amount) + self.SendFile(resultFilePath, wallet) + #end define + + def WithdrawFromController(self, controllerAddr, amount=None): + controllerData = self.GetControllerData(controllerAddr) + if controllerData["state"] == 0: + self.WithdrawFromControllerProcess(controllerAddr, amount) + else: + self.PendWithdrawFromController(controllerAddr, amount) + #end define + + def WithdrawFromControllerProcess(self, controllerAddr, amount): + if amount is None: + account = self.GetAccount(controllerAddr) + amount = account.balance-10.1 + if int(amount) < 3: + return + #end if + + self.local.add_log("start WithdrawFromControllerProcess function", "debug") + wallet = self.GetValidatorWallet() + fiftScript = self.contractsDir + "jetton_pool/fift-scripts/withdraw-controller.fif" + resultFilePath = self.tempDir + self.nodeName + wallet.name + "_withdraw_request.boc" + args = [fiftScript, amount, resultFilePath] + result = self.fift.Run(args) + resultFilePath = self.SignBocWithWallet(wallet, resultFilePath, controllerAddr, 1.06) + self.SendFile(resultFilePath, wallet) + #end define + + def PendWithdrawFromController(self, controllerAddr, amount): + self.local.add_log("start PendWithdrawFromController function", "debug") + controllerPendingWithdraws = self.GetControllerPendingWithdraws() + controllerPendingWithdraws[controllerAddr] = amount + self.local.save() + #end define + + def HandleControllerPendingWithdraw(self, controllerPendingWithdraws, controllerAddr): + amount = controllerPendingWithdraws.get(controllerAddr) + self.WithdrawFromControllerProcess(controllerAddr, amount) + controllerPendingWithdraws.pop(controllerAddr) + #end define + + def GetControllerPendingWithdraws(self): + bname = "controllerPendingWithdraws" + controllerPendingWithdraws = self.local.db.get(bname) + if controllerPendingWithdraws is None: + controllerPendingWithdraws = dict() + self.local.db[bname] = controllerPendingWithdraws + return controllerPendingWithdraws + #end define + + def SignElectionRequestWithController(self, controllerAddr, startWorkTime, adnlAddr, validatorPubkey_b64, validatorSignature, maxFactor, stake): + self.local.add_log("start SignElectionRequestWithController function", "debug") + fileName = self.tempDir + str(startWorkTime) + "_validator-query.boc" + fiftScript = self.contractsDir + "jetton_pool/fift-scripts/controller-elect-signed.fif" + args = [fiftScript, controllerAddr, startWorkTime, maxFactor, adnlAddr, validatorPubkey_b64, validatorSignature, fileName, stake] + self.local.add_log(f"SignElectionRequestWithController args: {args}", "debug") + result = self.fift.Run(args) + self.local.add_log(f"SignElectionRequestWithController result: {result}", "debug") + pubkey = parse(result, "validator public key ", '\n') + fileName = parse(result, "Saved to file ", '\n') + return pubkey, fileName + #end define + + def ControllersUpdateValidatorSet(self): + self.local.add_log("start ControllersUpdateValidatorSet function", "debug") + using_controllers = self.local.db.get("using_controllers") + user_controllers = self.local.db.get("user_controllers", list()) + old_controllers = self.local.db.get("old_controllers", list()) + for controller in using_controllers + user_controllers + old_controllers: + self.ControllerUpdateValidatorSet(controller) + #end define + + def ControllerUpdateValidatorSet(self, controllerAddr): + self.local.add_log("start ControllerUpdateValidatorSet function", "debug") + wallet = self.GetValidatorWallet() + controllers = self.GetControllers() + controllerData = self.GetControllerData(controllerAddr) + if controllerData is None: + return + #end if + + timeNow = int(time.time()) + config34 = self.GetConfig34() + fullElectorAddr = self.GetFullElectorAddr() + returnedStake = self.GetReturnedStake(fullElectorAddr, controllerAddr) + controllerPendingWithdraws = self.GetControllerPendingWithdraws() + if (controllerData["state"] == 3 and + controllerData["validator_set_changes_count"] < 2 and + controllerData["validator_set_change_time"] < config34["startWorkTime"]): + self.ControllerUpdateValidatorSetProcess(controllerAddr, wallet) + controllerData = self.GetControllerData(controllerAddr) + if (returnedStake > 0 and + controllerData["state"] == 3 and + controllerData["validator_set_changes_count"] >= 2 and + timeNow - controllerData["validator_set_change_time"] > controllerData["stake_held_for"] + 60): + self.ControllerRecoverStake(controllerAddr) + controllerData = self.GetControllerData(controllerAddr) + if (controllerData["state"] == 0 and + controllerData["borrowed_amount"] > 0 and + config34["startWorkTime"] > controllerData["borrowing_time"]): + self.ReturnUnusedLoan(controllerAddr) + if (controllerData["state"] == 0 and controllerAddr in controllerPendingWithdraws): + self.HandleControllerPendingWithdraw(controllerPendingWithdraws, controllerAddr) + if controllerAddr not in controllers: + self.WithdrawFromController(controllerAddr) + #end define + + def ControllerUpdateValidatorSetProcess(self, controllerAddr, wallet): + self.local.add_log("start ControllerUpdateValidatorSetProcess function", "debug") + fileName = self.contractsDir + "jetton_pool/fift-scripts/update_validator_hash.boc" + resultFilePath = self.SignBocWithWallet(wallet, fileName, controllerAddr, 1.07) + self.SendFile(resultFilePath, wallet) + self.local.add_log("ControllerUpdateValidatorSetProcess completed") + #end define + + def ControllerRecoverStake(self, controllerAddr): + wallet = self.GetValidatorWallet() + self.local.add_log("start ControllerRecoverStake function", "debug") + fileName = self.contractsDir + "jetton_pool/fift-scripts/recover_stake.boc" + resultFilePath = self.SignBocWithWallet(wallet, fileName, controllerAddr, 1.04) + self.SendFile(resultFilePath, wallet) + self.local.add_log("ControllerRecoverStake completed") + #end define + + def StopController(self, controllerAddr): + stop_controllers_list = self.local.db.get("stop_controllers_list") + if stop_controllers_list is None: + stop_controllers_list = list() + if controllerAddr not in stop_controllers_list: + stop_controllers_list.append(controllerAddr) + self.local.db["stop_controllers_list"] = stop_controllers_list + + user_controllers = self.local.db.get("user_controllers") + if user_controllers is not None and controllerAddr in user_controllers: + user_controllers.remove(controllerAddr) + self.local.save() + #end define + + def AddController(self, controllerAddr): + user_controllers = self.local.db.get("user_controllers") + if user_controllers is None: + user_controllers = list() + if controllerAddr not in user_controllers: + user_controllers.append(controllerAddr) + self.local.db["user_controllers"] = user_controllers + + stop_controllers_list = self.local.db.get("stop_controllers_list") + if stop_controllers_list is not None and controllerAddr in stop_controllers_list: + stop_controllers_list.remove(controllerAddr) + self.local.save() + #end define + + def CheckLiquidPool(self): + liquid_pool_addr = self.GetLiquidPoolAddr() + account = self.GetAccount(liquid_pool_addr) + history = self.GetAccountHistory(account, 5000) + addrs_list = list() + for message in history: + if message.srcAddr is None or message.value is None: + continue + srcAddrFull = f"{message.srcWorkchain}:{message.srcAddr}" + destAddFull = f"{message.destWorkchain}:{message.destAddr}" + if srcAddrFull == account.addrFull: + fromto = destAddFull + else: + fromto = srcAddrFull + fromto = self.AddrFull2AddrB64(fromto) + if fromto not in addrs_list: + addrs_list.append(fromto) + #end for + + for controllerAddr in addrs_list: + account = self.GetAccount(controllerAddr) + version = self.GetWalletVersionFromHash(account.codeHash) + if version is None or "controller" not in version: + continue + print(f"check controller: {controllerAddr}") + self.ControllerUpdateValidatorSet(controllerAddr) + #end define + def create_single_pool(self, pool_name, owner_address): self.local.add_log("start create_single_pool function", "debug") @@ -3778,3 +4200,8 @@ def Dec2HexAddr(dec): h64 = hu.rjust(64, "0") return h64 #end define + +def HexAddr2Dec(h): + d = int(h, 16) + return d +#end define diff --git a/mytonctrl/mytonctrl.py b/mytonctrl/mytonctrl.py index b4a7768a..2b47e6a6 100755 --- a/mytonctrl/mytonctrl.py +++ b/mytonctrl/mytonctrl.py @@ -138,6 +138,20 @@ def inject_globals(func): console.AddItem("activate_single_pool", inject_globals(activate_single_pool), local.translate("activate_single_pool_cmd")) console.AddItem("import_pool", inject_globals(import_pool), local.translate("import_pool_cmd")) + console.AddItem("create_controllers", inject_globals(CreateControllers), local.translate("_")) + console.AddItem("update_controllers", inject_globals(CreateControllers), local.translate("_")) + console.AddItem("controllers_list", inject_globals(PrintControllersList), local.translate("_")) + console.AddItem("get_controller_data", inject_globals(GetControllerData), local.translate("_")) + console.AddItem("deposit_to_controller", inject_globals(DepositToController), local.translate("_")) + console.AddItem("withdraw_from_controller", inject_globals(WithdrawFromController), local.translate("_")) + console.AddItem("calculate_annual_controller_percentage", inject_globals(CalculateAnnualControllerPercentage), local.translate("_")) + console.AddItem("controller_update_validator_set", inject_globals(ControllerUpdateValidatorSet), local.translate("_")) + console.AddItem("stop_controller", inject_globals(StopController), local.translate("_")) + console.AddItem("stop_and_withdraw_controller", inject_globals(StopAndWithdrawController), local.translate("_")) + console.AddItem("add_controller", inject_globals(AddController), local.translate("_")) + console.AddItem("check_liquid_pool", inject_globals(CheckLiquidPool), local.translate("_")) + console.AddItem("test_calculate_loan_amount", inject_globals(CalculateLoanAmount_test), local.translate("_")) + # console.AddItem("pt", inject_globals(PrintTest), "PrintTest") # console.AddItem("sl", inject_globals(sl), "sl") console.AddItem("cleanup", inject_globals(cleanup_validator_db), local.translate("cleanup_cmd")) @@ -1489,6 +1503,145 @@ def import_pool(ton, args): color_print("import_pool - {green}OK{endc}") #end define +def CreateControllers(ton, args): + ton.CreateControllers() + color_print("CreateControllers - {green}OK{endc}") +#end define + +def PrintControllersList(ton, args): + new_controllers = ton.GetControllers() + using_controllers = ton.GetSettings("using_controllers") + old_controllers = ton.GetSettings("old_controllers") + user_controllers_list = ton.GetSettings("user_controllers_list") + print("using controllers:") + PrintControllersListProcess(ton, using_controllers) + if new_controllers != using_controllers: + print() + print("new controllers:") + PrintControllersListProcess(ton, new_controllers) + if old_controllers is not None and len(old_controllers) > 0: + print() + print("old controllers:") + PrintControllersListProcess(ton, old_controllers) + if user_controllers_list is not None and len(user_controllers_list) > 0: + print() + print("user controllers:") + PrintControllersListProcess(ton, user_controllers_list) +#end define + +def PrintControllersListProcess(ton, controllers): + table = list() + table += [["Address", "Status", "Balance", "Approved", "State"]] + for controllerAddr in controllers: + account = ton.GetAccount(controllerAddr) + controllerData = ton.GetControllerData(controllerAddr) + approved = True if controllerData and controllerData["approved"] == -1 else False + state = controllerData["state"] if controllerData else None + table += [[controllerAddr, account.status, account.balance, approved, state]] + print_table(table) +#end define + +def GetControllerData(ton, args): + try: + controllerAddr = args[0] + except: + color_print("{red}Bad args. Usage:{endc} get_controller_data ") + return + controllerData = ton.GetControllerData(controllerAddr) + print(json.dumps(controllerData, indent=4)) +#end define + +def DepositToController(ton, args): + try: + controllerAddr = args[0] + amount = float(args[1]) + except: + color_print("{red}Bad args. Usage:{endc} deposit_to_controller ") + return + ton.DepositToController(controllerAddr, amount) +#end define + +def WithdrawFromController(ton, args): + try: + controllerAddr = args[0] + amount = GetItemFromList(args, 1) + except: + color_print("{red}Bad args. Usage:{endc} withdraw_from_controller [amount]") + return + ton.WithdrawFromController(controllerAddr, amount) +#end define + +def CalculateAnnualControllerPercentage(ton, args): + try: + percentPerRound = float(args[0]) + except: + percentPerRound = ton.GetSettings("max_interest_percent") + config15 = ton.GetConfig(15) + roundPeriod = config15["validators_elected_for"] + rounds = 365 * 24 * 3600 / roundPeriod + yearInterest = (1 + percentPerRound / 100) * rounds + yearInterestPercent = round(yearInterest / 100, 2) + print("roundPeriod", roundPeriod) + print("rounds", rounds) + print("percentPerRound", percentPerRound) + print("yearInterest", yearInterest) + print(f"yearInterestPercent: {yearInterestPercent}%") +#end define + +def ControllerUpdateValidatorSet(ton, args): + try: + controllerAddr = args[0] + except: + color_print("{red}Bad args. Usage:{endc} controller_update_validator_set ") + return + ton.ControllerUpdateValidatorSet(controllerAddr) + color_print("ControllerUpdateValidatorSet - {green}OK{endc}") +#end define + +def StopController(ton, args): + try: + controllerAddr = args[0] + except: + color_print("{red}Bad args. Usage:{endc} stop_controller ") + return + ton.StopController(controllerAddr) + color_print("StopController - {green}OK{endc}") +#end define + +def StopAndWithdrawController(ton, args): + try: + controllerAddr = args[0] + amount = GetItemFromList(args, 1) + except: + color_print("{red}Bad args. Usage:{endc} stop_and_withdraw_controller [amount]") + return + if amount is None: + account = ton.GetAccount(controllerAddr) + amount = account.balance-10.1 + ton.StopController(controllerAddr) + ton.WithdrawFromController(controllerAddr, amount) + color_print("StopAndWithdrawController - {green}OK{endc}") +#end define + +def AddController(ton, args): + try: + controllerAddr = args[0] + except: + color_print("{red}Bad args. Usage:{endc} add_controller ") + return + ton.AddController(controllerAddr) + color_print("AddController - {green}OK{endc}") +#end define + +def CheckLiquidPool(ton, args): + ton.CheckLiquidPool() + color_print("CheckLiquidPool - {green}OK{endc}") +#end define + +def CalculateLoanAmount_test(ton, args): + t = ton.CalculateLoanAmount_test() + print(t) + ### ### Start of the program diff --git a/mytonctrl/resources/translate.json b/mytonctrl/resources/translate.json index 5ce41b98..60350335 100644 --- a/mytonctrl/resources/translate.json +++ b/mytonctrl/resources/translate.json @@ -239,6 +239,11 @@ "ru": "Эффективность валидатора: {0}", "zh_TW": "驗證者效率: {0}" }, + "local_status_validator_efficiency_ver2": { + "en": "Validator efficiency, ver.2: {0}", + "ru": "Эффективность валидатора, вер.2: {0}", + "zh_TW": "驗證器效率，版本2： {0}" + }, "local_status_adnl_addr": { "en": "ADNL address of local validator: {0}", "ru": "ADNL адрес локального валидатора: {0}", diff --git a/mytonctrl/scripts/upgrade.sh b/mytonctrl/scripts/upgrade.sh index 44cac111..b5f3c8ed 100644 --- a/mytonctrl/scripts/upgrade.sh +++ b/mytonctrl/scripts/upgrade.sh @@ -50,8 +50,9 @@ fi # compile openssl_3 rm -rf ${bindir}/openssl_3 -git clone --branch openssl-3.1.4 https://github.com/openssl/openssl ${bindir}/openssl_3 +git clone https://github.com/openssl/openssl ${bindir}/openssl_3 cd ${bindir}/openssl_3 +git checkout openssl-3.1.4 ./config make build_libs -j$(nproc) opensslPath=`pwd` diff --git a/scripts/ton_http_api_installer.sh b/scripts/ton_http_api_installer.sh new file mode 100644 index 00000000..9327f870 --- /dev/null +++ b/scripts/ton_http_api_installer.sh @@ -0,0 +1,40 @@ +#!/bin/bash +set -e + +# Проверить sudo +if [ "$(id -u)" != "0" ]; then + echo "Please run script as root" + exit 1 +fi + +# Цвета +COLOR='\033[92m' +ENDC='\033[0m' + +# Установка компонентов python3 +pip3 install virtualenv + +# Подготовить папку с виртуальным окружением +echo -e "${COLOR}[1/4]${ENDC} Preparing the virtual environment" +venv_path="/opt/virtualenv/ton_http_api" +virtualenv ${venv_path} + +# Установка компонентов python3 +echo -e "${COLOR}[2/4]${ENDC} Installing required packages" +user=$(logname) +venv_pip3="${venv_path}/bin/pip3" +${venv_pip3} install ton-http-api +chown -R ${user}:${user} ${venv_path} + +# Прописать автозагрузку +echo -e "${COLOR}[3/4]${ENDC} Add to startup" +venv_ton_http_api="${venv_path}/bin/ton-http-api" +tonlib_path="/usr/bin/ton/tonlib/libtonlibjson.so" +ls_config="/usr/bin/ton/localhost.config.json" +cmd="from sys import path; path.append('/usr/src/mytonctrl/'); from mypylib.mypylib import add2systemd; add2systemd(name='ton_http_api', user='${user}', start='${venv_ton_http_api} --host 127.0.0.1 --port 8801 --liteserver-config ${ls_config} --cdll-path ${tonlib_path} --tonlib-keystore /tmp/tonlib_keystore/')" +python3 -c "${cmd}" +systemctl restart ton_http_api + +# Конец +echo -e "${COLOR}[4/4]${ENDC} ton_http_api service installation complete" +exit 0 diff --git a/scripts/ton_installer.sh b/scripts/ton_installer.sh index 6cf9a990..9f053379 100644 --- a/scripts/ton_installer.sh +++ b/scripts/ton_installer.sh @@ -92,10 +92,10 @@ pip3 install psutil crc16 requests # build openssl 3.0 echo -e "${COLOR}[2/6]${ENDC} Building OpenSSL 3.0" rm -rf $BIN_DIR/openssl_3 -git clone --branch openssl-3.1.4 https://github.com/openssl/openssl $BIN_DIR/openssl_3 +git clone https://github.com/openssl/openssl $BIN_DIR/openssl_3 cd $BIN_DIR/openssl_3 opensslPath=`pwd` -git checkout +git checkout openssl-3.1.4 ./config make build_libs -j$(nproc)