diff --git a/docs/en/controllers.md b/docs/en/controllers.md new file mode 100644 index 00000000..46b12c8a --- /dev/null +++ b/docs/en/controllers.md @@ -0,0 +1,94 @@ +# Controllers + +## Launching a Validator in Controller Mode + +1. Prepare the hardware for the validator - 32 virtual cores, 64GB of memory, 1TB SSD, fixed IP address, and 1Gb/s internet speed. + + To maintain network stability, it is recommended to place validators in different locations around the world, rather than concentrating them in a single data center. You can use [this site](https://status.toncenter.com/) to determine the load on various locations. According to the map, there is a high load on data centers in Europe, especially in Finland, Germany, and Paris. Therefore, using providers such as Hetzner and OVH is not recommended. + + > Ensure your hardware meets or exceeds the specified configuration. Running the validator on inappropriate hardware can harm the network and result in penalties. + > Since May 2021, Hetzner has prohibited mining on its servers. This rule currently applies to both PoW and PoS algorithms. Even installing a regular node will be considered a breach of contract. + +2. Install and synchronize **mytonctrl** according to the description in [this instruction](https://github.com/ton-blockchain/mytonctrl/blob/master/docs/en/manual-ubuntu.md) — follow **only** paragraphs 1, 2, and 3. + + You can also refer to this [Video Tutorial](https://docs.ton.org/participate/run-nodes/full-node#installation) for additional help. + +3. Transfer 1 TON to the validator wallet address, which is displayed in the `wl` list. + +4. Use the `aw` command to activate the validator's wallet. + +5. Transfer enough TON to the validator wallet address. + +6. Enable the ton-http-api service: + ``` + mytonctrl -> installer -> enable THA + ``` + Exit installer mode with `Ctrl+D` + +7. Set the liquid pool address, which will lend TON for validation: + ``` + set liquid_pool_addr + ``` + +8. Set the lending parameters that acceptable to you: + ``` + set min_loan 41000 + set max_loan 43000 + set max_interest_percent 1.5 + ``` + + where +* `41000` is the minimum loan amount we are willing to receive from the liquid pool, +* `43000` is the maximum loan amount we are willing to receive from the liquid pool, +* `1.5` 1.5 is the maximum interest rate value for the liquid pool per validation cycle, which we have agreed upon. + +9. Display the annual percentage of profit from the liquid pool: + ``` + calculate_annual_controller_percentage + ``` + +10. Create two controllers with a single command: + + ``` + new_controllers + ``` + +11. Enter `controllers_list` to display the controller addresses: + + ``` + controllers_list + Address Status Balance + kf89KYOeRPRRDyjt_3bPsz92cKSghRqw64efNr8mT1eeHDaS active 0.704345 + kf_lT8QNykLh5PN5ictNX22maiyIf9iK787fXg6nJ_zB-jbN active 0.720866 + ``` + +12. Make a validator deposit in each controller: + + +``` +deposit_to_controller kf89KYOeRPRRDyjt_3bPsz92cKSghRqw64efNr8mT1eeHDaS 10000 +deposit_to_controller kf_lT8QNykLh5PN5ictNX22maiyIf9iK787fXg6nJ_zB-jbN 10000 +``` + + +where `10000` TON is the deposit amount. + +13. Get approval for the controllers. Each pool may have its own approval issuance policy, check with the operator. + +14. Set controller mode: + + ```bash + set useController true + set stake null + ``` + +> (!) If you were previously using nominator pools, do not forget to disable them using the `set usePool false` command. + + +## Switching a Regular Validator to Controller Operating Mode + +1. Enter `set stake 0` to stop participating in elections. + +2. Wait until both of your deposits have been returned from the Elector. + +3. Follow the instructions under "Launching a Validator in Controller Mode", beginning with **Step 6**. \ No newline at end of file diff --git a/docs/ru/controllers.md b/docs/ru/controllers.md new file mode 100644 index 00000000..d3944549 --- /dev/null +++ b/docs/ru/controllers.md @@ -0,0 +1,93 @@ +# Контроллеры + +## Запуск валидатора в режиме работы контроллеров + +1. Подготовьте аппаратное обеспечение для валидатора - 32 виртуальных ядер, 64GB памяти, 1TB SSD, фиксированный IP-адрес, скорость интернета 1Gb/s. + + Для поддержания стабильности сети рекомендуется разместить валидаторы в различных местах по всему миру, а не концентрировать их в одном дата-центре. Вы можете использовать [этот сайт](https://status.toncenter.com/) для определения нагрузки на различные места. Согласно карте, высокая нагрузка на дата-центры в Европе, особенно в Финляндии, Германии и Париже. Поэтому использование провайдеров, таких как Hetzner и OVH, не рекомендуется. + + > Ваше оборудование должно соответствовать указанной конфигурации или быть выше. Запуск валидатора на слабом оборудовании негативно влияет на сеть и может привести к штрафам. + + > С мая 2021 года Hetzner запретил майнинг на своих серверах, в настоящее время под это правило попадают алгоритмы PoW и PoS. Установка даже обычного узла будет считаться нарушением условий договора. + +2. Установите и синхронизируйте **mytonctrl** в соответствии с описанием в [этой инструкции](https://github.com/ton-blockchain/mytonctrl/blob/master/docs/ru/manual-ubuntu.md) — следуйте **только** пунктам 1, 2 и 3. + + Вы также можете обратиться к этой [Видеоинструкции](https://docs.ton.org/participate/run-nodes/full-node#installation) для дополнительной помощи. + +3. Переведите 1 TON на адрес кошелька валидатора, который отображается в списке `wl`. + +4. Используйте команду `aw` для активации кошелька валидатора. + +5. Переведите достаточно TON на адрес кошелька валидатора. + +6. Включите ton-http-api службу: + ``` + mytonctrl -> installer -> enable THA + ``` + Выйдите из режима установщика сочетанием клавиш `Ctrl+D` + +7. Задайте адрес ликвидного пула, который будет одалживать TON для валидации: + ``` + set liquid_pool_addr + ``` + +8. Задайте параметры кредитования, которые вам подходят: + ``` + set min_loan 41000 + set max_loan 43000 + set max_interest_percent 1.5 + ``` + + где +* `41000` - это минимальная сумма кредита который мы готовы получить у ликвидного пула, +* `43000` - это максимальная сумма кредита который мы готовы получить у ликвидного пула, +* `1.5` - это максимальная процентная ставка ликвидного пула за один цикл валидации на которую мы согласны. + +9. Отобразите годовой процент прибыли от ликвидного пула: + ``` + calculate_annual_controller_percentage + ``` + +10. Создайте два контроллера одной командой: + + ``` + new_controllers + ``` + +11. Введите `controllers_list` чтобы отобразить адреса контроллеров: + + ``` + controllers_list + Address Status Balance + kf89KYOeRPRRDyjt_3bPsz92cKSghRqw64efNr8mT1eeHDaS active 0.704345 + kf_lT8QNykLh5PN5ictNX22maiyIf9iK787fXg6nJ_zB-jbN active 0.720866 + ``` + +12. Совершите депозит валидатора в каждый контроллер: + + ``` + deposit_to_controller kf89KYOeRPRRDyjt_3bPsz92cKSghRqw64efNr8mT1eeHDaS 10000 + deposit_to_controller kf_lT8QNykLh5PN5ictNX22maiyIf9iK787fXg6nJ_zB-jbN 10000 + ``` + + где `10000` TON - это сумма депозита. + +13. Получите аппрувал контроллеров. У каждого пула может быть своя политика выдачи аппруволов, уточняйте у оператора. + +14. Активируйте режим работы контроллеров: + + ```bash + set useController true + set stake null + ``` + + > (!) Если до этого вы использовали номинатор-пулы, не забудьте отключить их использование командой `set usePool false`. + + +## Переключение обычного валидатора в режим работы контроллеров + +1. Введите `set stake 0`, чтобы отключить участие в выборах. + +2. Дождитесь, когда оба ваших депозита вернутся от электора. + +3. Следуйте инструкциям "Запуск валидатора в режиме работы контроллеров", начиная с **6-го шага**. diff --git a/functions/session_stats.py b/functions/session_stats.py new file mode 100644 index 00000000..ba08b06e --- /dev/null +++ b/functions/session_stats.py @@ -0,0 +1,136 @@ +#!/usr/bin/env python3 +# -*- coding: utf_8 -*- + +import os +import sys +import time +import json +sys.path.append("/usr/src/mytonctrl/") +from mypylib.mypylib import Dict + +def read_session_stats(need_time_period): + #print(f"read_session_stats: {need_time_period}") + file_path = "/var/ton-work/log.session-stats" + average_block_time = calculate_average_block_time(file_path) + need_blocks = int(need_time_period/average_block_time) + + lines = read_last_lines(file_path, need_lines=need_blocks) + #lines = read_all_lines(file_path) + data = lines2data(lines) + + result = Dict() + result.my_blocks = 0 + result.my_need_blocks = 0 + result.all_master_blocks = 0 + result.all_blocks = len(data) + for buff in data: + if buff.id.workchain == -1: + result.all_master_blocks += 1 + first_producer = buff.rounds[0].producers[0] + if buff.self != first_producer.id: + continue + result.my_need_blocks += 1 + if buff.self == buff.creator: + result.my_blocks += 1 + #end for + return result +#end define + +def calculate_average_block_time(file_path): + blocks = 100 + lines = read_last_lines(file_path, need_lines=blocks) + data = lines2data(lines) + first_block, last_block = get_first_last_block(data) + + diff = int(last_block.timestamp) - int(first_block.timestamp) + average_block_time = round(diff/blocks, 2) + #print("average_block_time:", average_block_time) + return average_block_time +#end define + +def get_first_last_block(data, last_index=-1): + first_block = data[0] + last_block = data[last_index] + if first_block.id.workchain == last_block.id.workchain: + blocks = int(last_block.id.seqno) - int(first_block.id.seqno) + else: + first_block, last_block = get_first_last_block(data, last_index=last_index-1) + return first_block, last_block +#end define + +def lines2data(lines): + #start = time.time() + data = list() + for line in lines: + try: + buff = json.loads(line) + data.append(Dict(buff)) + except json.decoder.JSONDecodeError: + pass + #end for + + #end = time.time() + #diff = round(end - start, 2) + #print(f"lines2data completed in {diff} seconds") + return data +#end define + +def read_all_lines(file_path): + #start = time.time() + with open(file_path, 'rt') as file: + text = file.read() + #end with + lines = text.split('\n') + + #end = time.time() + #diff = round(end - start, 2) + #print(f"read_all_lines completed in {diff} seconds") + return lines +#end define + +def read_last_lines(file_path, need_lines): + lines = list() + max_lines = 30000 + if need_lines < 1: + return lines + elif need_lines > max_lines: + need_lines = max_lines + #print(f"read_last_lines: {need_lines}") + #start = time.time() + with open(file_path, 'rb') as file: + find_last_lines(file, need_lines) + for i in range(need_lines): + line = file.readline().decode() + lines.append(line) + #end with + + + #end = time.time() + #diff = round(end - start, 2) + #print(f"read_last_lines {len(lines)} completed in {diff} seconds") + return lines +#end define + +def find_last_lines(file, need_lines): + # catch OSError in case of a one line file + try: + find_lines = 0 + #file.seek(-2, os.SEEK_END) + #while find_lines != need_lines: + # if file.read(1) == b'\n': + # find_lines += 1 + # file.seek(-2, os.SEEK_CUR) + + file.seek(-100, os.SEEK_END) + while find_lines != need_lines: + if b'\n' in file.read(100): + find_lines += 1 + file.seek(-200, os.SEEK_CUR) + except OSError: + file.seek(0) +#end define + + + + + diff --git a/mytoncore/mytoncore.py b/mytoncore/mytoncore.py index bbc73c29..a6d0765c 100644 --- a/mytoncore/mytoncore.py +++ b/mytoncore/mytoncore.py @@ -534,6 +534,14 @@ def GetVersionFromCodeHash(self, inputHash): arr["pool"] = "399838da9489139680e90fd237382e96ba771fdf6ea27eb7d513965b355038b4" arr["spool"] = "fc2ae44bcaedfa357d0091769aabbac824e1c28f14cc180c0b52a57d83d29054" arr["spool_r2"] = "42bea8fea43bf803c652411976eb2981b9bdb10da84eb788a63ea7a01f2a044d" + arr["liquid_pool_r1"] = "82bc5760719c34395f80df76c42dc5d287f08f6562c643601ebed6944302dcc2" + arr["liquid_pool_r2"] = "95abec0a66ac63b0fbcf28466eb8240ddcd88f97300691511d9c9975d5521e4a" + arr["liquid_pool_r3"] = "22a023bc75b649ff2b5b183cd0d34cd413e6e27ee6d6ad0787f75ad39787ed4e" + arr["liquid_pool_r4"] = "77282b45fd7cfc72ca68fe97af33ad10078730ceaf55e20534c9526c48d602d2" + arr["controller_r1"] = "0949cf92963dd27bb1e6bf76487807f20409131b6110acbc18b7fbb90280ccf0" + arr["controller_r2"] = "01118b9553151fb9bc81704a4b3e0fc7b899871a527d44435a51574806863e2c" + arr["controller_r3"] = "e4d8ce8ff7b4b60c76b135eb8702ce3c86dc133fcee7d19c7aa18f71d9d91438" + arr["controller_r4"] = "dec125a4850c4ba24668d84252b04c6ad40abf5c9d413a429b56bfff09ea25d4" for version, hash in arr.items(): if hash == inputHash: return version @@ -1284,8 +1292,10 @@ def ProcessRecoverStake(self): def GetStake(self, account, args=None): stake = self.local.db.get("stake") usePool = self.local.db.get("usePool") + useController = self.local.db.get("useController") stakePercent = self.local.db.get("stakePercent", 99) vconfig = self.GetValidatorConfig() + validators = vconfig.get("validators") config17 = self.GetConfig17() # Check if optional arguments have been passed to us @@ -1312,6 +1322,8 @@ def GetStake(self, account, args=None): if stake is None and usePool and not is_single_nominator: stake = account.balance - 20 + if stake is None and useController: + stake = account.balance - 50 if stake is None: sp = stakePercent / 100 if sp > 1 or sp < 0: @@ -1397,17 +1409,13 @@ def GetValidatorWallet(self, mode="stake"): def ElectionEntry(self, args=None): usePool = self.local.db.get("usePool") + useController = self.local.db.get("useController") wallet = self.GetValidatorWallet() addrB64 = wallet.addrB64 if wallet is None: raise Exception("Validator wallet not found") #end if - if usePool: - pool = self.GetPool(mode="stake") - addrB64 = pool.addrB64 - #end if - self.local.add_log("start ElectionEntry function", "debug") # Check if validator is not synchronized validatorStatus = self.GetValidatorStatus() @@ -1456,6 +1464,16 @@ def ElectionEntry(self, args=None): return #end if + if usePool: + pool = self.GetPool(mode="stake") + addrB64 = pool.addrB64 + elif useController: + controllerAddr = self.GetController(mode="stake") + self.CheckController(controllerAddr) + self.CreateLoanRequest(controllerAddr) + addrB64 = controllerAddr + #end if + # Calculate stake account = self.GetAccount(addrB64) stake = self.GetStake(account, args) @@ -1483,6 +1501,14 @@ def ElectionEntry(self, args=None): # Send boc file to TON resultFilePath = self.SignBocWithWallet(wallet, resultFilePath, pool.addrB64, 1.3) self.SendFile(resultFilePath, wallet) + elif useController: + var1 = self.CreateElectionRequest(controllerAddr, startWorkTime, adnl_addr, maxFactor) + validatorSignature = self.GetValidatorSignature(validatorKey, var1) + validatorPubkey, resultFilePath = self.SignElectionRequestWithController(controllerAddr, startWorkTime, adnl_addr, validatorPubkey_b64, validatorSignature, maxFactor, stake) + + # Send boc file to TON + resultFilePath = self.SignBocWithWallet(wallet, resultFilePath, controllerAddr, 1.03) + self.SendFile(resultFilePath, wallet) else: var1 = self.CreateElectionRequest(wallet, startWorkTime, adnl_addr, maxFactor) validatorSignature = self.GetValidatorSignature(validatorKey, var1) @@ -1579,7 +1605,7 @@ def PoolUpdateValidatorSet(self, poolAddr, wallet): poolData = self.GetPoolData(poolAddr) if poolData is None: return - #en if + #end if timeNow = int(time.time()) config34 = self.GetConfig34() @@ -2043,6 +2069,11 @@ def GetSaveElectionEntries(self, electionId): return result #end define + def calculate_offer_pseudohash(self, offer_hash: str, param_id: int): + config_val = self.GetConfig(param_id) + pseudohash_bytes = offer_hash.encode() + json.dumps(config_val, sort_keys=True).encode() + return hashlib.sha256(pseudohash_bytes).hexdigest() + def GetOffers(self): self.local.add_log("start GetOffers function", "debug") fullConfigAddr = self.GetFullConfigAddr() @@ -2087,9 +2118,7 @@ def GetOffers(self): item["approvedPercent"] = round(availableWeight / totalWeight * 100, 3) item["isPassed"] = (weightRemaining < 0) #item["pseudohash"] = hashlib.sha256(param_val.encode()).hexdigest() - config_val = self.GetConfig(param_id) - pseudohash_bytes = hash.encode() + json.dumps(config_val, sort_keys=True).encode() - item["pseudohash"] = hashlib.sha256(pseudohash_bytes).hexdigest() + item['pseudohash'] = self.calculate_offer_pseudohash(hash, param_id) offers.append(item) #end for return offers @@ -2345,7 +2374,7 @@ def VoteOffer(self, offerHash): resultFilePath = self.SignProposalVoteRequestWithValidator(offerHash, validatorIndex, validatorPubkey_b64, validatorSignature) resultFilePath = self.SignBocWithWallet(wallet, resultFilePath, fullConfigAddr, 1.5) self.SendFile(resultFilePath, wallet) - self.AddSaveOffer(offer) + self.add_save_offer(offer) #end define def VoteComplaint(self, electionId, complaintHash): @@ -2968,9 +2997,15 @@ def WriteBookmarkData(self, bookmark): def offers_gc(self, save_offers): current_offers = self.GetOffers() current_offers_hashes = [offer.get("hash") for offer in current_offers] - for offer in list(save_offers.keys()): - if offer not in current_offers_hashes: - save_offers.pop(offer) + for offer_hash, offer in list(save_offers.items()): + if offer_hash not in current_offers_hashes: + if isinstance(offer, list): + param_id = offer[1] + if param_id is not None and offer[0] != self.calculate_offer_pseudohash(offer_hash, param_id): + # param has been changed so no need to keep anymore + save_offers.pop(offer) + else: # old version of offer in db + save_offers.pop(offer) return save_offers def GetSaveOffers(self): @@ -2983,12 +3018,12 @@ def GetSaveOffers(self): return save_offers #end define - def AddSaveOffer(self, offer): - offerHash = offer.get("hash") - offerPseudohash = offer.get("pseudohash") - saveOffers = self.GetSaveOffers() - if offerHash not in saveOffers: - saveOffers[offerHash] = offerPseudohash + def add_save_offer(self, offer): + offer_hash = offer.get("hash") + offer_pseudohash = offer.get("pseudohash") + save_offers = self.GetSaveOffers() + if offer_hash not in save_offers: + save_offers[offer_hash] = [offer_pseudohash, offer.get('config', {}).get("id")] self.local.save() #end define @@ -3025,6 +3060,8 @@ def GetDestinationAddr(self, destination): #end define def AddrFull2AddrB64(self, addrFull, bounceable=True): + if addrFull is None or "None" in addrFull: + return testnet = self.IsTestnet() buff = addrFull.split(':') workchain = int(buff[0]) @@ -3655,6 +3692,400 @@ def GetPoolData(self, addrB64): return poolData #end define + def GetLiquidPoolAddr(self): + liquid_pool_addr = self.local.db.get("liquid_pool_addr") + if liquid_pool_addr is None: + raise Exception("GetLiquidPoolAddr error: liquid_pool_addr not set") + return liquid_pool_addr + #end define + + def CreateControllers(self): + new_controllers = self.GetControllers() + old_controllers = self.local.db.get("using_controllers", list()) + if new_controllers == old_controllers: + return + #end if + + self.local.add_log("start CreateControllers function", "debug") + wallet = self.GetValidatorWallet() + liquid_pool_addr = self.GetLiquidPoolAddr() + contractPath = self.contractsDir + "jetton_pool/" + if not os.path.isdir(contractPath): + self.DownloadContract("https://github.com/igroman787/jetton_pool") + #end if + + fileName0 = contractPath + "fift-scripts/deploy_controller0.boc" + fileName1 = contractPath + "fift-scripts/deploy_controller1.boc" + resultFilePath0 = self.SignBocWithWallet(wallet, fileName0, liquid_pool_addr, 1) + self.SendFile(resultFilePath0, wallet) + time.sleep(10) + resultFilePath1 = self.SignBocWithWallet(wallet, fileName1, liquid_pool_addr, 1) + self.SendFile(resultFilePath1, wallet) + + # Сохранить новые контроллеры + self.local.db["old_controllers"] = old_controllers + self.local.db["using_controllers"] = new_controllers + self.local.save() + #end define + + def GetControllerAddress(self, controller_id): + wallet = self.GetValidatorWallet() + addr_hash = HexAddr2Dec(wallet.addr) + liquid_pool_addr = self.GetLiquidPoolAddr() + cmd = f"runmethodfull {liquid_pool_addr} get_controller_address_legacy {controller_id} {wallet.workchain} {addr_hash}" + result = self.liteClient.Run(cmd) + buff = self.Result2List(result) + wc = buff[0] + addr_hash = Dec2HexAddr(buff[1]) + addrFull = f"{wc}:{addr_hash}" + controllerAddr = self.AddrFull2AddrB64(addrFull) + return controllerAddr + #end define + + def CheckController(self, controllerAddr): + self.local.add_log("start CheckController function", "debug") + controllerData = self.GetControllerData(controllerAddr) + using_controllers = self.local.db.get("using_controllers", list()) + if controllerData is None: + raise Exception(f"CheckController error: controller not initialized. Use new_controllers") + if controllerData["approved"] != -1: + raise Exception(f"CheckController error: controller not approved: {controllerAddr}") + if controllerAddr not in using_controllers: + raise Exception("CheckController error: controller is not up to date. Use new_controllers") + #end define + + def GetControllers(self): + self.local.add_log("start GetControllers function", "debug") + controller0 = self.GetControllerAddress(controller_id=0) + controller1 = self.GetControllerAddress(controller_id=1) + controllers = [controller0, controller1] + return controllers + #end define + + def GetController(self, mode): + controllers = self.GetControllers() + for controllerAddr in controllers: + if mode == "stake" and self.IsControllerReadyToStake(controllerAddr): + return controllerAddr + if mode == "vote" and self.IsControllerReadyToVote(controllerAddr): + return controllerAddr + raise Exception("Validator controller not found or not ready") + #end define + + def GetControllerRequiredBalanceForLoan(self, controllerAddr, credit, interest): + cmd = f"runmethodfull {controllerAddr} required_balance_for_loan {credit} {interest}" + result = self.liteClient.Run(cmd) + data = self.Result2List(result) + if data is None: + return + min_amount = data[0] + validator_amount = data[1] + return min_amount, validator_amount + #end define + + def IsControllerReadyToStake(self, addrB64): + stop_controllers_list = self.local.db.get("stop_controllers_list") + if stop_controllers_list is not None and addrB64 in stop_controllers_list: + return False + now = get_timestamp() + config15 = self.GetConfig15() + controllerData = self.GetControllerData(addrB64) + if controllerData is None: + raise Exception(f"IsControllerReadyToStake error: controller not initialized. Use new_controllers") + lastSentStakeTime = controllerData["stake_at"] + stakeFreezeDelay = config15["validatorsElectedFor"] + config15["stakeHeldFor"] + result = lastSentStakeTime + stakeFreezeDelay < now + print(f"{addrB64}: {result}. {lastSentStakeTime}, {stakeFreezeDelay}, {now}") + return result + #end define + + def IsControllerReadyToVote(self, addrB64): + vwl = self.GetValidatorsWalletsList() + result = addrB64 in vwl + return result + #end define + + def GetControllerData(self, controllerAddr): + cmd = f"runmethodfull {controllerAddr} get_validator_controller_data" + result = self.liteClient.Run(cmd) + data = self.Result2List(result) + if data is None: + return + result_vars = ["state", "halted", "approved", "stake_amount_sent", "stake_at", "saved_validator_set_hash", "validator_set_changes_count", "validator_set_change_time", "stake_held_for", "borrowed_amount", "borrowing_time"] + controllerData = dict() + for name in result_vars: + controllerData[name] = data.pop(0) + return controllerData + #end define + + def CreateLoanRequest(self, controllerAddr): + self.local.add_log("start CreateLoanRequest function", "debug") + min_loan = self.local.db.get("min_loan", 41000) + max_loan = self.local.db.get("max_loan", 43000) + max_interest_percent = self.local.db.get("max_interest_percent", 1.5) + max_interest = int(max_interest_percent/100*16777216) + + # Проверить наличие действующего кредита + controllerData = self.GetControllerData(controllerAddr) + if controllerData["borrowed_amount"] > 0: + self.local.add_log("CreateLoanRequest warning: past loan found", "warning") + return + #end define + + # Проверить наличие средств у ликвидного пула + if self.CalculateLoanAmount(min_loan, max_loan, max_interest) == '-0x1': + raise Exception("CreateLoanRequest error: The liquid pool cannot issue the required amount of credit") + #end if + + # Проверить хватает ли ставки валидатора + min_amount, validator_amount = self.GetControllerRequiredBalanceForLoan(controllerAddr, max_loan, max_interest) + if min_amount > validator_amount: + raise Exception("CreateLoanRequest error: Validator stake is too low. Use deposit_to_controller") + #end if + + wallet = self.GetValidatorWallet() + fiftScript = self.contractsDir + "jetton_pool/fift-scripts/generate-loan-request.fif" + resultFilePath = self.tempDir + self.nodeName + wallet.name + "_loan_request.boc" + args = [fiftScript, min_loan, max_loan, max_interest, resultFilePath] + result = self.fift.Run(args) + resultFilePath = self.SignBocWithWallet(wallet, resultFilePath, controllerAddr, 1.01) + self.SendFile(resultFilePath, wallet) + self.WaitLoan(controllerAddr) + #end define + + def CalculateLoanAmount(self, min_loan, max_loan, max_interest): + data = dict() + data["address"] = self.GetLiquidPoolAddr() + data["method"] = "calculate_loan_amount" + data["stack"] = [ + ["num", min_loan*10**9], + ["num", max_loan*10**9], + ["num", max_interest], + ] + print(f"CalculateLoanAmount data: {data}") + + url = "http://127.0.0.1:8801/runGetMethod" + res = requests.post(url, json=data) + res_data = res.json() + if res_data.get("ok") is False: + error = res_data.get("error") + raise Exception(error) + result = res_data.get("result").get("stack").pop().pop() + return result + #end define + + def CalculateLoanAmount_test(self): + min_loan = self.local.db.get("min_loan", 41000) + max_loan = self.local.db.get("max_loan", 43000) + max_interest_percent = self.local.db.get("max_interest_percent", 1.5) + max_interest = int(max_interest_percent/100*16777216) + return self.CalculateLoanAmount(min_loan, max_loan, max_interest) + #end define + + def WaitLoan(self, controllerAddr): + self.local.add_log("start WaitLoan function", "debug") + for i in range(10): + time.sleep(3) + controllerData = self.GetControllerData(controllerAddr) + if controllerData["borrowed_amount"] != 0: + return + raise Exception("WaitLoan error: time out") + #end define + + def ReturnUnusedLoan(self, controllerAddr): + self.local.add_log("start ReturnUnusedLoan function", "debug") + wallet = self.GetValidatorWallet() + fileName = self.contractsDir + "jetton_pool/fift-scripts/return_unused_loan.boc" + resultFilePath = self.SignBocWithWallet(wallet, fileName, controllerAddr, 1.05) + self.SendFile(resultFilePath, wallet) + #end define + + def DepositToController(self, controllerAddr, amount): + self.local.add_log("start DepositToController function", "debug") + wallet = self.GetValidatorWallet() + fileName = self.contractsDir + "jetton_pool/fift-scripts/top-up.boc" + resultFilePath = self.SignBocWithWallet(wallet, fileName, controllerAddr, amount) + self.SendFile(resultFilePath, wallet) + #end define + + def WithdrawFromController(self, controllerAddr, amount=None): + controllerData = self.GetControllerData(controllerAddr) + if controllerData["state"] == 0: + self.WithdrawFromControllerProcess(controllerAddr, amount) + else: + self.PendWithdrawFromController(controllerAddr, amount) + #end define + + def WithdrawFromControllerProcess(self, controllerAddr, amount): + if amount is None: + account = self.GetAccount(controllerAddr) + amount = account.balance-10.1 + if int(amount) < 3: + return + #end if + + self.local.add_log("start WithdrawFromControllerProcess function", "debug") + wallet = self.GetValidatorWallet() + fiftScript = self.contractsDir + "jetton_pool/fift-scripts/withdraw-controller.fif" + resultFilePath = self.tempDir + self.nodeName + wallet.name + "_withdraw_request.boc" + args = [fiftScript, amount, resultFilePath] + result = self.fift.Run(args) + resultFilePath = self.SignBocWithWallet(wallet, resultFilePath, controllerAddr, 1.06) + self.SendFile(resultFilePath, wallet) + #end define + + def PendWithdrawFromController(self, controllerAddr, amount): + self.local.add_log("start PendWithdrawFromController function", "debug") + controllerPendingWithdraws = self.GetControllerPendingWithdraws() + controllerPendingWithdraws[controllerAddr] = amount + self.local.save() + #end define + + def HandleControllerPendingWithdraw(self, controllerPendingWithdraws, controllerAddr): + amount = controllerPendingWithdraws.get(controllerAddr) + self.WithdrawFromControllerProcess(controllerAddr, amount) + controllerPendingWithdraws.pop(controllerAddr) + #end define + + def GetControllerPendingWithdraws(self): + bname = "controllerPendingWithdraws" + controllerPendingWithdraws = self.local.db.get(bname) + if controllerPendingWithdraws is None: + controllerPendingWithdraws = dict() + self.local.db[bname] = controllerPendingWithdraws + return controllerPendingWithdraws + #end define + + def SignElectionRequestWithController(self, controllerAddr, startWorkTime, adnlAddr, validatorPubkey_b64, validatorSignature, maxFactor, stake): + self.local.add_log("start SignElectionRequestWithController function", "debug") + fileName = self.tempDir + str(startWorkTime) + "_validator-query.boc" + fiftScript = self.contractsDir + "jetton_pool/fift-scripts/controller-elect-signed.fif" + args = [fiftScript, controllerAddr, startWorkTime, maxFactor, adnlAddr, validatorPubkey_b64, validatorSignature, fileName, stake] + self.local.add_log(f"SignElectionRequestWithController args: {args}", "debug") + result = self.fift.Run(args) + self.local.add_log(f"SignElectionRequestWithController result: {result}", "debug") + pubkey = parse(result, "validator public key ", '\n') + fileName = parse(result, "Saved to file ", '\n') + return pubkey, fileName + #end define + + def ControllersUpdateValidatorSet(self): + self.local.add_log("start ControllersUpdateValidatorSet function", "debug") + using_controllers = self.local.db.get("using_controllers") + user_controllers = self.local.db.get("user_controllers", list()) + old_controllers = self.local.db.get("old_controllers", list()) + for controller in using_controllers + user_controllers + old_controllers: + self.ControllerUpdateValidatorSet(controller) + #end define + + def ControllerUpdateValidatorSet(self, controllerAddr): + self.local.add_log("start ControllerUpdateValidatorSet function", "debug") + wallet = self.GetValidatorWallet() + controllers = self.GetControllers() + controllerData = self.GetControllerData(controllerAddr) + if controllerData is None: + return + #end if + + timeNow = int(time.time()) + config34 = self.GetConfig34() + fullElectorAddr = self.GetFullElectorAddr() + returnedStake = self.GetReturnedStake(fullElectorAddr, controllerAddr) + controllerPendingWithdraws = self.GetControllerPendingWithdraws() + if (controllerData["state"] == 3 and + controllerData["validator_set_changes_count"] < 2 and + controllerData["validator_set_change_time"] < config34["startWorkTime"]): + self.ControllerUpdateValidatorSetProcess(controllerAddr, wallet) + controllerData = self.GetControllerData(controllerAddr) + if (returnedStake > 0 and + controllerData["state"] == 3 and + controllerData["validator_set_changes_count"] >= 2 and + timeNow - controllerData["validator_set_change_time"] > controllerData["stake_held_for"] + 60): + self.ControllerRecoverStake(controllerAddr) + controllerData = self.GetControllerData(controllerAddr) + if (controllerData["state"] == 0 and + controllerData["borrowed_amount"] > 0 and + config34["startWorkTime"] > controllerData["borrowing_time"]): + self.ReturnUnusedLoan(controllerAddr) + if (controllerData["state"] == 0 and controllerAddr in controllerPendingWithdraws): + self.HandleControllerPendingWithdraw(controllerPendingWithdraws, controllerAddr) + if controllerAddr not in controllers: + self.WithdrawFromController(controllerAddr) + #end define + + def ControllerUpdateValidatorSetProcess(self, controllerAddr, wallet): + self.local.add_log("start ControllerUpdateValidatorSetProcess function", "debug") + fileName = self.contractsDir + "jetton_pool/fift-scripts/update_validator_hash.boc" + resultFilePath = self.SignBocWithWallet(wallet, fileName, controllerAddr, 1.07) + self.SendFile(resultFilePath, wallet) + self.local.add_log("ControllerUpdateValidatorSetProcess completed") + #end define + + def ControllerRecoverStake(self, controllerAddr): + wallet = self.GetValidatorWallet() + self.local.add_log("start ControllerRecoverStake function", "debug") + fileName = self.contractsDir + "jetton_pool/fift-scripts/recover_stake.boc" + resultFilePath = self.SignBocWithWallet(wallet, fileName, controllerAddr, 1.04) + self.SendFile(resultFilePath, wallet) + self.local.add_log("ControllerRecoverStake completed") + #end define + + def StopController(self, controllerAddr): + stop_controllers_list = self.local.db.get("stop_controllers_list") + if stop_controllers_list is None: + stop_controllers_list = list() + if controllerAddr not in stop_controllers_list: + stop_controllers_list.append(controllerAddr) + self.local.db["stop_controllers_list"] = stop_controllers_list + + user_controllers = self.local.db.get("user_controllers") + if user_controllers is not None and controllerAddr in user_controllers: + user_controllers.remove(controllerAddr) + self.local.save() + #end define + + def AddController(self, controllerAddr): + user_controllers = self.local.db.get("user_controllers") + if user_controllers is None: + user_controllers = list() + if controllerAddr not in user_controllers: + user_controllers.append(controllerAddr) + self.local.db["user_controllers"] = user_controllers + + stop_controllers_list = self.local.db.get("stop_controllers_list") + if stop_controllers_list is not None and controllerAddr in stop_controllers_list: + stop_controllers_list.remove(controllerAddr) + self.local.save() + #end define + + def CheckLiquidPool(self): + liquid_pool_addr = self.GetLiquidPoolAddr() + account = self.GetAccount(liquid_pool_addr) + history = self.GetAccountHistory(account, 5000) + addrs_list = list() + for message in history: + if message.srcAddr is None or message.value is None: + continue + srcAddrFull = f"{message.srcWorkchain}:{message.srcAddr}" + destAddFull = f"{message.destWorkchain}:{message.destAddr}" + if srcAddrFull == account.addrFull: + fromto = destAddFull + else: + fromto = srcAddrFull + fromto = self.AddrFull2AddrB64(fromto) + if fromto not in addrs_list: + addrs_list.append(fromto) + #end for + + for controllerAddr in addrs_list: + account = self.GetAccount(controllerAddr) + version = self.GetWalletVersionFromHash(account.codeHash) + if version is None or "controller" not in version: + continue + print(f"check controller: {controllerAddr}") + self.ControllerUpdateValidatorSet(controllerAddr) + #end define + def create_single_pool(self, pool_name, owner_address): self.local.add_log("start create_single_pool function", "debug") @@ -3769,3 +4200,8 @@ def Dec2HexAddr(dec): h64 = hu.rjust(64, "0") return h64 #end define + +def HexAddr2Dec(h): + d = int(h, 16) + return d +#end define diff --git a/mytonctrl/mytonctrl.py b/mytonctrl/mytonctrl.py index b4a7768a..2b47e6a6 100755 --- a/mytonctrl/mytonctrl.py +++ b/mytonctrl/mytonctrl.py @@ -138,6 +138,20 @@ def inject_globals(func): console.AddItem("activate_single_pool", inject_globals(activate_single_pool), local.translate("activate_single_pool_cmd")) console.AddItem("import_pool", inject_globals(import_pool), local.translate("import_pool_cmd")) + console.AddItem("create_controllers", inject_globals(CreateControllers), local.translate("_")) + console.AddItem("update_controllers", inject_globals(CreateControllers), local.translate("_")) + console.AddItem("controllers_list", inject_globals(PrintControllersList), local.translate("_")) + console.AddItem("get_controller_data", inject_globals(GetControllerData), local.translate("_")) + console.AddItem("deposit_to_controller", inject_globals(DepositToController), local.translate("_")) + console.AddItem("withdraw_from_controller", inject_globals(WithdrawFromController), local.translate("_")) + console.AddItem("calculate_annual_controller_percentage", inject_globals(CalculateAnnualControllerPercentage), local.translate("_")) + console.AddItem("controller_update_validator_set", inject_globals(ControllerUpdateValidatorSet), local.translate("_")) + console.AddItem("stop_controller", inject_globals(StopController), local.translate("_")) + console.AddItem("stop_and_withdraw_controller", inject_globals(StopAndWithdrawController), local.translate("_")) + console.AddItem("add_controller", inject_globals(AddController), local.translate("_")) + console.AddItem("check_liquid_pool", inject_globals(CheckLiquidPool), local.translate("_")) + console.AddItem("test_calculate_loan_amount", inject_globals(CalculateLoanAmount_test), local.translate("_")) + # console.AddItem("pt", inject_globals(PrintTest), "PrintTest") # console.AddItem("sl", inject_globals(sl), "sl") console.AddItem("cleanup", inject_globals(cleanup_validator_db), local.translate("cleanup_cmd")) @@ -1489,6 +1503,145 @@ def import_pool(ton, args): color_print("import_pool - {green}OK{endc}") #end define +def CreateControllers(ton, args): + ton.CreateControllers() + color_print("CreateControllers - {green}OK{endc}") +#end define + +def PrintControllersList(ton, args): + new_controllers = ton.GetControllers() + using_controllers = ton.GetSettings("using_controllers") + old_controllers = ton.GetSettings("old_controllers") + user_controllers_list = ton.GetSettings("user_controllers_list") + print("using controllers:") + PrintControllersListProcess(ton, using_controllers) + if new_controllers != using_controllers: + print() + print("new controllers:") + PrintControllersListProcess(ton, new_controllers) + if old_controllers is not None and len(old_controllers) > 0: + print() + print("old controllers:") + PrintControllersListProcess(ton, old_controllers) + if user_controllers_list is not None and len(user_controllers_list) > 0: + print() + print("user controllers:") + PrintControllersListProcess(ton, user_controllers_list) +#end define + +def PrintControllersListProcess(ton, controllers): + table = list() + table += [["Address", "Status", "Balance", "Approved", "State"]] + for controllerAddr in controllers: + account = ton.GetAccount(controllerAddr) + controllerData = ton.GetControllerData(controllerAddr) + approved = True if controllerData and controllerData["approved"] == -1 else False + state = controllerData["state"] if controllerData else None + table += [[controllerAddr, account.status, account.balance, approved, state]] + print_table(table) +#end define + +def GetControllerData(ton, args): + try: + controllerAddr = args[0] + except: + color_print("{red}Bad args. Usage:{endc} get_controller_data ") + return + controllerData = ton.GetControllerData(controllerAddr) + print(json.dumps(controllerData, indent=4)) +#end define + +def DepositToController(ton, args): + try: + controllerAddr = args[0] + amount = float(args[1]) + except: + color_print("{red}Bad args. Usage:{endc} deposit_to_controller ") + return + ton.DepositToController(controllerAddr, amount) +#end define + +def WithdrawFromController(ton, args): + try: + controllerAddr = args[0] + amount = GetItemFromList(args, 1) + except: + color_print("{red}Bad args. Usage:{endc} withdraw_from_controller [amount]") + return + ton.WithdrawFromController(controllerAddr, amount) +#end define + +def CalculateAnnualControllerPercentage(ton, args): + try: + percentPerRound = float(args[0]) + except: + percentPerRound = ton.GetSettings("max_interest_percent") + config15 = ton.GetConfig(15) + roundPeriod = config15["validators_elected_for"] + rounds = 365 * 24 * 3600 / roundPeriod + yearInterest = (1 + percentPerRound / 100) * rounds + yearInterestPercent = round(yearInterest / 100, 2) + print("roundPeriod", roundPeriod) + print("rounds", rounds) + print("percentPerRound", percentPerRound) + print("yearInterest", yearInterest) + print(f"yearInterestPercent: {yearInterestPercent}%") +#end define + +def ControllerUpdateValidatorSet(ton, args): + try: + controllerAddr = args[0] + except: + color_print("{red}Bad args. Usage:{endc} controller_update_validator_set ") + return + ton.ControllerUpdateValidatorSet(controllerAddr) + color_print("ControllerUpdateValidatorSet - {green}OK{endc}") +#end define + +def StopController(ton, args): + try: + controllerAddr = args[0] + except: + color_print("{red}Bad args. Usage:{endc} stop_controller ") + return + ton.StopController(controllerAddr) + color_print("StopController - {green}OK{endc}") +#end define + +def StopAndWithdrawController(ton, args): + try: + controllerAddr = args[0] + amount = GetItemFromList(args, 1) + except: + color_print("{red}Bad args. Usage:{endc} stop_and_withdraw_controller [amount]") + return + if amount is None: + account = ton.GetAccount(controllerAddr) + amount = account.balance-10.1 + ton.StopController(controllerAddr) + ton.WithdrawFromController(controllerAddr, amount) + color_print("StopAndWithdrawController - {green}OK{endc}") +#end define + +def AddController(ton, args): + try: + controllerAddr = args[0] + except: + color_print("{red}Bad args. Usage:{endc} add_controller ") + return + ton.AddController(controllerAddr) + color_print("AddController - {green}OK{endc}") +#end define + +def CheckLiquidPool(ton, args): + ton.CheckLiquidPool() + color_print("CheckLiquidPool - {green}OK{endc}") +#end define + +def CalculateLoanAmount_test(ton, args): + t = ton.CalculateLoanAmount_test() + print(t) + ### ### Start of the program diff --git a/mytonctrl/resources/translate.json b/mytonctrl/resources/translate.json index 5ce41b98..60350335 100644 --- a/mytonctrl/resources/translate.json +++ b/mytonctrl/resources/translate.json @@ -239,6 +239,11 @@ "ru": "Эффективность валидатора: {0}", "zh_TW": "驗證者效率: {0}" }, + "local_status_validator_efficiency_ver2": { + "en": "Validator efficiency, ver.2: {0}", + "ru": "Эффективность валидатора, вер.2: {0}", + "zh_TW": "驗證器效率,版本2: {0}" + }, "local_status_adnl_addr": { "en": "ADNL address of local validator: {0}", "ru": "ADNL адрес локального валидатора: {0}", diff --git a/mytoninstaller.py b/mytoninstaller.py new file mode 100644 index 00000000..f3726178 --- /dev/null +++ b/mytoninstaller.py @@ -0,0 +1,1051 @@ +#!/usr/bin/env python3 +# -*- coding: utf_8 -*- + +import pwd +import random +import requests +from mypylib.mypylib import * +from mypyconsole.mypyconsole import * + +local = MyPyClass(__file__) +console = MyPyConsole() +defaultLocalConfigPath = "/usr/bin/ton/local.config.json" + + +def Init(): + local.db.config.isStartOnlyOneProcess = False + local.db.config.logLevel = "debug" + local.db.config.isIgnorLogWarning = True # disable warning + local.run() + local.db.config.isIgnorLogWarning = False # enable warning + + + # create variables + user = os.environ.get("USER", "root") + local.buffer.user = user + local.buffer.vuser = "validator" + local.buffer.cport = random.randint(2000, 65000) + local.buffer.lport = random.randint(2000, 65000) + + # Create user console + console.name = "MyTonInstaller" + console.color = console.RED + console.AddItem("status", Status, "Print TON component status") + console.AddItem("enable", Enable, "Enable some function: 'FN' - Full node, 'VC' - Validator console, 'LS' - Liteserver, 'DS' - DHT-Server, 'JR' - jsonrpc, 'THA' - ton-http-api. Example: 'enable FN'") + console.AddItem("update", Enable, "Update some function: 'JR' - jsonrpc. Example: 'update JR'") + console.AddItem("plsc", PrintLiteServerConfig, "Print LiteServer config") + console.AddItem("clcf", CreateLocalConfigFile, "CreateLocalConfigFile") + console.AddItem("drvcf", DRVCF, "Dangerous recovery validator config file") + console.AddItem("setwebpass", SetWebPassword, "Set a password for the web admin interface") + + Refresh() +#end define + +def Refresh(): + user = local.buffer.user + local.buffer.mconfig_path = "/home/{user}/.local/share/mytoncore/mytoncore.db".format(user=user) + if user == 'root': + local.buffer.mconfig_path = "/usr/local/bin/mytoncore/mytoncore.db" + #end if + + # create variables + bin_dir = "/usr/bin/" + src_dir = "/usr/src/" + ton_work_dir = "/var/ton-work/" + ton_bin_dir = bin_dir + "ton/" + ton_src_dir = src_dir + "ton/" + local.buffer.bin_dir = bin_dir + local.buffer.src_dir = src_dir + local.buffer.ton_work_dir = ton_work_dir + local.buffer.ton_bin_dir = ton_bin_dir + local.buffer.ton_src_dir = ton_src_dir + ton_db_dir = ton_work_dir + "db/" + keys_dir = ton_work_dir + "keys/" + local.buffer.ton_db_dir = ton_db_dir + local.buffer.keys_dir = keys_dir + local.buffer.ton_log_path = ton_work_dir + "log" + local.buffer.validator_app_path = ton_bin_dir + "validator-engine/validator-engine" + local.buffer.global_config_path = ton_bin_dir + "global.config.json" + local.buffer.vconfig_path = ton_db_dir + "config.json" +#end define + +def Status(args): + keys_dir = local.buffer.keys_dir + server_key = keys_dir + "server" + client_key = keys_dir + "client" + liteserver_key = keys_dir + "liteserver" + liteserver_pubkey = liteserver_key + ".pub" + + + fnStatus = os.path.isfile(local.buffer.vconfig_path) + mtcStatus = os.path.isfile(local.buffer.mconfig_path) + vcStatus = os.path.isfile(server_key) or os.path.isfile(client_key) + lsStatus = os.path.isfile(liteserver_pubkey) + + print("Full node status:", fnStatus) + print("Mytoncore status:", mtcStatus) + print("V.console status:", vcStatus) + print("Liteserver status:", lsStatus) +#end define + +def Enable(args): + name = args[0] + if name == "THA": + CreateLocalConfigFile(args, localhost=True) + args = ["python3", local.buffer.my_path, "-u", local.buffer.user, "-e", "enable{name}".format(name=name)] + run_as_root(args) +#end define + +def DRVCF(args): + args = ["python3", local.buffer.my_path, "-u", local.buffer.user, "-e", "drvcf"] + run_as_root(args) +#end define + +def get_own_ip(): + requests.packages.urllib3.util.connection.HAS_IPV6 = False + ip = requests.get("https://ifconfig.me/ip").text + return ip +#end define + +def GetLiteServerConfig(): + keys_dir = local.buffer.keys_dir + liteserver_key = keys_dir + "liteserver" + liteserver_pubkey = liteserver_key + ".pub" + result = Dict() + file = open(liteserver_pubkey, 'rb') + data = file.read() + file.close() + key = base64.b64encode(data[4:]) + ip = get_own_ip() + mconfig = GetConfig(path=local.buffer.mconfig_path) + result.ip = ip2int(ip) + result.port = mconfig.liteClient.liteServer.port + result.id = Dict() + result.id["@type"]= "pub.ed25519" + result.id.key= key.decode() + return result +#end define + +def GetInitBlock(): + from mytoncore import MyTonCore + ton = MyTonCore() + initBlock = ton.GetInitBlock() + return initBlock +#end define + +def CreateLocalConfig(initBlock, localConfigPath=defaultLocalConfigPath, localhost=False): + # dirty hack, but GetInitBlock() function uses the same technique + from mytoncore import hex2base64 + + # read global config file + file = open("/usr/bin/ton/global.config.json", 'rt') + text = file.read() + data = json.loads(text) + file.close() + + # edit config + liteServerConfig = GetLiteServerConfig() + if localhost is True: + liteServerConfig["ip"] = 2130706433 # 127.0.0.1 + localConfigPath = localConfigPath.replace("local", "localhost") + data["liteservers"] = [liteServerConfig] + data["validator"]["init_block"]["seqno"] = initBlock["seqno"] + data["validator"]["init_block"]["root_hash"] = hex2base64(initBlock["rootHash"]) + data["validator"]["init_block"]["file_hash"] = hex2base64(initBlock["fileHash"]) + text = json.dumps(data, indent=4) + + # write local config file + file = open(localConfigPath, 'wt') + file.write(text) + file.close() + + # chown + user = local.buffer.user + args = ["chown", "-R", user + ':' + user, localConfigPath] + + print("Local config file created:", localConfigPath) +#end define + +def PrintLiteServerConfig(args): + liteServerConfig = GetLiteServerConfig() + text = json.dumps(liteServerConfig, indent=4) + print(text) +#end define + +def CreateLocalConfigFile(args, localhost=False): + if localhost is True: + event_name = "clcl" + else: + event_name = "clc" + initBlock = GetInitBlock() + initBlock_b64 = dict2b64(initBlock) + args = ["python3", local.buffer.my_path, "-u", local.buffer.user, "-e", event_name, "-i", initBlock_b64] + run_as_root(args) +#end define + +def Event(name): + if name == "enableFN": + FirstNodeSettings() + if name == "enableVC": + EnableValidatorConsole() + if name == "enableLS": + EnableLiteServer() + if name == "enableDS": + EnableDhtServer() + if name == "drvcf": + DangerousRecoveryValidatorConfigFile() + if name == "enableJR": + EnableJsonRpc() + if name == "enableTHA": + EnableTonHttpApi() + if name in ["clc", "clcl"]: + ix = sys.argv.index("-i") + initBlock_b64 = sys.argv[ix+1] + initBlock = b642dict(initBlock_b64) + if name == "clcl": + CreateLocalConfig(initBlock, localhost=True) + else: + CreateLocalConfig(initBlock) +#end define + +def General(): + if "-u" in sys.argv: + ux = sys.argv.index("-u") + user = sys.argv[ux+1] + local.buffer.user = user + Refresh() + if "-e" in sys.argv: + ex = sys.argv.index("-e") + name = sys.argv[ex+1] + Event(name) + if "-m" in sys.argv: + mx = sys.argv.index("-m") + mode = sys.argv[mx+1] + if "-t" in sys.argv: + mx = sys.argv.index("-t") + telemetry = sys.argv[mx+1] + local.buffer.telemetry = Str2Bool(telemetry) + if "--dump" in sys.argv: + mx = sys.argv.index("--dump") + dump = sys.argv[mx+1] + local.buffer.dump = Str2Bool(dump) + #end if + + # Создать настройки для mytoncore.py + FirstMytoncoreSettings() + + if mode == "full": + FirstNodeSettings() + EnableValidatorConsole() + EnableLiteServer() + BackupVconfig() + BackupMconfig() + #end if + + # Создать символические ссылки + CreateSymlinks() + #end if +#end define + +def Str2Bool(str): + if str == "true": + return True + return False +#end define + +def FirstNodeSettings(): + local.add_log("start FirstNodeSettings fuction", "debug") + + # Создать переменные + user = local.buffer.user + vuser = local.buffer.vuser + ton_work_dir = local.buffer.ton_work_dir + ton_db_dir = local.buffer.ton_db_dir + keys_dir = local.buffer.keys_dir + tonLogPath = local.buffer.ton_log_path + validatorAppPath = local.buffer.validator_app_path + globalConfigPath = local.buffer.global_config_path + vconfig_path = local.buffer.vconfig_path + + # Проверить конфигурацию + if os.path.isfile(vconfig_path): + local.add_log("Validators config.json already exist. Break FirstNodeSettings fuction", "warning") + return + #end if + + # Создать пользователя + file = open("/etc/passwd", 'rt') + text = file.read() + file.close() + if vuser not in text: + local.add_log("Creating new user: " + vuser, "debug") + args = ["/usr/sbin/useradd", "-d", "/dev/null", "-s", "/dev/null", vuser] + subprocess.run(args) + #end if + + # Подготовить папки валидатора + os.makedirs(ton_db_dir, exist_ok=True) + os.makedirs(keys_dir, exist_ok=True) + + # Прописать автозагрузку + cpus = psutil.cpu_count() - 1 + cmd = "{validatorAppPath} --threads {cpus} --daemonize --global-config {globalConfigPath} --db {ton_db_dir} --logname {tonLogPath} --state-ttl 604800 --verbosity 1" + cmd = cmd.format(validatorAppPath=validatorAppPath, globalConfigPath=globalConfigPath, ton_db_dir=ton_db_dir, tonLogPath=tonLogPath, cpus=cpus) + add2systemd(name="validator", user=vuser, start=cmd) # post="/usr/bin/python3 /usr/src/mytonctrl/mytoncore.py -e \"validator down\"" + + # Получить внешний ip адрес + ip = get_own_ip() + vport = random.randint(2000, 65000) + addr = "{ip}:{vport}".format(ip=ip, vport=vport) + local.add_log("Use addr: " + addr, "debug") + + # Первый запуск + local.add_log("First start validator - create config.json", "debug") + args = [validatorAppPath, "--global-config", globalConfigPath, "--db", ton_db_dir, "--ip", addr, "--logname", tonLogPath] + subprocess.run(args) + + # Скачать дамп + DownloadDump() + + # chown 1 + local.add_log("Chown ton-work dir", "debug") + args = ["chown", "-R", vuser + ':' + vuser, ton_work_dir] + subprocess.run(args) + + # start validator + StartValidator() +#end define + +def DownloadDump(): + dump = local.buffer.dump + if dump == False: + return + #end if + + local.add_log("start DownloadDump fuction", "debug") + url = "https://dump.ton.org" + dumpSize = requests.get(url + "/dumps/latest.size.archive.txt").text + print("dumpSize:", dumpSize) + needSpace = int(dumpSize) * 3 + diskSpace = psutil.disk_usage("/var") + if needSpace > diskSpace.free: + return + #end if + + # apt install + cmd = "apt install plzip pv -y" + os.system(cmd) + + # download dump + cmd = "curl -s {url}/dumps/latest.tar.lz | pv | plzip -d -n8 | tar -xC /var/ton-work/db".format(url=url) + os.system(cmd) +#end define + +def FirstMytoncoreSettings(): + local.add_log("start FirstMytoncoreSettings fuction", "debug") + user = local.buffer.user + + # Прописать mytoncore.py в автозагрузку + add2systemd(name="mytoncore", user=user, start="/usr/bin/python3 /usr/src/mytonctrl/mytoncore.py") + + # Проверить конфигурацию + path = "/home/{user}/.local/share/mytoncore/mytoncore.db".format(user=user) + path2 = "/usr/local/bin/mytoncore/mytoncore.db" + if os.path.isfile(path) or os.path.isfile(path2): + local.add_log("mytoncore.db already exist. Break FirstMytoncoreSettings fuction", "warning") + return + #end if + + #amazon bugfix + path1 = "/home/{user}/.local/".format(user=user) + path2 = path1 + "share/" + chownOwner = "{user}:{user}".format(user=user) + os.makedirs(path1, exist_ok=True) + os.makedirs(path2, exist_ok=True) + args = ["chown", chownOwner, path1, path2] + subprocess.run(args) + + # Подготовить папку mytoncore + mconfig_path = local.buffer.mconfig_path + mconfigDir = get_dir_from_path(mconfig_path) + os.makedirs(mconfigDir, exist_ok=True) + + # create variables + src_dir = local.buffer.src_dir + ton_bin_dir = local.buffer.ton_bin_dir + ton_src_dir = local.buffer.ton_src_dir + + # general config + mconfig = Dict() + mconfig.config = Dict() + mconfig.config.logLevel = "debug" + mconfig.config.isLocaldbSaving = True + + # fift + fift = Dict() + fift.appPath = ton_bin_dir + "crypto/fift" + fift.libsPath = ton_src_dir + "crypto/fift/lib" + fift.smartcontsPath = ton_src_dir + "crypto/smartcont" + mconfig.fift = fift + + # lite-client + liteClient = Dict() + liteClient.appPath = ton_bin_dir + "lite-client/lite-client" + liteClient.configPath = ton_bin_dir + "global.config.json" + mconfig.liteClient = liteClient + + # Telemetry + mconfig.sendTelemetry = local.buffer.telemetry + + # Записать настройки в файл + SetConfig(path=mconfig_path, data=mconfig) + + # chown 1 + args = ["chown", user + ':' + user, mconfigDir, mconfig_path] + subprocess.run(args) + + # start mytoncore + StartMytoncore() +#end define + +def EnableValidatorConsole(): + local.add_log("start EnableValidatorConsole function", "debug") + + # Create variables + user = local.buffer.user + vuser = local.buffer.vuser + cport = local.buffer.cport + src_dir = local.buffer.src_dir + ton_db_dir = local.buffer.ton_db_dir + ton_bin_dir = local.buffer.ton_bin_dir + vconfig_path = local.buffer.vconfig_path + generate_random_id = ton_bin_dir + "utils/generate-random-id" + keys_dir = local.buffer.keys_dir + client_key = keys_dir + "client" + server_key = keys_dir + "server" + client_pubkey = client_key + ".pub" + server_pubkey = server_key + ".pub" + + # Check if key exist + if os.path.isfile(server_key) or os.path.isfile(client_key): + local.add_log("Server or client key already exist. Break EnableValidatorConsole fuction", "warning") + return + #end if + + # generate server key + args = [generate_random_id, "--mode", "keys", "--name", server_key] + process = subprocess.run(args, stdout=subprocess.PIPE) + output = process.stdout.decode("utf-8") + output_arr = output.split(' ') + server_key_hex = output_arr[0] + server_key_b64 = output_arr[1].replace('\n', '') + + # move key + newKeyPath = ton_db_dir + "/keyring/" + server_key_hex + args = ["mv", server_key, newKeyPath] + subprocess.run(args) + + # generate client key + args = [generate_random_id, "--mode", "keys", "--name", client_key] + process = subprocess.run(args, stdout=subprocess.PIPE) + output = process.stdout.decode("utf-8") + output_arr = output.split(' ') + client_key_hex = output_arr[0] + client_key_b64 = output_arr[1].replace('\n', '') + + # chown 1 + args = ["chown", vuser + ':' + vuser, newKeyPath] + subprocess.run(args) + + # chown 2 + args = ["chown", user + ':' + user, server_pubkey, client_key, client_pubkey] + subprocess.run(args) + + # read vconfig + vconfig = GetConfig(path=vconfig_path) + + # prepare config + control = Dict() + control.id = server_key_b64 + control.port = cport + allowed = Dict() + allowed.id = client_key_b64 + allowed.permissions = 15 + control.allowed = [allowed] # fix me + vconfig.control.append(control) + + # write vconfig + SetConfig(path=vconfig_path, data=vconfig) + + # restart validator + StartValidator() + + # read mconfig + mconfig_path = local.buffer.mconfig_path + mconfig = GetConfig(path=mconfig_path) + + # edit mytoncore config file + validatorConsole = Dict() + validatorConsole.appPath = ton_bin_dir + "validator-engine-console/validator-engine-console" + validatorConsole.privKeyPath = client_key + validatorConsole.pubKeyPath = server_pubkey + validatorConsole.addr = "127.0.0.1:{cport}".format(cport=cport) + mconfig.validatorConsole = validatorConsole + + # write mconfig + SetConfig(path=mconfig_path, data=mconfig) + + # Подтянуть событие в mytoncore.py + cmd = "python3 {src_dir}mytonctrl/mytoncore.py -e \"enableVC\"".format(src_dir=src_dir) + args = ["su", "-l", user, "-c", cmd] + subprocess.run(args) + + # restart mytoncore + StartMytoncore() +#end define + +def EnableLiteServer(): + local.add_log("start EnableLiteServer function", "debug") + + # Create variables + user = local.buffer.user + vuser = local.buffer.vuser + lport = local.buffer.lport + src_dir = local.buffer.src_dir + ton_db_dir = local.buffer.ton_db_dir + keys_dir = local.buffer.keys_dir + ton_bin_dir = local.buffer.ton_bin_dir + vconfig_path = local.buffer.vconfig_path + generate_random_id = ton_bin_dir + "utils/generate-random-id" + liteserver_key = keys_dir + "liteserver" + liteserver_pubkey = liteserver_key + ".pub" + + # Check if key exist + if os.path.isfile(liteserver_pubkey): + local.add_log("Liteserver key already exist. Break EnableLiteServer fuction", "warning") + return + #end if + + # generate liteserver key + local.add_log("generate liteserver key", "debug") + args = [generate_random_id, "--mode", "keys", "--name", liteserver_key] + process = subprocess.run(args, stdout=subprocess.PIPE) + output = process.stdout.decode("utf-8") + output_arr = output.split(' ') + liteserver_key_hex = output_arr[0] + liteserver_key_b64 = output_arr[1].replace('\n', '') + + # move key + local.add_log("move key", "debug") + newKeyPath = ton_db_dir + "/keyring/" + liteserver_key_hex + args = ["mv", liteserver_key, newKeyPath] + subprocess.run(args) + + # chown 1 + local.add_log("chown 1", "debug") + args = ["chown", vuser + ':' + vuser, newKeyPath] + subprocess.run(args) + + # chown 2 + local.add_log("chown 2", "debug") + args = ["chown", user + ':' + user, liteserver_pubkey] + subprocess.run(args) + + # read vconfig + local.add_log("read vconfig", "debug") + vconfig = GetConfig(path=vconfig_path) + + # prepare vconfig + local.add_log("prepare vconfig", "debug") + liteserver = Dict() + liteserver.id = liteserver_key_b64 + liteserver.port = lport + vconfig.liteservers.append(liteserver) + + # write vconfig + local.add_log("write vconfig", "debug") + SetConfig(path=vconfig_path, data=vconfig) + + # restart validator + StartValidator() + + # edit mytoncore config file + # read mconfig + local.add_log("read mconfig", "debug") + mconfig_path = local.buffer.mconfig_path + mconfig = GetConfig(path=mconfig_path) + + # edit mytoncore config file + local.add_log("edit mytoncore config file", "debug") + liteServer = Dict() + liteServer.pubkeyPath = liteserver_pubkey + liteServer.ip = "127.0.0.1" + liteServer.port = lport + mconfig.liteClient.liteServer = liteServer + + # write mconfig + local.add_log("write mconfig", "debug") + SetConfig(path=mconfig_path, data=mconfig) + + # restart mytoncore + StartMytoncore() +#end define + +def StartValidator(): + # restart validator + local.add_log("Start/restart validator service", "debug") + args = ["systemctl", "restart", "validator"] + subprocess.run(args) + + # sleep 10 sec + local.add_log("sleep 10 sec", "debug") + time.sleep(10) +#end define + +def StartMytoncore(): + # restart mytoncore + local.add_log("Start/restart mytoncore service", "debug") + args = ["systemctl", "restart", "mytoncore"] + subprocess.run(args) +#end define + +def GetConfig(**kwargs): + path = kwargs.get("path") + file = open(path, 'rt') + text = file.read() + file.close() + config = Dict(json.loads(text)) + return config +#end define + +def SetConfig(**kwargs): + path = kwargs.get("path") + data = kwargs.get("data") + + # write config + text = json.dumps(data, indent=4) + file = open(path, 'wt') + file.write(text) + file.close() +#end define + +def BackupVconfig(): + local.add_log("Backup validator config file 'config.json' to 'config.json.backup'", "debug") + vconfig_path = local.buffer.vconfig_path + backupPath = vconfig_path + ".backup" + args = ["cp", vconfig_path, backupPath] + subprocess.run(args) +#end define + +def BackupMconfig(): + local.add_log("Backup mytoncore config file 'mytoncore.db' to 'mytoncore.db.backup'", "debug") + mconfig_path = local.buffer.mconfig_path + backupPath = mconfig_path + ".backup" + args = ["cp", mconfig_path, backupPath] + subprocess.run(args) +#end define + +def GetPortsFromVconfig(): + vconfig_path = local.buffer.vconfig_path + + # read vconfig + local.add_log("read vconfig", "debug") + vconfig = GetConfig(path=vconfig_path) + + # read mconfig + local.add_log("read mconfig", "debug") + mconfig_path = local.buffer.mconfig_path + mconfig = GetConfig(path=mconfig_path) + + # edit mytoncore config file + local.add_log("edit mytoncore config file", "debug") + mconfig.liteClient.liteServer.port = mconfig.liteservers[0].port + mconfig.validatorConsole.addr = f"127.0.0.1:{mconfig.control[0].port}" + + # write mconfig + local.add_log("write mconfig", "debug") + SetConfig(path=mconfig_path, data=mconfig) + + # restart mytoncore + StartMytoncore() +#end define + +def DangerousRecoveryValidatorConfigFile(): + local.add_log("start DangerousRecoveryValidatorConfigFile function", "info") + + # install and import cryptography library + args = ["pip3", "install", "cryptography"] + subprocess.run(args) + from cryptography.hazmat.primitives import serialization + from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey + + # Get keys from keyring + keys = list() + keyringDir = "/var/ton-work/db/keyring/" + keyring = os.listdir(keyringDir) + os.chdir(keyringDir) + sorted(keyring, key=os.path.getmtime) + for item in keyring: + b64String = hex2b64(item) + keys.append(b64String) + #end for + + # Create config object + vconfig = Dict() + vconfig["@type"] = "engine.validator.config" + vconfig.out_port = 3278 + + # Create addrs object + buff = Dict() + buff["@type"] = "engine.addr" + buff.ip = ip2int(get_own_ip()) + buff.port = None + buff.categories = [0, 1, 2, 3] + buff.priority_categories = [] + vconfig.addrs = [buff] + + # Get liteserver fragment + mconfig_path = local.buffer.mconfig_path + mconfig = GetConfig(path=mconfig_path) + lkey = mconfig.liteClient.liteServer.pubkeyPath + lport = mconfig.liteClient.liteServer.port + + # Read lite server pubkey + file = open(lkey, 'rb') + data = file.read() + file.close() + ls_pubkey = data[4:] + + # Search lite server priv key + for item in keyring: + path = keyringDir + item + file = open(path, 'rb') + data = file.read() + file.close() + peivkey = data[4:] + privkeyObject = Ed25519PrivateKey.from_private_bytes(peivkey) + pubkeyObject = privkeyObject.public_key() + pubkey = pubkeyObject.public_bytes(encoding=serialization.Encoding.Raw, format=serialization.PublicFormat.Raw) + if pubkey == ls_pubkey: + ls_id = hex2b64(item) + keys.remove(ls_id) + #end for + + # Create LS object + buff = Dict() + buff["@type"] = "engine.liteServer" + buff.id = ls_id + buff.port = lport + vconfig.liteservers = [buff] + + # Get validator-console fragment + ckey = mconfig.validatorConsole.pubKeyPath + addr = mconfig.validatorConsole.addr + buff = addr.split(':') + cport = int(buff[1]) + + # Read validator-console pubkey + file = open(ckey, 'rb') + data = file.read() + file.close() + vPubkey = data[4:] + + # Search validator-console priv key + for item in keyring: + path = keyringDir + item + file = open(path, 'rb') + data = file.read() + file.close() + peivkey = data[4:] + privkeyObject = Ed25519PrivateKey.from_private_bytes(peivkey) + pubkeyObject = privkeyObject.public_key() + pubkey = pubkeyObject.public_bytes(encoding=serialization.Encoding.Raw, format=serialization.PublicFormat.Raw) + if pubkey == vPubkey: + vcId = hex2b64(item) + keys.remove(vcId) + #end for + + # Create VC object + buff = Dict() + buff2 = Dict() + buff["@type"] = "engine.controlInterface" + buff.id = vcId + buff.port = cport + buff2["@type"] = "engine.controlProcess" + buff2.id = None + buff2.permissions = 15 + buff.allowed = buff2 + vconfig.control = [buff] + + # Get dht fragment + files = os.listdir("/var/ton-work/db") + for item in files: + if item[:3] == "dht": + dhtS = item[4:] + dhtS = dhtS.replace('_', '/') + dhtS = dhtS.replace('-', '+') + break + #end for + + # Get ght from keys + for item in keys: + if dhtS in item: + dhtId = item + keys.remove(dhtId) + #end for + + # Create dht object + buff = Dict() + buff["@type"] = "engine.dht" + buff.id = dhtId + vconfig.dht = [buff] + + # Create adnl object + adnl2 = Dict() + adnl2["@type"] = "engine.adnl" + adnl2.id = dhtId + adnl2.category = 0 + + # Create adnl object + adnlId = hex2b64(mconfig["adnlAddr"]) + keys.remove(adnlId) + adnl3 = Dict() + adnl3["@type"] = "engine.adnl" + adnl3.id = adnlId + adnl3.category = 0 + + # Create adnl object + adnl1 = Dict() + adnl1["@type"] = "engine.adnl" + adnl1.id = keys.pop(0) + adnl1.category = 1 + + vconfig.adnl = [adnl1, adnl2, adnl3] + + # Get dumps from tmp + dumps = list() + dumpsDir = "/tmp/mytoncore/" + dumpsList = os.listdir(dumpsDir) + os.chdir(dumpsDir) + sorted(dumpsList, key=os.path.getmtime) + for item in dumpsList: + if "ElectionEntry.json" in item: + dumps.append(item) + #end for + + # Create validators object + validators = list() + + # Read dump file + while len(keys) > 0: + dumpPath = dumps.pop() + file = open(dumpPath, 'rt') + data = file.read() + file.close() + dump = json.loads(data) + vkey = hex2b64(dump["validatorKey"]) + temp_key = Dict() + temp_key["@type"] = "engine.validatorTempKey" + temp_key.key = vkey + temp_key.expire_at = dump["endWorkTime"] + adnl_addr = Dict() + adnl_addr["@type"] = "engine.validatorAdnlAddress" + adnl_addr.id = adnlId + adnl_addr.expire_at = dump["endWorkTime"] + + # Create validator object + validator = Dict() + validator["@type"] = "engine.validator" + validator.id = vkey + validator.temp_keys = [temp_key] + validator.adnl_addrs = [adnl_addr] + validator.election_date = dump["startWorkTime"] + validator.expire_at = dump["endWorkTime"] + if vkey in keys: + validators.append(validator) + keys.remove(vkey) + #end if + #end while + + # Add validators object to vconfig + vconfig.validators = validators + + + print("vconfig:", json.dumps(vconfig, indent=4)) + print("keys:", keys) +#end define + +def hex2b64(input): + hexBytes = bytes.fromhex(input) + b64Bytes = base64.b64encode(hexBytes) + b64String = b64Bytes.decode() + return b64String +#end define + +def b642hex(input): + b64Bytes = input.encode() + hexBytes = base64.b64decode(b64Bytes) + hexString = hexBytes.hex() + return hexString +#end define + +def CreateSymlinks(): + local.add_log("start CreateSymlinks fuction", "debug") + cport = local.buffer.cport + + mytonctrl_file = "/usr/bin/mytonctrl" + fift_file = "/usr/bin/fift" + liteclient_file = "/usr/bin/lite-client" + validator_console_file = "/usr/bin/validator-console" + env_file = "/etc/environment" + file = open(mytonctrl_file, 'wt') + file.write("/usr/bin/python3 /usr/src/mytonctrl/mytonctrl.py $@") + file.close() + file = open(fift_file, 'wt') + file.write("/usr/bin/ton/crypto/fift $@") + file.close() + file = open(liteclient_file, 'wt') + file.write("/usr/bin/ton/lite-client/lite-client -C /usr/bin/ton/global.config.json $@") + file.close() + if cport: + file = open(validator_console_file, 'wt') + file.write("/usr/bin/ton/validator-engine-console/validator-engine-console -k /var/ton-work/keys/client -p /var/ton-work/keys/server.pub -a 127.0.0.1:" + str(cport) + " $@") + file.close() + args = ["chmod", "+x", validator_console_file] + subprocess.run(args) + args = ["chmod", "+x", mytonctrl_file, fift_file, liteclient_file] + subprocess.run(args) + + # env + fiftpath = "export FIFTPATH=/usr/src/ton/crypto/fift/lib/:/usr/src/ton/crypto/smartcont/" + file = open(env_file, 'rt+') + text = file.read() + if fiftpath not in text: + file.write(fiftpath + '\n') + file.close() +#end define + +def EnableDhtServer(): + local.add_log("start EnableDhtServer function", "debug") + vuser = local.buffer.vuser + ton_bin_dir = local.buffer.ton_bin_dir + globalConfigPath = local.buffer.global_config_path + dht_server = ton_bin_dir + "dht-server/dht-server" + generate_random_id = ton_bin_dir + "utils/generate-random-id" + tonDhtServerDir = "/var/ton-dht-server/" + tonDhtKeyringDir = tonDhtServerDir + "keyring/" + + # Проверить конфигурацию + if os.path.isfile("/var/ton-dht-server/config.json"): + local.add_log("DHT-Server config.json already exist. Break EnableDhtServer fuction", "warning") + return + #end if + + # Подготовить папку + os.makedirs(tonDhtServerDir, exist_ok=True) + + # Прописать автозагрузку + cmd = "{dht_server} -C {globalConfigPath} -D {tonDhtServerDir}" + cmd = cmd.format(dht_server=dht_server, globalConfigPath=globalConfigPath, tonDhtServerDir=tonDhtServerDir) + add2systemd(name="dht-server", user=vuser, start=cmd) + + # Получить внешний ip адрес + ip = get_own_ip() + port = random.randint(2000, 65000) + addr = "{ip}:{port}".format(ip=ip, port=port) + + # Первый запуск + args = [dht_server, "-C", globalConfigPath, "-D", tonDhtServerDir, "-I", addr] + subprocess.run(args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + + # Получить вывод конфига + key = os.listdir(tonDhtKeyringDir)[0] + ip = ip2int(ip) + text = '{"@type": "adnl.addressList", "addrs": [{"@type": "adnl.address.udp", "ip": ' + str(ip) + ', "port": ' + str(port) + '}], "version": 0, "reinit_date": 0, "priority": 0, "expire_at": 0}' + args = [generate_random_id, "-m", "dht", "-k", tonDhtKeyringDir + key, "-a", text] + process = subprocess.run(args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=3) + output = process.stdout.decode("utf-8") + err = process.stderr.decode("utf-8") + if len(err) > 0: + raise Exeption(err) + #end if + + data = json.loads(output) + text = json.dumps(data, indent=4) + print(text) + + # chown 1 + args = ["chown", "-R", vuser + ':' + vuser, tonDhtServerDir] + subprocess.run(args) + + # start DHT-Server + args = ["systemctl", "restart", "dht-server"] + subprocess.run(args) +#end define + +def SetWebPassword(args): + args = ["python3", "/usr/src/mtc-jsonrpc/mtc-jsonrpc.py", "-p"] + subprocess.run(args) +#end define + +def EnableJsonRpc(): + local.add_log("start EnableJsonRpc function", "debug") + user = local.buffer.user + exitCode = run_as_root(["bash", "/usr/src/mytonctrl/scripts/jsonrpcinstaller.sh", "-u", user]) + if exitCode == 0: + text = "EnableJsonRpc - {green}OK{endc}" + else: + text = "EnableJsonRpc - {red}Error{endc}" + color_print(text) +#end define + +def EnableTonHttpApi(): + local.add_log("start EnableTonHttpApi function", "debug") + exitCode = run_as_root(["bash", "/usr/src/mytonctrl/scripts/ton_http_api_installer.sh", "-u", local.buffer.user]) + if exitCode == 0: + text = "EnableTonHttpApi - {green}OK{endc}" + else: + text = "EnableTonHttpApi - {red}Error{endc}" + color_print(text) +#end define + +def str2b64(s): + b = s.encode("utf-8") + b64 = base64.b64encode(b) + b64 = b64.decode("utf-8") + return b64 +#end define + +def b642str(b64): + b64 = b64.encode("utf-8") + b = base64.b64decode(b64) + s = b.decode("utf-8") + return s +#end define + +def dict2b64(d): + s = json.dumps(d) + b64 = str2b64(s) + return b64 +#end define + +def b642dict(b64): + s = b642str(b64) + d = json.loads(s) + return d +#end define + + + +### +### Start of the program +### + +if __name__ == "__main__": + Init() + if len(sys.argv) > 1: + General() + else: + console.Run() + local.exit() +#end if diff --git a/scripts/ton_http_api_installer.sh b/scripts/ton_http_api_installer.sh new file mode 100644 index 00000000..9327f870 --- /dev/null +++ b/scripts/ton_http_api_installer.sh @@ -0,0 +1,40 @@ +#!/bin/bash +set -e + +# Проверить sudo +if [ "$(id -u)" != "0" ]; then + echo "Please run script as root" + exit 1 +fi + +# Цвета +COLOR='\033[92m' +ENDC='\033[0m' + +# Установка компонентов python3 +pip3 install virtualenv + +# Подготовить папку с виртуальным окружением +echo -e "${COLOR}[1/4]${ENDC} Preparing the virtual environment" +venv_path="/opt/virtualenv/ton_http_api" +virtualenv ${venv_path} + +# Установка компонентов python3 +echo -e "${COLOR}[2/4]${ENDC} Installing required packages" +user=$(logname) +venv_pip3="${venv_path}/bin/pip3" +${venv_pip3} install ton-http-api +chown -R ${user}:${user} ${venv_path} + +# Прописать автозагрузку +echo -e "${COLOR}[3/4]${ENDC} Add to startup" +venv_ton_http_api="${venv_path}/bin/ton-http-api" +tonlib_path="/usr/bin/ton/tonlib/libtonlibjson.so" +ls_config="/usr/bin/ton/localhost.config.json" +cmd="from sys import path; path.append('/usr/src/mytonctrl/'); from mypylib.mypylib import add2systemd; add2systemd(name='ton_http_api', user='${user}', start='${venv_ton_http_api} --host 127.0.0.1 --port 8801 --liteserver-config ${ls_config} --cdll-path ${tonlib_path} --tonlib-keystore /tmp/tonlib_keystore/')" +python3 -c "${cmd}" +systemctl restart ton_http_api + +# Конец +echo -e "${COLOR}[4/4]${ENDC} ton_http_api service installation complete" +exit 0