diff --git a/docs/Build/node-cli.md b/docs/Build/node-cli.md index 8bc8a7add..88d532bf9 100644 --- a/docs/Build/node-cli.md +++ b/docs/Build/node-cli.md @@ -35,11 +35,11 @@ Execute `cardano-cli` and `cardano-node` to verify output as below (the exact ve ```bash cardano-cli version -# cardano-cli 1.32.1 - linux-x86_64 - ghc-8.10 -# git rev 4f65fb9a27aa7e3a1873ab4211e412af780a3648 +# cardano-cli 1.35.0 - linux-x86_64 - ghc-8.10 +# git rev <...> cardano-node version -# cardano-node 1.32.1 - linux-x86_64 - ghc-8.10 -# git rev 4f65fb9a27aa7e3a1873ab4211e412af780a3648 +# cardano-node 1.35.0 - linux-x86_64 - ghc-8.10 +# git rev <...> ``` #### Update port number or pool name for relative paths diff --git a/docs/Scripts/cncli.md b/docs/Scripts/cncli.md index 6ffd34310..37370fcf5 100644 --- a/docs/Scripts/cncli.md +++ b/docs/Scripts/cncli.md @@ -1,7 +1,7 @@ !!! info "Reminder !!" Ensure the [Pre-Requisites](../basics.md#pre-requisites) are in place before you proceed. -`cncli.sh` is a script to download and deploy [CNCLI](https://github.com/AndrewWestberg/cncli) created and maintained by Andrew Westberg. It's a community-based CLI tool written in RUST for low-level `cardano-node` communication. Usage is **optional** and no script is dependent on it. The main features include: +`cncli.sh` is a script to download and deploy [CNCLI](https://github.com/cardano-community/cncli) created and maintained by Andrew Westberg. It's a community-based CLI tool written in RUST for low-level `cardano-node` communication. Usage is **optional** and no script is dependent on it. The main features include: - **PING** - Validates that the remote server is on the given network and returns its response time. Utilized by `gLiveView` for peer analysis if available. - **SYNC** - Connects to a node (local or remote) and synchronizes blocks to a local `sqlite` database. diff --git a/docs/Scripts/cntools-changelog.md b/docs/Scripts/cntools-changelog.md index 0d63550e4..8dbbe9b86 100644 --- a/docs/Scripts/cntools-changelog.md +++ b/docs/Scripts/cntools-changelog.md @@ -6,6 +6,17 @@ All notable changes to this tool will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [10.0.0] - 2022-06-28 +#### Added +- Support for Vasil Fork +- Preliminary support for Post HF updates (a short release will follow post fork in coming days) +- Minimum version for Node bumped to 1.35.0 + +#### Changed +- Pool > Rotate code now uses kes-periodinfo CLI query to get counter from node (fallback for Koios) +- Pool > Show Info updated to include current KES counter +- Update getEraIdentifier to include Babbage era + ## [9.1.0] - 2022-05-11 #### Changed - Harmonize flow for reusing old wallet configuration on pool modification vs setting new wallets. diff --git a/docs/Scripts/gliveview.md b/docs/Scripts/gliveview.md index af5e0d3e3..9e92bf392 100644 --- a/docs/Scripts/gliveview.md +++ b/docs/Scripts/gliveview.md @@ -56,6 +56,12 @@ Displays live metrics from cardano-node gathered through the nodes EKG/Prometheu - **Tip (diff) / Status** - Will either show node status as `starting|sync xx.x%` or if close to reference tip, the tip difference `Tip (ref) - Tip (node)` to see how far of the tip (diff value) the node is. With current parameters a slot diff up to 40 from reference tip is considered good but it should usually stay below 30. It's perfectly normal to see big differences in slots between blocks. It's the built in randomness at play. To see if a node is really healthy and staying on tip you would need to compare the tip between multiple nodes. - **Forks** - The number of forks since node start. Each fork means the blockchain evolved in a different direction, thereby discarding blocks. A high number of forks means there is a higher chance of orphaned blocks. - **Peers In / Out** - Shows how many connections the node has established in and out. See [Peer analysis](#peer-analysis) section for how to get more details of incoming and outgoing connections. +- **P2P Mode** + - `Cold` peers indicate the number of inactive but known peers to the node. + - `Warm` peers tell how many established connections the node has. + - `Hot` peers how many established connections are actually active. + - `Bi-Dir`(bidirectional) and `Uni-Dir`(unidirectional) indicate how the handshake protocol negotiated the connection. The connection between p2p nodes will always be bidirectional, but it will be unidirectional between p2p nodes and non-p2p nodes. + - `Duplex` shows the connections that are actually used in both directions, only bidirectional connections have this potential. - **Mem (RSS)** - RSS is the Resident Set Size and shows how much memory is allocated to cardano-node and that is in RAM. It does not include memory that is swapped out. It does include memory from shared libraries as long as the pages from those libraries are actually in memory. It does include all stack and heap memory. - **Mem (Live) / (Heap)** - GC (Garbage Collector) values that show how much memory is used for live/heap data. A large difference between them (or the heap approaching the physical memory limit) means the node is struggling with the garbage collector and/or may begin swapping. - **GC Minor / Major** - Collecting garbage from "Young space" is called a Minor GC. Major (Full) GC is done more rarily and is a more expensive operation. Explaining garbage collection is a topic outside the scope of this documentation and google is your friend for this. diff --git a/files/docker/grest/scripts/docker-getmetrics.sh b/files/docker/grest/scripts/docker-getmetrics.sh index 4b156aa6d..b9b0f5300 100755 --- a/files/docker/grest/scripts/docker-getmetrics.sh +++ b/files/docker/grest/scripts/docker-getmetrics.sh @@ -119,7 +119,7 @@ function get-metrics() { export METRIC_grestschsize="${grestschsize}" export METRIC_dbsize="${dbsize}" #export METRIC_cnodeversion="$(echo $(cardano-node --version) | awk '{print $2 "-" $9}')" - #export METRIC_dbsyncversion="$(echo $(cardano-db-sync-extended --version) | awk '{print $2 "-" $9}')" + #export METRIC_dbsyncversion="$(echo $(cardano-db-sync --version) | awk '{print $2 "-" $9}')" #export METRIC_psqlversion="$(echo "" | psql -U "${POSTGRES_USER}" -d "${POSTGRES_DB}" -c "SELECT version();" | grep PostgreSQL | awk '{print $2}')" for metric_var_name in $(env | grep ^METRIC | sort | awk -F= '{print $1}') diff --git a/files/tests/pre-merge/amazonlinux2-cabal.containerfile b/files/tests/pre-merge/amazonlinux2-cabal.containerfile index 5ac1b01d6..cf6e1935a 100644 --- a/files/tests/pre-merge/amazonlinux2-cabal.containerfile +++ b/files/tests/pre-merge/amazonlinux2-cabal.containerfile @@ -6,10 +6,11 @@ ARG CNODE_HOME=/opt/cardano/cnode ENV \ LANG=C.UTF-8 \ USER=root \ - PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH + PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH \ + LD_LIBRARY_PATH=/usr/lib:/usr/lib64:/usr/local/lib:$LD_LIBRARY_PATH RUN git clone https://github.com/input-output-hk/cardano-node &&\ - pwd ; ls -l + pwd WORKDIR /cardano-node @@ -21,4 +22,4 @@ RUN curl -o cardano-node-latest.txt "https://raw.githubusercontent.com/cardano-c git status &&\ /opt/cardano/cnode/scripts/cabal-build-all.sh &&\ cabal install cardano-ping &&\ - /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version \ No newline at end of file + /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version diff --git a/files/tests/pre-merge/amazonlinux2-cabal_-l.containerfile b/files/tests/pre-merge/amazonlinux2-cabal_-l.containerfile index ec5e092ef..dfec000ec 100644 --- a/files/tests/pre-merge/amazonlinux2-cabal_-l.containerfile +++ b/files/tests/pre-merge/amazonlinux2-cabal_-l.containerfile @@ -6,10 +6,11 @@ ARG CNODE_HOME=/opt/cardano/cnode ENV \ LANG=C.UTF-8 \ USER=root \ - PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH + PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH \ + LD_LIBRARY_PATH=/usr/lib:/usr/lib64:/usr/local/lib:$LD_LIBRARY_PATH RUN git clone https://github.com/input-output-hk/cardano-node &&\ - pwd ; ls -l + pwd WORKDIR /cardano-node @@ -21,4 +22,4 @@ RUN curl -o cardano-node-latest.txt "https://raw.githubusercontent.com/cardano-c git status &&\ /opt/cardano/cnode/scripts/cabal-build-all.sh -l &&\ cabal install cardano-ping &&\ - /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version \ No newline at end of file + /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version diff --git a/files/tests/pre-merge/debian-cabal.containerfile b/files/tests/pre-merge/debian-cabal.containerfile index a7a1a6445..d87f8b40d 100644 --- a/files/tests/pre-merge/debian-cabal.containerfile +++ b/files/tests/pre-merge/debian-cabal.containerfile @@ -10,7 +10,7 @@ ENV \ PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH RUN git clone https://github.com/input-output-hk/cardano-node &&\ - pwd ; ls -l + pwd WORKDIR /cardano-node diff --git a/files/tests/pre-merge/debian-cabal_-l.containerfile b/files/tests/pre-merge/debian-cabal_-l.containerfile index b813563ca..f6bd03abc 100644 --- a/files/tests/pre-merge/debian-cabal_-l.containerfile +++ b/files/tests/pre-merge/debian-cabal_-l.containerfile @@ -10,7 +10,7 @@ ENV \ PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH RUN git clone https://github.com/input-output-hk/cardano-node &&\ - pwd ; ls -l + pwd WORKDIR /cardano-node diff --git a/files/tests/pre-merge/rockylinux8-cabal.containerfile b/files/tests/pre-merge/rockylinux8-cabal.containerfile index 27504bb22..a79fb14f5 100644 --- a/files/tests/pre-merge/rockylinux8-cabal.containerfile +++ b/files/tests/pre-merge/rockylinux8-cabal.containerfile @@ -9,7 +9,7 @@ ENV \ PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH RUN git clone https://github.com/input-output-hk/cardano-node &&\ - pwd ; ls -l + pwd WORKDIR /cardano-node @@ -21,4 +21,4 @@ RUN curl -o cardano-node-latest.txt "https://raw.githubusercontent.com/cardano-c git status &&\ /opt/cardano/cnode/scripts/cabal-build-all.sh &&\ cabal install cardano-ping &&\ - /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version \ No newline at end of file + /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version diff --git a/files/tests/pre-merge/rockylinux8-cabal_-l.containerfile b/files/tests/pre-merge/rockylinux8-cabal_-l.containerfile index b6d1f634f..7c5a46a6b 100644 --- a/files/tests/pre-merge/rockylinux8-cabal_-l.containerfile +++ b/files/tests/pre-merge/rockylinux8-cabal_-l.containerfile @@ -9,7 +9,7 @@ ENV \ PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH RUN git clone https://github.com/input-output-hk/cardano-node &&\ - pwd ; ls -l + pwd WORKDIR /cardano-node @@ -21,4 +21,4 @@ RUN curl -o cardano-node-latest.txt "https://raw.githubusercontent.com/cardano-c git status &&\ /opt/cardano/cnode/scripts/cabal-build-all.sh -l &&\ cabal install cardano-ping &&\ - /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version \ No newline at end of file + /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version diff --git a/files/tests/pre-merge/ubuntu20-cabal.containerfile b/files/tests/pre-merge/ubuntu20-cabal.containerfile index 54f5c5bf0..e85fd4fdf 100644 --- a/files/tests/pre-merge/ubuntu20-cabal.containerfile +++ b/files/tests/pre-merge/ubuntu20-cabal.containerfile @@ -10,7 +10,7 @@ ENV \ PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH RUN git clone https://github.com/input-output-hk/cardano-node &&\ - pwd ; ls -l + pwd WORKDIR /cardano-node diff --git a/files/tests/pre-merge/ubuntu20-cabal_-l.containerfile b/files/tests/pre-merge/ubuntu20-cabal_-l.containerfile index dcdab6b37..958ef81ed 100644 --- a/files/tests/pre-merge/ubuntu20-cabal_-l.containerfile +++ b/files/tests/pre-merge/ubuntu20-cabal_-l.containerfile @@ -10,7 +10,7 @@ ENV \ PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH RUN git clone https://github.com/input-output-hk/cardano-node &&\ - pwd ; ls -l + pwd WORKDIR /cardano-node diff --git a/scripts/cnode-helper-scripts/cabal-build-all.sh b/scripts/cnode-helper-scripts/cabal-build-all.sh index e4a5789af..c7104fd3c 100755 --- a/scripts/cnode-helper-scripts/cabal-build-all.sh +++ b/scripts/cnode-helper-scripts/cabal-build-all.sh @@ -6,22 +6,42 @@ echo "Deleting build config artifact to remove cached version, this prevents invalid Git Rev" find dist-newstyle/build/x86_64-linux/ghc-8.10.?/cardano-config-* >/dev/null 2>&1 && rm -rf "dist-newstyle/build/x86_64-linux/ghc-8.*/cardano-config-*" +[[ -f /usr/lib/libsecp256k1.so ]] && export LD_LIBRARY_PATH=/usr/lib:"${LD_LIBRARY_PATH}" +[[ -f /usr/lib64/libsecp256k1.so ]] && export LD_LIBRARY_PATH=/usr/lib64:"${LD_LIBRARY_PATH}" +[[ -f /usr/local/lib/libsecp256k1.so ]] && export LD_LIBRARY_PATH=/usr/local/lib:"${LD_LIBRARY_PATH}" +[[ -d /usr/lib/pkgconfig ]] && export PKG_CONFIG_PATH=/usr/lib/pkgconfig:"${PKG_CONFIG_PATH}" +[[ -d /usr/lib64/pkgconfig ]] && export PKG_CONFIG_PATH=/usr/lib64/pkgconfig:"${PKG_CONFIG_PATH}" +[[ -d /usr/local/lib/pkgconfig ]] && export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:"${PKG_CONFIG_PATH}" + if [[ "$1" == "-l" ]] ; then USE_SYSTEM_LIBSODIUM="package cardano-crypto-praos flags: -external-libsodium-vrf" - # In case Custom libsodium module is present, exclude it from Load Library Path - [[ -f /usr/local/lib/libsodium.so ]] && export LD_LIBRARY_PATH=${LD_LIBRARY_PATH/\/usr\/local\/lib:/} else unset USE_SYSTEM_LIBSODIUM - source "${HOME}"/.bashrc - [[ -d /usr/local/lib/pkgconfig ]] && export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:"${PKG_CONFIG_PATH}" - [[ -f /usr/local/lib/libsodium.so ]] && export LD_LIBRARY_PATH=/usr/local/lib:"${LD_LIBRARY_PATH}" fi [[ -f cabal.project.local ]] && mv cabal.project.local cabal.project.local.bkp_"$(date +%s)" cat <<-EOF > .tmp.cabal.project.local ${USE_SYSTEM_LIBSODIUM} + source-repository-package + type: git + location: https://github.com/input-output-hk/hjsonpointer + tag: bb99294424e0c5b3c2942c743b545e4b01c12ce8 + --sha256: 11z5s4xmm6cxy6sdcf9ly4lr0qh3c811hpm0bmlp4c3yq8v3m9rk + + source-repository-package + type: git + location: https://github.com/input-output-hk/hjsonschema + tag: 1546af7fc267d5eea805ef223dd2b59ac120b784 + --sha256: 0sdikhsq6nnhmmvcpwzzrwfn9cn7rg75629qnrx01i5vm5ci4314 + + source-repository-package + type: git + location: https://github.com/haskell-works/hw-aeson + tag: d99d2f3e39a287607418ae605b132a3deb2b753f + --sha256: 1vxqcwjg9q37wbwi27y9ba5163lzfz51f1swbi0rp681yg63zvn4 + source-repository-package type: git location: https://github.com/input-output-hk/bech32 @@ -31,34 +51,40 @@ cat <<-EOF > .tmp.cabal.project.local source-repository-package type: git location: https://github.com/input-output-hk/cardano-addresses - tag: 71006f9eb956b0004022e80aadd4ad50d837b621 + tag: b6f2f3cef01a399376064194fd96711a5bdba4a7 subdir: command-line core + allow-newer: + *:aeson + EOF chmod 640 .tmp.cabal.project.local -if [[ -z "${USE_SYSTEM_LIBSODIUM}" ]] ; then - echo "Running cabal update to ensure you're on latest dependencies.." - cabal update 2>&1 | tee /tmp/cabal-update.log - echo "Building.." - cabal build all 2>&1 | tee tee /tmp/build.log +echo "Running cabal update to ensure you're on latest dependencies.." +cabal update 2>&1 | tee /tmp/cabal-update.log +echo "Building.." + +if [[ -z "${USE_SYSTEM_LIBSODIUM}" ]] ; then # Build using default cabal.project first and then add cabal.project.local for additional packages if [[ "${PWD##*/}" == "cardano-node" ]] || [[ "${PWD##*/}" == "cardano-db-sync" ]]; then - echo "Overwriting cabal.project.local to include cardano-addresses and bech32 .." + #cabal install cardano-crypto-class --disable-tests --disable-profiling | tee /tmp/build.log + [[ "${PWD##*/}" == "cardano-node" ]] && cabal build cardano-node cardano-cli cardano-submit-api --disable-tests --disable-profiling | tee /tmp/build.log + [[ "${PWD##*/}" == "cardano-db-sync" ]] && cabal build cardano-db-sync --disable-tests --disable-profiling | tee /tmp/build.log mv .tmp.cabal.project.local cabal.project.local - cabal install bech32 cardano-addresses-cli --overwrite-policy=always 2>&1 | tee /tmp/build-b32-caddr.log + cabal install bech32 cardano-addresses-cli cardano-ping --overwrite-policy=always 2>&1 | tee /tmp/build-b32-caddr.log + else + cabal build all --disable-tests --disable-profiling 2>&1 | tee /tmp/build.log fi -else +else # Add cabal.project.local customisations first before building if [[ "${PWD##*/}" == "cardano-node" ]] || [[ "${PWD##*/}" == "cardano-db-sync" ]]; then - echo "Overwriting cabal.project.local to include cardano-addresses and bech32 .." mv .tmp.cabal.project.local cabal.project.local + [[ "${PWD##*/}" == "cardano-node" ]] && cabal build cardano-node cardano-cli cardano-submit-api --disable-tests --disable-profiling | tee /tmp/build.log + [[ "${PWD##*/}" == "cardano-db-sync" ]] && cabal build cardano-db-sync --disable-tests --disable-profiling | tee /tmp/build.log + else + cabal build all --disable-tests --disable-profiling 2>&1 | tee /tmp/build.log fi - echo "Running cabal update to ensure you're on latest dependencies.." - cabal update 2>&1 | tee /tmp/cabal-update.log - echo "Building.." - cabal build all 2>&1 | tee tee /tmp/build.log - [[ -f cabal.project.local ]] && cabal install bech32 cardano-addresses-cli --overwrite-policy=always 2>&1 | tee /tmp/build-b32-caddr.log + [[ -f cabal.project.local ]] && cabal install bech32 cardano-ping cardano-addresses-cli --overwrite-policy=always 2>&1 | tee /tmp/build-b32-caddr.log fi grep "^Linking" /tmp/build.log | grep -Ev 'test|golden|demo|chairman|locli|ledger|topology' | while read -r line ; do diff --git a/scripts/cnode-helper-scripts/cntools.library b/scripts/cnode-helper-scripts/cntools.library index fb79650bc..a0ca2973f 100644 --- a/scripts/cnode-helper-scripts/cntools.library +++ b/scripts/cnode-helper-scripts/cntools.library @@ -7,9 +7,9 @@ # The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) # and this adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) # Major: Any considerable change in the code base, big feature, workflow or breaking change from previous version -CNTOOLS_MAJOR_VERSION=9 +CNTOOLS_MAJOR_VERSION=10 # Minor: Changes and features of minor character that can be applied without breaking existing functionality or workflow -CNTOOLS_MINOR_VERSION=1 +CNTOOLS_MINOR_VERSION=0 # Patch: Backwards compatible bug fixes. No additional functionality or major changes CNTOOLS_PATCH_VERSION=0 @@ -590,9 +590,10 @@ isPoolRegistered() { .[0].pool_status //"-", .[0].retiring_epoch //"-", .[0].op_cert //"-", - .[0].op_cert_counter //0, + .[0].op_cert_counter //"null", .[0].active_stake //0, - .[0].epoch_block_cnt //0, + .[0].block_count //0, + .[0].live_pledge //0, .[0].live_stake //0, .[0].live_delegators //0, .[0].live_saturation //0 @@ -616,10 +617,11 @@ isPoolRegistered() { p_op_cert=${pool_info_arr[13]} p_op_cert_counter=${pool_info_arr[14]} p_active_stake=${pool_info_arr[15]} - p_epoch_block_cnt=${pool_info_arr[16]} - p_live_stake=${pool_info_arr[17]} - p_live_delegators=${pool_info_arr[18]} - p_live_saturation=${pool_info_arr[19]} + p_block_count=${pool_info_arr[16]} + p_live_pledge=${pool_info_arr[17]} + p_live_stake=${pool_info_arr[18]} + p_live_delegators=${pool_info_arr[19]} + p_live_saturation=${pool_info_arr[20]} [[ ${p_pool_status} = 'registered' ]] && return 2 || return 3 fi @@ -1993,7 +1995,7 @@ deRegisterPool() { # Command : rotatePoolKeys # Description : Rotate pool's KES keys -# Parameters : pool name > the pool name to rotate KES keys for +# parameters : $1 = cold counter (offline mode) rotatePoolKeys() { # cold keys pool_coldkey_sk_file="${POOL_FOLDER}/${pool_name}/${POOL_COLDKEY_SK_FILENAME}" @@ -2019,18 +2021,34 @@ rotatePoolKeys() { ! ${CCLI} node key-gen-KES --verification-key-file "${pool_hotkey_vk_file}" --signing-key-file "${pool_hotkey_sk_file}" && return 1 p_opcert="" - if [[ -n ${KOIOS_API} ]]; then + if [[ $# -eq 1 ]]; then + println ACTION "${CCLI} node new-counter --cold-verification-key-file ${pool_coldkey_vk_file} --counter-value $1 --operational-certificate-issue-counter-file ${pool_opcert_counter_file}" + ! ${CCLI} node new-counter --cold-verification-key-file "${pool_coldkey_vk_file}" --counter-value $1 --operational-certificate-issue-counter-file "${pool_opcert_counter_file}" && return 1 + elif [[ -n ${KOIOS_API} ]]; then ! getPoolID "${pool_name}" && println "ERROR" "\n${FG_RED}ERROR${NC}: failed to get pool ID!\n" && return 1 println ACTION "curl -sSL -f -X POST -H \"Content-Type: application/json\" -d '{\"_pool_bech32_ids\":[\"${pool_id_bech32}\"]}' ${KOIOS_API}/pool_info" ! pool_info=$(curl -sSL -f -X POST -H "Content-Type: application/json" -d '{"_pool_bech32_ids":["'${pool_id_bech32}'"]}' "${KOIOS_API}/pool_info" 2>&1) && println "ERROR" "\n${FG_RED}KOIOS_API ERROR${NC}: ${pool_info}\n" && p_opcert="" # print error but ignore if old_counter_nbr=$(jq -er '.[0].op_cert_counter' <<< "${pool_info}" 2>/dev/null); then - println ACTION "${CCLI} node new-counter --cold-verification-key-file ${pool_coldkey_vk_file} --counter-value $(( old_counter_nbr + 1 )) --operational-certificate-issue-counter-file ${pool_opcert_counter_file}" - ! ${CCLI} node new-counter --cold-verification-key-file "${pool_coldkey_vk_file}" --counter-value $(( old_counter_nbr + 1 )) --operational-certificate-issue-counter-file "${pool_opcert_counter_file}" && return 1 - elif [[ ! -f ${pool_opcert_counter_file} ]]; then - println "ERROR" "\n${FG_RED}ERROR${NC}: op cert counter file missing and unable to get previous counter value!\n" && return 1 + new_counter_nbr=$(( old_counter_nbr + 1 )) + else + new_counter_nbr=0 # null returned = no block on chain for this pool + fi + println ACTION "${CCLI} node new-counter --cold-verification-key-file ${pool_coldkey_vk_file} --counter-value ${new_counter_nbr} --operational-certificate-issue-counter-file ${pool_opcert_counter_file}" + ! ${CCLI} node new-counter --cold-verification-key-file "${pool_coldkey_vk_file}" --counter-value ${new_counter_nbr} --operational-certificate-issue-counter-file "${pool_opcert_counter_file}" && return 1 + elif [[ -f ${pool_opcert_file} ]]; then + println ACTION "${CCLI} query kes-period-info --op-cert-file ${pool_opcert_file} ${NETWORK_IDENTIFIER}" + if ! kes_period_info=$(${CCLI} query kes-period-info --op-cert-file "${pool_opcert_file}" ${NETWORK_IDENTIFIER}); then + println "ERROR" "\n${FG_RED}ERROR${NC}: failed to grab counter from node: [${kes_period_info}]\n" && return 1 fi - elif [[ ! -f ${pool_opcert_counter_file} ]]; then - println "ERROR" "\n${FG_RED}ERROR${NC}: op cert counter file missing and unable to get previous counter value!\n" && return 1 + if old_counter_nbr=$(awk '/{/,0' <<< "${kes_period_info}" | jq -er '.qKesNodeStateOperationalCertificateNumber' 2>/dev/null); then + new_counter_nbr=$(( old_counter_nbr + 1 )) + else + new_counter_nbr=0 # null returned = no block on chain for this pool + fi + println ACTION "${CCLI} node new-counter --cold-verification-key-file ${pool_coldkey_vk_file} --counter-value ${new_counter_nbr} --operational-certificate-issue-counter-file ${pool_opcert_counter_file}" + ! ${CCLI} node new-counter --cold-verification-key-file "${pool_coldkey_vk_file}" --counter-value ${new_counter_nbr} --operational-certificate-issue-counter-file "${pool_opcert_counter_file}" && return 1 + else + println "ERROR" "\n${FG_RED}ERROR${NC}: op cert file missing and Koios disabled/unavailable. Unable to get current on-chain counter value!\n" && return 1 fi println ACTION "${CCLI} node issue-op-cert --kes-verification-key-file ${pool_hotkey_vk_file} --cold-signing-key-file ${pool_coldkey_sk_file} --operational-certificate-issue-counter-file ${pool_opcert_counter_file} --kes-period ${current_kes_period} --out-file ${pool_opcert_file}" diff --git a/scripts/cnode-helper-scripts/cntools.sh b/scripts/cnode-helper-scripts/cntools.sh index b79d3c274..83c6b521d 100755 --- a/scripts/cnode-helper-scripts/cntools.sh +++ b/scripts/cnode-helper-scripts/cntools.sh @@ -2606,6 +2606,7 @@ function main { else println "$(printf "%-15s (${FG_YELLOW}%s${NC}) : ${FG_LBLUE}%s${NC} Ada" "Pledge" "new" "$(formatAsset "${fPParams_pledge::-6}")" )" fi + [[ -n ${KOIOS_API} ]] && println "$(printf "%-21s : ${FG_LBLUE}%s${NC} Ada" "Live Pledge" "$(formatLovelace "${p_live_pledge}")")" # get margin if [[ -z ${KOIOS_API} ]]; then @@ -2738,14 +2739,29 @@ function main { else # get active/live stake/block info println "$(printf "%-21s : ${FG_LBLUE}%s${NC} Ada" "Active Stake" "$(formatLovelace "${p_active_stake}")")" - println "$(printf "%-21s : ${FG_LBLUE}%s${NC}" "Epoch Blocks" "${p_epoch_block_cnt}")" + println "$(printf "%-21s : ${FG_LBLUE}%s${NC}" "Epoch Blocks" "${p_block_count}")" println "$(printf "%-21s : ${FG_LBLUE}%s${NC} Ada" "Live Stake" "$(formatLovelace "${p_live_stake}")")" println "$(printf "%-21s : ${FG_LBLUE}%s${NC} (incl owners)" "Delegators" "${p_live_delegators}")" println "$(printf "%-21s : ${FG_LBLUE}%s${NC} %%" "Saturation" "${p_live_saturation}")" fi unset pool_kes_start - if [[ ${CNTOOLS_MODE} = "CONNECTED" ]]; then + if [[ -n ${KOIOS_API} ]]; then + [[ ${p_op_cert_counter} != null ]] && kes_counter_str="${FG_LBLUE}${p_op_cert_counter}${FG_LGRAY} - use counter ${FG_LBLUE}$((p_op_cert_counter+1))${FG_LGRAY} for rotation in offline mode.${NC}" || kes_counter_str="${FG_LGRAY}No blocks minted so far with active operational certificate. Use counter ${FG_LBLUE}0${FG_LGRAY} for rotation in offline mode.${NC}" + println "$(printf "%-21s : %s" "KES counter" "${kes_counter_str}")" + elif [[ ${CNTOOLS_MODE} = "CONNECTED" ]]; then + pool_opcert_file="${POOL_FOLDER}/${pool_name}/${POOL_OPCERT_FILENAME}" + println ACTION "${CCLI} query kes-period-info --op-cert-file ${pool_opcert_file} ${NETWORK_IDENTIFIER}" + if ! kes_period_info=$(${CCLI} query kes-period-info --op-cert-file "${pool_opcert_file}" ${NETWORK_IDENTIFIER}); then + kes_counter_str="${FG_RED}ERROR${NC}: failed to grab counter from node: [${FG_LGRAY}${kes_period_info}${NC}]" + else + if op_cert_counter=$(awk '/{/,0' <<< "${kes_period_info}" | jq -er '.qKesNodeStateOperationalCertificateNumber' 2>/dev/null); then + kes_counter_str="${FG_LBLUE}${op_cert_counter}${FG_LGRAY} - use counter ${FG_LBLUE}$((op_cert_counter+1))${FG_LGRAY} for rotation in offline mode.${NC}" + else + kes_counter_str="${FG_LGRAY}No blocks minted so far with active operational certificate. Use counter ${FG_LBLUE}0${FG_LGRAY} for rotation in offline mode.${NC}" + fi + fi + println "$(printf "%-21s : %s" "KES counter" "${kes_counter_str}")" getNodeMetrics else [[ -f "${POOL_FOLDER}/${pool_name}/${POOL_CURRENT_KES_START}" ]] && pool_kes_start="$(cat "${POOL_FOLDER}/${pool_name}/${POOL_CURRENT_KES_START}")" @@ -2774,13 +2790,27 @@ function main { println " >> POOL >> ROTATE KES" println DEBUG "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~" echo + if [[ ${CNTOOLS_MODE} = "OFFLINE" ]]; then + println DEBUG "${FG_LGRAY}OFFLINE MODE${NC}: CNTools started in offline mode, please grab correct counter value from online node using pool info!\n" + fi [[ ! $(ls -A "${POOL_FOLDER}" 2>/dev/null) ]] && println "${FG_YELLOW}No pools available!${NC}" && waitForInput && continue println DEBUG "# Select pool to rotate KES keys on" if ! selectPool "all" "${POOL_COLDKEY_SK_FILENAME}"; then # ${pool_name} populated by selectPool function waitForInput && continue fi - if ! rotatePoolKeys; then - waitForInput && continue + if [[ ${CNTOOLS_MODE} = "OFFLINE" ]]; then + getAnswerAnyCust new_counter "Enter new counter number" + if ! isNumber ${new_counter}; then + println ERROR "\n${FG_RED}ERROR${NC}: not a number" + waitForInput && continue + fi + if ! rotatePoolKeys ${new_counter}; then + waitForInput && continue + fi + else + if ! rotatePoolKeys; then + waitForInput && continue + fi fi echo println "Pool KES keys successfully updated" diff --git a/scripts/cnode-helper-scripts/dbsync.sh b/scripts/cnode-helper-scripts/dbsync.sh index a1068a5d9..388610cc4 100755 --- a/scripts/cnode-helper-scripts/dbsync.sh +++ b/scripts/cnode-helper-scripts/dbsync.sh @@ -9,7 +9,7 @@ ###################################### #PGPASSFILE="${CNODE_HOME}/priv/.pgpass" # PGPass file containing connection information for the postgres instance -#DBSYNCBIN="${HOME}/.cabal/bin/cardano-db-sync-extended" # Path for cardano-db-sync-extended binary, assumed to be available in $PATH +#DBSYNCBIN="${HOME}/.cabal/bin/cardano-db-sync" # Path for cardano-db-sync binary, assumed to be available in $PATH #DBSYNC_STATE_DIR="${CNODE_HOME}/guild-db/ledger-state" # Folder where DBSync instance will dump ledger-state files #DBSYNC_SCHEMA_DIR="${CNODE_HOME}/guild-db/schema" # Path to DBSync repository's schema folder #DBSYNC_CONFIG="${CNODE_HOME}/files/dbsync.json" # Config file for dbsync instance @@ -36,7 +36,7 @@ usage() { } set_defaults() { - [[ -z "${DBSYNCBIN}" ]] && DBSYNCBIN="${HOME}/.cabal/bin/cardano-db-sync-extended" + [[ -z "${DBSYNCBIN}" ]] && DBSYNCBIN="${HOME}/.cabal/bin/cardano-db-sync" [[ -z "${PGPASSFILE}" ]] && PGPASSFILE="${CNODE_HOME}/priv/.pgpass" [[ -z "${DBSYNC_CONFIG}" ]] && DBSYNC_CONFIG="${CNODE_HOME}/files/dbsync.json" [[ -z "${DBSYNC_SCHEMA_DIR}" ]] && DBSYNC_SCHEMA_DIR="${CNODE_HOME}/guild-db/schema" @@ -45,8 +45,8 @@ set_defaults() { } check_defaults() { - if [[ ! -f "${DBSYNCBIN}" ]] && [[ ! $(command -v cardano-db-sync-extended &>/dev/null) ]]; then - echo "ERROR: cardano-db-sync-extended seems to be absent in PATH, please investigate \$PATH environment variable!" && exit 1 + if [[ ! -f "${DBSYNCBIN}" ]] && [[ ! $(command -v cardano-db-sync &>/dev/null) ]]; then + echo "ERROR: cardano-db-sync seems to be absent in PATH, please investigate \$PATH environment variable!" && exit 1 elif [[ ! -f "${PGPASSFILE}" ]]; then echo "ERROR: The PGPASSFILE (${PGPASSFILE}) not found, please ensure you've followed the instructions on guild-operators website!" && exit 1 exit 1 diff --git a/scripts/cnode-helper-scripts/env b/scripts/cnode-helper-scripts/env index 2239b643e..3a51c7de1 100644 --- a/scripts/cnode-helper-scripts/env +++ b/scripts/cnode-helper-scripts/env @@ -882,8 +882,8 @@ if ! command -v "jq" &>/dev/null; then return 1 fi -read -ra CONFIG_CONTENTS <<<"$(jq -r '[ .AlonzoGenesisFile, .ByronGenesisFile, .ShelleyGenesisFile, .Protocol, .TraceChainDb]| @tsv' "${CONFIG}" 2>/dev/null)" -if [[ -z "${CONFIG_CONTENTS[4]}" ]]; then +read -ra CONFIG_CONTENTS <<<"$(jq -r '[ .AlonzoGenesisFile //"null", .ByronGenesisFile //"null", .ShelleyGenesisFile //"null", .Protocol //"Cardano", .TraceChainDb //"null", .EnableP2P //"false"]| @tsv' "${CONFIG}" 2>/dev/null)" +if [ ${CONFIG_CONTENTS[4]} != "true" ]; then echo "Could not find TraceChainDb when attempting to parse ${CONFIG} file in JSON format, please double-check the syntax of your config, or simply download it from guild-operators repository!" return 1 else @@ -899,6 +899,7 @@ else [[ ! -f "${BYRON_GENESIS_JSON}" ]] && echo "Byron genesis file not found: ${BYRON_GENESIS_JSON}" && return 1 [[ ! -f "${GENESIS_JSON}" ]] && echo "Shelley genesis file not found: ${GENESIS_JSON}" && return 1 PROTOCOL="${CONFIG_CONTENTS[3]}" + P2P_ENABLED="${CONFIG_CONTENTS[5]}" fi [[ -z ${EKG_TIMEOUT} ]] && EKG_TIMEOUT=3 @@ -974,8 +975,8 @@ if [[ -n "${CNODE_PID}" ]]; then fi node_version="$(${CCLI} version | head -1 | cut -d' ' -f2)" -if ! versionCheck "1.32.1" "${node_version}"; then - echo -e "\nGuild scripts has now been upgraded to support cardano-node 1.32.1 or higher (${node_version} found).\nPlease update cardano-node (note that you should ideally update your config too) or use tagged branches for older node version.\n\n" +if ! versionCheck "1.35.0" "${node_version}"; then + echo -e "\nGuild scripts has now been upgraded to support cardano-node 1.35.0 or higher (${node_version} found).\nPlease update cardano-node (note that you should ideally update your config too) or use tagged branches for older node version.\n\n" return 1 fi diff --git a/scripts/cnode-helper-scripts/gLiveView.sh b/scripts/cnode-helper-scripts/gLiveView.sh index ccdf210e4..91da33f1a 100755 --- a/scripts/cnode-helper-scripts/gLiveView.sh +++ b/scripts/cnode-helper-scripts/gLiveView.sh @@ -57,7 +57,7 @@ setTheme() { # Do NOT modify code below # ###################################### -GLV_VERSION=v1.26.5 +GLV_VERSION=v1.27.0 PARENT="$(dirname $0)" @@ -603,7 +603,6 @@ checkPeers() { ##################################### check_peers="false" show_peers="false" -p2p_enabled=$(jq -r '.EnableP2P //false' ${CONFIG} 2>/dev/null) getNodeMetrics curr_epoch=${epochnum} getShelleyTransitionEpoch @@ -687,7 +686,7 @@ while true; do if [[ ${show_peers} = "false" ]]; then - if [[ ${p2p_enabled} != true ]]; then + if [[ ${P2P_ENABLED} != true ]]; then if [[ ${use_lsof} = 'Y' ]]; then peers_in=$(lsof -Pnl +M | grep ESTABLISHED | awk -v pid="${CNODE_PID}" -v port=":${CNODE_PORT}->" '$2 == pid && $9 ~ port {print $9}' | awk -F "->" '{print $2}' | wc -l) peers_out=$(lsof -Pnl +M | grep ESTABLISHED | awk -v pid="${CNODE_PID}" -v port=":(${CNODE_PORT}|${EKG_PORT}|${PROM_PORT})->" '$2 == pid && $9 !~ port {print $9}' | awk -F "->" '{print $2}' | wc -l) @@ -970,7 +969,7 @@ while true; do echo "${conndivider}" && ((line++)) - if [[ ${p2p_enabled} = true ]]; then + if [[ ${P2P_ENABLED} = true ]]; then # row 1 printf "${VL} P2P : ${style_status_1}%-${three_col_2_value_width}s${NC}" "enabled" diff --git a/scripts/cnode-helper-scripts/prereqs.sh b/scripts/cnode-helper-scripts/prereqs.sh index fc9dacfa5..c07dd26e1 100755 --- a/scripts/cnode-helper-scripts/prereqs.sh +++ b/scripts/cnode-helper-scripts/prereqs.sh @@ -151,6 +151,8 @@ if [[ ${UPDATE_CHECK} = 'Y' ]] && curl -s -f -m ${CURL_TIMEOUT} -o "${PARENT}"/p fi rm -f "${PARENT}"/prereqs.sh.tmp +mkdir -p "${HOME}"/git > /dev/null 2>&1 # To hold git repositories that will be used for building binaries + if [[ "${INTERACTIVE}" = 'Y' ]]; then clear CNODE_PATH=$(get_input "Please enter the project path" ${CNODE_PATH}) @@ -206,7 +208,7 @@ if [ "$WANT_BUILD_DEPS" = 'Y' ]; then elif [[ "${VERSION_ID}" == "7" ]]; then #RHEL/CentOS7 pkg_list="${pkg_list} libusb pkgconfig srm" - elif [[ "${VERSION_ID}" =~ "8" ]]; then + elif [[ "${VERSION_ID}" =~ "8" ]] || [[ "${VERSION_ID}" =~ "9" ]]; then #RHEL/CentOS/RockyLinux8 pkg_opts="${pkg_opts} --allowerasing" pkg_list="${pkg_list} libusbx ncurses-compat-libs pkgconf-pkg-config" @@ -215,7 +217,7 @@ if [ "$WANT_BUILD_DEPS" = 'Y' ]; then pkg_opts="${pkg_opts} --allowerasing" pkg_list="${pkg_list} libusbx ncurses-compat-libs pkgconf-pkg-config srm" fi - ! grep -q ^epel <<< "$(yum repolist)" && $sudo yum ${pkg_opts} install https://dl.fedoraproject.org/pub/epel/epel-release-latest-"$(grep ^VERSION_ID /etc/os-release | cut -d\" -f2)".noarch.rpm > /dev/null + ! grep -q ^epel <<< "$(yum repolist)" && $sudo yum ${pkg_opts} install https://dl.fedoraproject.org/pub/epel/epel-release-latest-"${VERSION_ID}".noarch.rpm > /dev/null $sudo yum ${pkg_opts} install ${pkg_list} > /dev/null;rc=$? if [ $rc != 0 ]; then echo "An error occurred while installing the prerequisite packages, please investigate by using the command below:" @@ -249,6 +251,20 @@ if [ "$WANT_BUILD_DEPS" = 'Y' ]; then echo "CentOS: curl pkgconfig libffi-devel gmp-devel openssl-devel ncurses-libs ncurses-compat-libs systemd-devel zlib-devel tmux procps-ng" err_exit fi + echo "Install libsecp256k1 ... " + if ! grep -q "/usr/local/lib:\$LD_LIBRARY_PATH" "${HOME}"/.bashrc; then + echo "export LD_LIBRARY_PATH=/usr/local/lib:\$LD_LIBRARY_PATH" >> "${HOME}"/.bashrc + export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH + fi + pushd "${HOME}"/git >/dev/null || err_exit + [[ ! -d "./secp256k1" ]] && git clone https://github.com/bitcoin-core/secp256k1 &>/dev/null + pushd secp256k1 >/dev/null || err_exit + git checkout ac83be33 &>/dev/null + ./autogen.sh > autogen.log > /tmp/secp256k1.log 2>&1 + ./configure --prefix=/usr --enable-module-schnorrsig --enable-experimental > configure.log >> /tmp/secp256k1.log 2>&1 + make > make.log 2>&1 + make check >>make.log 2>&1 + $sudo make install > install.log 2>&1 export BOOTSTRAP_HASKELL_NO_UPGRADE=1 export BOOTSTRAP_HASKELL_GHC_VERSION=8.10.7 export BOOTSTRAP_HASKELL_CABAL_VERSION=3.6.2.0 @@ -295,8 +311,6 @@ else fi -mkdir -p "${HOME}"/git > /dev/null 2>&1 # To hold git repositories that will be used for building binaries - if [[ "${LIBSODIUM_FORK}" = "Y" ]]; then if ! grep -q "/usr/local/lib:\$LD_LIBRARY_PATH" "${HOME}"/.bashrc; then echo "export LD_LIBRARY_PATH=/usr/local/lib:\$LD_LIBRARY_PATH" >> "${HOME}"/.bashrc @@ -324,6 +338,7 @@ if [[ "${INSTALL_CNCLI}" = "Y" ]]; then if ! output=$(git clone https://github.com/cardano-community/cncli.git 2>&1); then echo -e "${output}" && err_exit; fi fi pushd ./cncli >/dev/null || err_exit + git remote set-url origin https://github.com/cardano-community/cncli >/dev/null if ! output=$(git fetch --all --prune 2>&1); then echo -e "${output}" && err_exit; fi cncli_git_latestTag=$(git describe --tags "$(git rev-list --tags --max-count=1)") if ! output=$(git checkout ${cncli_git_latestTag} 2>&1 && git submodule update --init --recursive --force 2>&1); then echo -e "${output}" && err_exit; fi @@ -445,6 +460,7 @@ fi # Download dbsync config curl -sL -f -m ${CURL_TIMEOUT} -o dbsync.json.tmp ${URL_RAW}/files/config-dbsync.json +[[ "${NETWORK}" != "mainnet" ]] && sed -i 's#NetworkName": "mainnet"#NetworkName": "testnet"#g' dbsync.json.tmp # Download node config, genesis and topology from template if [[ ${NETWORK} = "guild" ]]; then diff --git a/scripts/cnode-helper-scripts/topologyUpdater.sh b/scripts/cnode-helper-scripts/topologyUpdater.sh index fa31a8730..0c555c230 100755 --- a/scripts/cnode-helper-scripts/topologyUpdater.sh +++ b/scripts/cnode-helper-scripts/topologyUpdater.sh @@ -129,39 +129,44 @@ if [[ ${TU_PUSH} = "Y" ]]; then curl -s -f -6 "https://api.clio.one/htopology/v1/?port=${CNODE_PORT}&blockNo=${blockNo}&valency=${CNODE_VALENCY}&magic=${NWMAGIC}${T_HOSTNAME}" | tee -a "${LOG_DIR}"/topologyUpdater_lastresult.json fi fi + if [[ ${TU_FETCH} = "Y" ]]; then - if [[ ${IP_VERSION} = "4" || ${IP_VERSION} = "mix" ]]; then - curl -s -f -4 -o "${TOPOLOGY}".tmp "https://api.clio.one/htopology/v1/fetch/?max=${MAX_PEERS}&magic=${NWMAGIC}&ipv=${IP_VERSION}" + if [[ ${P2P_ENABLED} = "true" ]]; then + echo "INFO: Skipping the TU fetch request because the node is running in P2P mode" else - curl -s -f -6 -o "${TOPOLOGY}".tmp "https://api.clio.one/htopology/v1/fetch/?max=${MAX_PEERS}&magic=${NWMAGIC}&ipv=${IP_VERSION}" - fi - [[ ! -s "${TOPOLOGY}".tmp ]] && echo "ERROR: The downloaded file is empty!" && exit 1 - if [[ -n "${CUSTOM_PEERS}" ]]; then - topo="$(cat "${TOPOLOGY}".tmp)" - IFS='|' read -ra cpeers <<< "${CUSTOM_PEERS}" - for cpeer in "${cpeers[@]}"; do - IFS=',' read -ra cpeer_attr <<< "${cpeer}" - case ${#cpeer_attr[@]} in - 2) addr="${cpeer_attr[0]}" - port=${cpeer_attr[1]} - valency=1 ;; - 3) addr="${cpeer_attr[0]}" - port=${cpeer_attr[1]} - valency=${cpeer_attr[2]} ;; - *) echo "ERROR: Invalid Custom Peer definition '${cpeer}'. Please double check CUSTOM_PEERS definition" - exit 1 ;; - esac - if [[ ${addr} = *.* ]]; then - ! isValidIPv4 "${addr}" && echo "ERROR: Invalid IPv4 address or hostname '${addr}'. Please check CUSTOM_PEERS definition" && continue - elif [[ ${addr} = *:* ]]; then - ! isValidIPv6 "${addr}" && echo "ERROR: Invalid IPv6 address '${addr}'. Please check CUSTOM_PEERS definition" && continue - fi - ! isNumber ${port} && echo "ERROR: Invalid port number '${port}'. Please check CUSTOM_PEERS definition" && continue - ! isNumber ${valency} && echo "ERROR: Invalid valency number '${valency}'. Please check CUSTOM_PEERS definition" && continue - topo=$(jq '.Producers += [{"addr": $addr, "port": $port|tonumber, "valency": $valency|tonumber}]' --arg addr "${addr}" --arg port ${port} --arg valency ${valency} <<< "${topo}") - done - echo "${topo}" | jq -r . >/dev/null 2>&1 && echo "${topo}" > "${TOPOLOGY}".tmp + if [[ ${IP_VERSION} = "4" || ${IP_VERSION} = "mix" ]]; then + curl -s -f -4 -o "${TOPOLOGY}".tmp "https://api.clio.one/htopology/v1/fetch/?max=${MAX_PEERS}&magic=${NWMAGIC}&ipv=${IP_VERSION}" + else + curl -s -f -6 -o "${TOPOLOGY}".tmp "https://api.clio.one/htopology/v1/fetch/?max=${MAX_PEERS}&magic=${NWMAGIC}&ipv=${IP_VERSION}" + fi + [[ ! -s "${TOPOLOGY}".tmp ]] && echo "ERROR: The downloaded file is empty!" && exit 1 + if [[ -n "${CUSTOM_PEERS}" ]]; then + topo="$(cat "${TOPOLOGY}".tmp)" + IFS='|' read -ra cpeers <<< "${CUSTOM_PEERS}" + for cpeer in "${cpeers[@]}"; do + IFS=',' read -ra cpeer_attr <<< "${cpeer}" + case ${#cpeer_attr[@]} in + 2) addr="${cpeer_attr[0]}" + port=${cpeer_attr[1]} + valency=1 ;; + 3) addr="${cpeer_attr[0]}" + port=${cpeer_attr[1]} + valency=${cpeer_attr[2]} ;; + *) echo "ERROR: Invalid Custom Peer definition '${cpeer}'. Please double check CUSTOM_PEERS definition" + exit 1 ;; + esac + if [[ ${addr} = *.* ]]; then + ! isValidIPv4 "${addr}" && echo "ERROR: Invalid IPv4 address or hostname '${addr}'. Please check CUSTOM_PEERS definition" && continue + elif [[ ${addr} = *:* ]]; then + ! isValidIPv6 "${addr}" && echo "ERROR: Invalid IPv6 address '${addr}'. Please check CUSTOM_PEERS definition" && continue + fi + ! isNumber ${port} && echo "ERROR: Invalid port number '${port}'. Please check CUSTOM_PEERS definition" && continue + ! isNumber ${valency} && echo "ERROR: Invalid valency number '${valency}'. Please check CUSTOM_PEERS definition" && continue + topo=$(jq '.Producers += [{"addr": $addr, "port": $port|tonumber, "valency": $valency|tonumber}]' --arg addr "${addr}" --arg port ${port} --arg valency ${valency} <<< "${topo}") + done + echo "${topo}" | jq -r . >/dev/null 2>&1 && echo "${topo}" > "${TOPOLOGY}".tmp + fi + mv "${TOPOLOGY}".tmp "${TOPOLOGY}" fi - mv "${TOPOLOGY}".tmp "${TOPOLOGY}" fi exit 0 diff --git a/scripts/grest-helper-scripts/getmetrics.sh b/scripts/grest-helper-scripts/getmetrics.sh index cec740721..14b7c0aea 100755 --- a/scripts/grest-helper-scripts/getmetrics.sh +++ b/scripts/grest-helper-scripts/getmetrics.sh @@ -63,7 +63,7 @@ function get-metrics() { export METRIC_grestschsize="${grestschsize}" export METRIC_dbsize="${dbsize}" #export METRIC_cnodeversion="$(echo $(cardano-node --version) | awk '{print $2 "-" $9}')" - #export METRIC_dbsyncversion="$(echo $(cardano-db-sync-extended --version) | awk '{print $2 "-" $9}')" + #export METRIC_dbsyncversion="$(echo $(cardano-db-sync --version) | awk '{print $2 "-" $9}')" #export METRIC_psqlversion="$(echo "" | psql cexplorer -c "SELECT version();" | grep PostgreSQL | awk '{print $2}')" for metric_var_name in $(env | grep ^METRIC | sort | awk -F= '{print $1}')