Updates for Vasil HF (non-grest) (#1433)

- [x] Op-Cert to increment using previous block + 1 (or prompt for input) - [x] Update prereqs to include libsecp256k1 - [x] Retire cardano-db-sync-extended to use cardano-db-sync instead - [x] Add CNCLI v5 flag for consensus, update remote repo URL to `cardano-community` org account - [x] Docs update for cncli, node, CNTools and gLV - [x] Topology Updater to skip fetch if P2P is enabled. - [x] Updates to protocol-parameter references - [x] Fix workflow tests Future To-Do: - Update tx fee handling changes in babbage era (currently there are places that use fallback of 1 ADA as minUtxo)
cardano-community · Jun 28, 2022 · 813c7a0 · 813c7a0
1 parent d53c586
commit 813c7a0
Show file tree

Hide file tree

Showing 22 changed files with 222 additions and 108 deletions.
diff --git a/docs/Build/node-cli.md b/docs/Build/node-cli.md
@@ -35,11 +35,11 @@ Execute `cardano-cli` and `cardano-node` to verify output as below (the exact ve
 
 ```bash
 cardano-cli version
-# cardano-cli 1.32.1 - linux-x86_64 - ghc-8.10
-# git rev 4f65fb9a27aa7e3a1873ab4211e412af780a3648
+# cardano-cli 1.35.0 - linux-x86_64 - ghc-8.10
+# git rev <...>
 cardano-node version
-# cardano-node 1.32.1 - linux-x86_64 - ghc-8.10
-# git rev 4f65fb9a27aa7e3a1873ab4211e412af780a3648
+# cardano-node 1.35.0 - linux-x86_64 - ghc-8.10
+# git rev <...>
 ```
 
 #### Update port number or pool name for relative paths

diff --git a/docs/Scripts/cncli.md b/docs/Scripts/cncli.md
@@ -1,7 +1,7 @@
 !!! info "Reminder !!"
     Ensure the [Pre-Requisites](../basics.md#pre-requisites) are in place before you proceed.
 
-`cncli.sh` is a script to download and deploy [CNCLI](https://github.com/AndrewWestberg/cncli) created and maintained by Andrew Westberg. It's a community-based CLI tool written in RUST for low-level `cardano-node` communication. Usage is **optional** and no script is dependent on it. The main features include:
+`cncli.sh` is a script to download and deploy [CNCLI](https://github.com/cardano-community/cncli) created and maintained by Andrew Westberg. It's a community-based CLI tool written in RUST for low-level `cardano-node` communication. Usage is **optional** and no script is dependent on it. The main features include:
 
 - **PING** - Validates that the remote server is on the given network and returns its response time. Utilized by `gLiveView` for peer analysis if available. 
 - **SYNC** - Connects to a node (local or remote) and synchronizes blocks to a local `sqlite` database. 

diff --git a/docs/Scripts/cntools-changelog.md b/docs/Scripts/cntools-changelog.md
@@ -6,6 +6,17 @@ All notable changes to this tool will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [10.0.0] - 2022-06-28
+#### Added
+- Support for Vasil Fork
+- Preliminary support for Post HF updates (a short release will follow post fork in coming days)
+- Minimum version for Node bumped to 1.35.0
+
+#### Changed
+- Pool > Rotate code now uses kes-periodinfo CLI query to get counter from node (fallback for Koios)
+- Pool > Show Info updated to include current KES counter
+- Update getEraIdentifier to include Babbage era
+
 ## [9.1.0] - 2022-05-11
 #### Changed
 - Harmonize flow for reusing old wallet configuration on pool modification vs setting new wallets.

diff --git a/docs/Scripts/gliveview.md b/docs/Scripts/gliveview.md
@@ -56,6 +56,12 @@ Displays live metrics from cardano-node gathered through the nodes EKG/Prometheu
 - **Tip (diff) / Status** - Will either show node status as `starting|sync xx.x%` or if close to reference tip, the tip difference `Tip (ref) - Tip (node)` to see how far of the tip (diff value) the node is. With current parameters a slot diff up to 40 from reference tip is considered good but it should usually stay below 30. It's perfectly normal to see big differences in slots between blocks. It's the built in randomness at play. To see if a node is really healthy and staying on tip you would need to compare the tip between multiple nodes.  
 - **Forks** - The number of forks since node start. Each fork means the blockchain evolved in a different direction, thereby discarding blocks. A high number of forks means there is a higher chance of orphaned blocks.  
 - **Peers In / Out** - Shows how many connections the node has established in and out. See [Peer analysis](#peer-analysis) section for how to get more details of incoming and outgoing connections.  
+- **P2P Mode** 
+  - `Cold` peers indicate the number of inactive but known peers to the node.
+  - `Warm` peers tell how many established connections the node has.
+  - `Hot` peers how many established connections are actually active.
+  - `Bi-Dir`(bidirectional) and `Uni-Dir`(unidirectional) indicate how the handshake protocol negotiated the connection. The connection between p2p nodes will always be bidirectional, but it will be unidirectional between p2p nodes and non-p2p nodes. 
+  - `Duplex` shows the connections that are actually used in both directions, only bidirectional connections have this potential.
 - **Mem (RSS)** - RSS is the Resident Set Size and shows how much memory is allocated to cardano-node and that is in RAM. It does not include memory that is swapped out. It does include memory from shared libraries as long as the pages from those libraries are actually in memory. It does include all stack and heap memory.  
 - **Mem (Live) / (Heap)** - GC (Garbage Collector) values that show how much memory is used for live/heap data. A large difference between them (or the heap approaching the physical memory limit) means the node is struggling with the garbage collector and/or may begin swapping.  
 - **GC Minor / Major** - Collecting garbage from "Young space" is called a Minor GC. Major (Full) GC is done more rarily and is a more expensive operation. Explaining garbage collection is a topic outside the scope of this documentation and google is your friend for this.  

diff --git a/files/docker/grest/scripts/docker-getmetrics.sh b/files/docker/grest/scripts/docker-getmetrics.sh
@@ -119,7 +119,7 @@ function get-metrics() {
   export METRIC_grestschsize="${grestschsize}"
   export METRIC_dbsize="${dbsize}"
   #export METRIC_cnodeversion="$(echo $(cardano-node --version) | awk '{print $2 "-" $9}')"
-  #export METRIC_dbsyncversion="$(echo $(cardano-db-sync-extended --version) | awk '{print $2 "-" $9}')"
+  #export METRIC_dbsyncversion="$(echo $(cardano-db-sync --version) | awk '{print $2 "-" $9}')"
   #export METRIC_psqlversion="$(echo "" | psql -U "${POSTGRES_USER}" -d "${POSTGRES_DB}" -c "SELECT version();" | grep PostgreSQL | awk '{print $2}')"
 
   for metric_var_name in $(env | grep ^METRIC | sort | awk -F= '{print $1}')

diff --git a/files/tests/pre-merge/amazonlinux2-cabal.containerfile b/files/tests/pre-merge/amazonlinux2-cabal.containerfile
@@ -6,10 +6,11 @@ ARG CNODE_HOME=/opt/cardano/cnode
 ENV \
   LANG=C.UTF-8 \
   USER=root \
-  PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH
+  PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH \
+  LD_LIBRARY_PATH=/usr/lib:/usr/lib64:/usr/local/lib:$LD_LIBRARY_PATH
 
 RUN git clone https://github.com/input-output-hk/cardano-node &&\
-    pwd ; ls -l
+    pwd
 
 WORKDIR /cardano-node
 
@@ -21,4 +22,4 @@ RUN curl -o cardano-node-latest.txt "https://raw.githubusercontent.com/cardano-c
     git status &&\
     /opt/cardano/cnode/scripts/cabal-build-all.sh &&\
     cabal install cardano-ping &&\
-    /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version
+    /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version
diff --git a/files/tests/pre-merge/amazonlinux2-cabal_-l.containerfile b/files/tests/pre-merge/amazonlinux2-cabal_-l.containerfile
@@ -6,10 +6,11 @@ ARG CNODE_HOME=/opt/cardano/cnode
 ENV \
   LANG=C.UTF-8 \
   USER=root \
-  PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH
+  PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH \
+  LD_LIBRARY_PATH=/usr/lib:/usr/lib64:/usr/local/lib:$LD_LIBRARY_PATH
 
 RUN git clone https://github.com/input-output-hk/cardano-node &&\
-    pwd ; ls -l
+    pwd
 
 WORKDIR /cardano-node
 
@@ -21,4 +22,4 @@ RUN curl -o cardano-node-latest.txt "https://raw.githubusercontent.com/cardano-c
     git status &&\
     /opt/cardano/cnode/scripts/cabal-build-all.sh -l &&\
     cabal install cardano-ping &&\
-    /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version
+    /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version
diff --git a/files/tests/pre-merge/debian-cabal.containerfile b/files/tests/pre-merge/debian-cabal.containerfile
@@ -10,7 +10,7 @@ ENV \
   PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH
 
 RUN git clone https://github.com/input-output-hk/cardano-node &&\
-    pwd ; ls -l
+    pwd
 
 WORKDIR /cardano-node
 

diff --git a/files/tests/pre-merge/debian-cabal_-l.containerfile b/files/tests/pre-merge/debian-cabal_-l.containerfile
@@ -10,7 +10,7 @@ ENV \
   PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH
 
 RUN git clone https://github.com/input-output-hk/cardano-node &&\
-    pwd ; ls -l
+    pwd
 
 WORKDIR /cardano-node
 

diff --git a/files/tests/pre-merge/rockylinux8-cabal.containerfile b/files/tests/pre-merge/rockylinux8-cabal.containerfile
@@ -9,7 +9,7 @@ ENV \
   PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH
 
 RUN git clone https://github.com/input-output-hk/cardano-node &&\
-    pwd ; ls -l
+    pwd
 
 WORKDIR /cardano-node
 
@@ -21,4 +21,4 @@ RUN curl -o cardano-node-latest.txt "https://raw.githubusercontent.com/cardano-c
     git status &&\
     /opt/cardano/cnode/scripts/cabal-build-all.sh &&\
     cabal install cardano-ping &&\
-    /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version
+    /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version
diff --git a/files/tests/pre-merge/rockylinux8-cabal_-l.containerfile b/files/tests/pre-merge/rockylinux8-cabal_-l.containerfile
@@ -9,7 +9,7 @@ ENV \
   PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH
 
 RUN git clone https://github.com/input-output-hk/cardano-node &&\
-    pwd ; ls -l
+    pwd
 
 WORKDIR /cardano-node
 
@@ -21,4 +21,4 @@ RUN curl -o cardano-node-latest.txt "https://raw.githubusercontent.com/cardano-c
     git status &&\
     /opt/cardano/cnode/scripts/cabal-build-all.sh -l &&\
     cabal install cardano-ping &&\
-    /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version
+    /root/.cabal/bin/cardano-cli version ; /root/.cabal/bin/cardano-node version
diff --git a/files/tests/pre-merge/ubuntu20-cabal.containerfile b/files/tests/pre-merge/ubuntu20-cabal.containerfile
@@ -10,7 +10,7 @@ ENV \
   PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH
 
 RUN git clone https://github.com/input-output-hk/cardano-node &&\
-    pwd ; ls -l
+    pwd
 
 WORKDIR /cardano-node
 

diff --git a/files/tests/pre-merge/ubuntu20-cabal_-l.containerfile b/files/tests/pre-merge/ubuntu20-cabal_-l.containerfile
@@ -10,7 +10,7 @@ ENV \
   PATH=$CNODE_HOME/scripts:/root/.cabal/bin:/root/.ghcup/bin:$PATH
 
 RUN git clone https://github.com/input-output-hk/cardano-node &&\
-    pwd ; ls -l
+    pwd
 
 WORKDIR /cardano-node
 

diff --git a/scripts/cnode-helper-scripts/cabal-build-all.sh b/scripts/cnode-helper-scripts/cabal-build-all.sh
@@ -6,22 +6,42 @@
 echo "Deleting build config artifact to remove cached version, this prevents invalid Git Rev"
 find dist-newstyle/build/x86_64-linux/ghc-8.10.?/cardano-config-* >/dev/null 2>&1 && rm -rf "dist-newstyle/build/x86_64-linux/ghc-8.*/cardano-config-*"
 
+[[ -f /usr/lib/libsecp256k1.so ]] && export LD_LIBRARY_PATH=/usr/lib:"${LD_LIBRARY_PATH}"
+[[ -f /usr/lib64/libsecp256k1.so ]] && export LD_LIBRARY_PATH=/usr/lib64:"${LD_LIBRARY_PATH}"
+[[ -f /usr/local/lib/libsecp256k1.so ]] && export LD_LIBRARY_PATH=/usr/local/lib:"${LD_LIBRARY_PATH}"
+[[ -d /usr/lib/pkgconfig ]] && export PKG_CONFIG_PATH=/usr/lib/pkgconfig:"${PKG_CONFIG_PATH}"
+[[ -d /usr/lib64/pkgconfig ]] && export PKG_CONFIG_PATH=/usr/lib64/pkgconfig:"${PKG_CONFIG_PATH}"
+[[ -d /usr/local/lib/pkgconfig ]] && export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:"${PKG_CONFIG_PATH}"
+
 if [[ "$1" == "-l" ]] ; then
   USE_SYSTEM_LIBSODIUM="package cardano-crypto-praos
     flags: -external-libsodium-vrf"
-  # In case Custom libsodium module is present, exclude it from Load Library Path
-  [[ -f /usr/local/lib/libsodium.so ]] && export LD_LIBRARY_PATH=${LD_LIBRARY_PATH/\/usr\/local\/lib:/}
 else
   unset USE_SYSTEM_LIBSODIUM
-  source "${HOME}"/.bashrc
-  [[ -d /usr/local/lib/pkgconfig ]] && export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:"${PKG_CONFIG_PATH}"
-  [[ -f /usr/local/lib/libsodium.so ]] && export LD_LIBRARY_PATH=/usr/local/lib:"${LD_LIBRARY_PATH}"
 fi
 
 [[ -f cabal.project.local ]] && mv cabal.project.local cabal.project.local.bkp_"$(date +%s)"
 cat <<-EOF > .tmp.cabal.project.local
 	${USE_SYSTEM_LIBSODIUM}
 	
+	source-repository-package
+	    type: git
+	    location: https://github.com/input-output-hk/hjsonpointer
+	    tag: bb99294424e0c5b3c2942c743b545e4b01c12ce8
+	    --sha256: 11z5s4xmm6cxy6sdcf9ly4lr0qh3c811hpm0bmlp4c3yq8v3m9rk
+	
+	source-repository-package
+	    type: git
+	    location: https://github.com/input-output-hk/hjsonschema
+	    tag: 1546af7fc267d5eea805ef223dd2b59ac120b784
+	    --sha256: 0sdikhsq6nnhmmvcpwzzrwfn9cn7rg75629qnrx01i5vm5ci4314
+	
+	source-repository-package
+	    type: git
+	    location: https://github.com/haskell-works/hw-aeson
+	    tag: d99d2f3e39a287607418ae605b132a3deb2b753f
+	    --sha256: 1vxqcwjg9q37wbwi27y9ba5163lzfz51f1swbi0rp681yg63zvn4
+	
 	source-repository-package
 	  type: git
 	  location: https://github.com/input-output-hk/bech32
@@ -31,34 +51,40 @@ cat <<-EOF > .tmp.cabal.project.local
 	source-repository-package
 	  type: git
 	  location: https://github.com/input-output-hk/cardano-addresses
-	  tag: 71006f9eb956b0004022e80aadd4ad50d837b621
+	  tag: b6f2f3cef01a399376064194fd96711a5bdba4a7
 	  subdir:
 	    command-line
 	    core
 	
+	allow-newer:
+	  *:aeson
+	
 	EOF
 chmod 640 .tmp.cabal.project.local
 
-if [[ -z "${USE_SYSTEM_LIBSODIUM}" ]] ; then
-  echo "Running cabal update to ensure you're on latest dependencies.."
-  cabal update 2>&1 | tee /tmp/cabal-update.log
-  echo "Building.."
-  cabal build all 2>&1 | tee tee /tmp/build.log
+echo "Running cabal update to ensure you're on latest dependencies.."
+cabal update 2>&1 | tee /tmp/cabal-update.log
+echo "Building.."
+
+if [[ -z "${USE_SYSTEM_LIBSODIUM}" ]] ; then # Build using default cabal.project first and then add cabal.project.local for additional packages
   if [[ "${PWD##*/}" == "cardano-node" ]] || [[ "${PWD##*/}" == "cardano-db-sync" ]]; then
-    echo "Overwriting cabal.project.local to include cardano-addresses and bech32 .."
+    #cabal install cardano-crypto-class --disable-tests --disable-profiling | tee /tmp/build.log
+    [[ "${PWD##*/}" == "cardano-node" ]] && cabal build cardano-node cardano-cli cardano-submit-api --disable-tests --disable-profiling | tee /tmp/build.log
+    [[ "${PWD##*/}" == "cardano-db-sync" ]] && cabal build cardano-db-sync --disable-tests --disable-profiling | tee /tmp/build.log
     mv .tmp.cabal.project.local cabal.project.local
-    cabal install bech32 cardano-addresses-cli  --overwrite-policy=always 2>&1 | tee /tmp/build-b32-caddr.log
+    cabal install bech32 cardano-addresses-cli cardano-ping --overwrite-policy=always 2>&1 | tee /tmp/build-b32-caddr.log
+  else
+    cabal build all --disable-tests --disable-profiling 2>&1 | tee /tmp/build.log
   fi
-else
+else # Add cabal.project.local customisations first before building
   if [[ "${PWD##*/}" == "cardano-node" ]] || [[ "${PWD##*/}" == "cardano-db-sync" ]]; then
-    echo "Overwriting cabal.project.local to include cardano-addresses and bech32 .."
     mv .tmp.cabal.project.local cabal.project.local
+    [[ "${PWD##*/}" == "cardano-node" ]] && cabal build cardano-node cardano-cli cardano-submit-api --disable-tests --disable-profiling | tee /tmp/build.log
+    [[ "${PWD##*/}" == "cardano-db-sync" ]] && cabal build cardano-db-sync --disable-tests --disable-profiling | tee /tmp/build.log
+  else
+    cabal build all --disable-tests --disable-profiling 2>&1 | tee /tmp/build.log
   fi
-  echo "Running cabal update to ensure you're on latest dependencies.."
-  cabal update 2>&1 | tee /tmp/cabal-update.log
-  echo "Building.."
-  cabal build all 2>&1 | tee tee /tmp/build.log
-  [[ -f cabal.project.local ]] && cabal install bech32 cardano-addresses-cli  --overwrite-policy=always 2>&1 | tee /tmp/build-b32-caddr.log
+  [[ -f cabal.project.local ]] && cabal install bech32 cardano-ping cardano-addresses-cli --overwrite-policy=always 2>&1 | tee /tmp/build-b32-caddr.log
 fi
 
 grep "^Linking" /tmp/build.log | grep -Ev 'test|golden|demo|chairman|locli|ledger|topology' | while read -r line ; do