Chore/merge 2.4.0.1.0 to develop

CHANGELOG.md

@@ -5,9 +5,24 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to the versioning scheme outlined in the [README.md](README.md).
 
-## [2.4.0.0.5]
+## [2.4.0.1.0]
 
-This introduces a set of improvements to the Stacks miner behavior.  In
+### Added
+
+- When the Clarity library is built with feature flag `developer-mode`, comments
+  from the source code are now attached to the `SymbolicExpression` nodes. This
+  will be useful for tools that use the Clarity library to analyze and
+  manipulate Clarity source code, e.g. a formatter.
+- New RPC endpoint at /v2/constant_val to fetch a constant from a contract.
+- A new subsystem, called StackerDB, has been added, which allows a set of
+  Stacks nodes to store off-chain data on behalf of a specially-crafter smart
+  contract. This is an opt-in feature; Stacks nodes explicitly subscribe to
+  StackerDB replicas in their config files.
+- Message definitions and codecs for Stacker DB, a replicated off-chain DB
+  hosted by subscribed Stacks nodes and controlled by smart contracts
+- Added 3 new public and regionally diverse bootstrap nodes: est.stacksnodes.org, cet.stacksnodes.org, sgt.stacksnodes.org
+
+In addition, this introduces a set of improvements to the Stacks miner behavior.  In
 particular:
 * The VRF public key can be re-used across node restarts.
 * Settings that affect mining are hot-reloaded from the config file.  They take
@@ -26,6 +41,22 @@ contract-calls).
 * When configured, the node will optionally only RBF block-commits if it can
   produce a block with strictly more transactions.
 
+### Changed
+
+- `developer-mode` is no longer enabled in the default feature set. This is the correct default behavior, since the stacks-node should NOT build with developer-mode enabled by default. Tools that need to use developer-mode should enable it explicitly.
+
+### Fixed
+
+- The transaction receipts for smart contract publish transactions now indicate
+  a result of `(err none)` if the top-level code of the smart contract contained
+  runtime error and include details about the error in the `vm_error` field of
+  the receipt. Fixes issues #3154, #3328.
+- Added config setting `burnchain.wallet_name` which addresses blank wallets no
+  longer being created by default in recent bitcoin versions. Fixes issue #3596
+- Use the current burnchain tip to lookup UTXOs (Issue #3733)
+- The node now gracefully shuts down even if it is in the middle of a handshake with
+  bitcoind. Fixes issue #3734.
+
 ## [2.4.0.0.4]
 
 This is a high-priority hotfix that addresses a bug in transaction processing which
@@ -56,41 +87,6 @@ could impact miner availability.
 This is a hotfix that changes the logging failure behavior from panicking to dropping
 the log message (PR #3784).
 
-## [Unreleased]
-
-### Added
-
-- When the Clarity library is built with feature flag `developer-mode`, comments
-  from the source code are now attached to the `SymbolicExpression` nodes. This
-  will be useful for tools that use the Clarity library to analyze and
-  manipulate Clarity source code, e.g. a formatter.
-- New RPC endpoint at /v2/constant_val to fetch a constant from a contract.
-- A new subsystem, called StackerDB, has been added, which allows a set of
-  Stacks nodes to store off-chain data on behalf of a specially-crafter smart
-  contract. This is an opt-in feature; Stacks nodes explicitly subscribe to
-  StackerDB replicas in their config files.
-- Message definitions and codecs for Stacker DB, a replicated off-chain DB
-  hosted by subscribed Stacks nodes and controlled by smart contracts
-- Added 3 new public and regionally diverse bootstrap nodes: est.stacksnodes.org, cet.stacksnodes.org, sgt.stacksnodes.org
-- satoshis_per_byte can be changed in the config file and miners will always use
-  the most up to date value
-
-### Changed
-
-- `developer-mode` is no longer enabled in the default feature set. This is the correct default behavior, since the stacks-node should NOT build with developer-mode enabled by default. Tools that need to use developer-mode should enable it explicitly.
-
-### Fixed
-
-- The transaction receipts for smart contract publish transactions now indicate
-  a result of `(err none)` if the top-level code of the smart contract contained
-  runtime error and include details about the error in the `vm_error` field of
-  the receipt. Fixes issues #3154, #3328.
-- Added config setting `burnchain.wallet_name` which addresses blank wallets no
-  longer being created by default in recent bitcoin versions. Fixes issue #3596
-- Use the current burnchain tip to lookup UTXOs (Issue #3733)
-- The node now gracefully shuts down even if it is in the middle of a handshake with
-  bitcoind. Fixes issue #3734.
-
 ## [2.4.0.0.1]
 
 This is a minor change to add `txid` fields into the log messages from failing

build-scripts/Dockerfile.linux-glibc-arm64

@@ -18,7 +18,7 @@ RUN --mount=type=tmpfs,target=${BUILD_DIR} cp -R /src/. ${BUILD_DIR}/ \
     && CC=aarch64-linux-gnu-gcc \
     CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc \
     CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc \
-    cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} \
+    cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} --bin stacks-node --bin stacks-inspect --bin clarity-cli --bin blockstack-cli \
     && mkdir -p /out \
     && cp -R ${BUILD_DIR}/target/${TARGET}/release/. /out
 

build-scripts/Dockerfile.linux-glibc-armv7

@@ -18,9 +18,9 @@ RUN --mount=type=tmpfs,target=${BUILD_DIR} cp -R /src/. ${BUILD_DIR}/ \
     && CC=arm-linux-gnueabihf-gcc \
     CC_armv7_unknown_linux_gnueabihf=arm-linux-gnueabihf-gcc \
     CARGO_TARGET_ARMV7_UNKNOWN_LINUX_GNUEABIHF_LINKER=arm-linux-gnueabihf-gcc \
-    cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} \
+    cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} --bin stacks-node --bin stacks-inspect --bin clarity-cli --bin blockstack-cli \
     && mkdir -p /out \
     && cp -R ${BUILD_DIR}/target/${TARGET}/release/. /out
 
 FROM scratch AS export-stage
-COPY --from=build /out/stacks-inspect /out/blockstack-cli /out/clarity-cli /out/stacks-node /
+COPY --from=build /out/stacks-inspect /out/blockstack-cli /out/clarity-cli /out/stacks-node /

build-scripts/Dockerfile.linux-glibc-x64

@@ -15,7 +15,7 @@ RUN apt-get update && apt-get install -y git
 RUN --mount=type=tmpfs,target=${BUILD_DIR} cp -R /src/. ${BUILD_DIR}/ \
     && cd ${BUILD_DIR} \
     && rustup target add ${TARGET} \
-    && cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} \
+    && cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} --bin stacks-node --bin stacks-inspect --bin clarity-cli --bin blockstack-cli \
     && mkdir -p /out \
     && cp -R ${BUILD_DIR}/target/${TARGET}/release/. /out
 

build-scripts/Dockerfile.linux-musl-arm64

@@ -13,9 +13,10 @@ COPY . .
 RUN --mount=type=tmpfs,target=${BUILD_DIR} cp -R /src/. ${BUILD_DIR}/ \
     && cd ${BUILD_DIR} \
     && rustup target add ${TARGET} \
-    && cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} \
+    && cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} --bin stacks-node --bin stacks-inspect --bin clarity-cli --bin blockstack-cli \
     && mkdir -p /out \
     && cp -R ${BUILD_DIR}/target/${TARGET}/release/. /out
 
 FROM scratch AS export-stage
 COPY --from=build /out/stacks-inspect /out/blockstack-cli /out/clarity-cli /out/stacks-node /
+

build-scripts/Dockerfile.linux-musl-armv7

@@ -13,9 +13,9 @@ COPY . .
 RUN --mount=type=tmpfs,target=${BUILD_DIR} cp -R /src/. ${BUILD_DIR}/ \
     && cd ${BUILD_DIR} \
     && rustup target add ${TARGET} \
-    && cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} \
+    && cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} --bin stacks-node --bin stacks-inspect --bin clarity-cli --bin blockstack-cli \
     && mkdir -p /out \
     && cp -R ${BUILD_DIR}/target/${TARGET}/release/. /out
 
 FROM scratch AS export-stage
-COPY --from=build /out/stacks-inspect /out/blockstack-cli /out/clarity-cli /out/stacks-node /
+COPY --from=build /out/stacks-inspect /out/blockstack-cli /out/clarity-cli /out/stacks-node /

build-scripts/Dockerfile.linux-musl-x64

@@ -15,9 +15,10 @@ RUN apk update && apk add git musl-dev
 RUN --mount=type=tmpfs,target=${BUILD_DIR} cp -R /src/. ${BUILD_DIR}/ \
     && cd ${BUILD_DIR} \
     && rustup target add ${TARGET} \
-    && cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} \
+    && cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} --bin stacks-node --bin stacks-inspect --bin clarity-cli --bin blockstack-cli \
     && mkdir -p /out \
     && cp -R ${BUILD_DIR}/target/${TARGET}/release/. /out
 
 FROM scratch AS export-stage
 COPY --from=build /out/stacks-inspect /out/blockstack-cli /out/clarity-cli /out/stacks-node /
+

build-scripts/Dockerfile.macos-arm64

@@ -21,9 +21,10 @@ RUN --mount=type=tmpfs,target=${BUILD_DIR} cp -R /src/. ${BUILD_DIR}/ \
     && cd ${BUILD_DIR} \
     && rustup target add ${TARGET} \
     && . /opt/osxcross/env-macos-aarch64 \
-    && cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} \
+    && cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} --bin stacks-node --bin stacks-inspect --bin clarity-cli --bin blockstack-cli \
     && mkdir -p /out \
     && cp -R ${BUILD_DIR}/target/${TARGET}/release/. /out
 
 FROM scratch AS export-stage
 COPY --from=build /out/stacks-inspect /out/blockstack-cli /out/clarity-cli /out/stacks-node /
+

build-scripts/Dockerfile.macos-x64

@@ -21,9 +21,10 @@ RUN --mount=type=tmpfs,target=${BUILD_DIR} cp -R /src/. ${BUILD_DIR}/ \
     && cd ${BUILD_DIR} \
     && rustup target add ${TARGET} \
     && . /opt/osxcross/env-macos-x86_64 \
-    && cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} \
+    && cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} --bin stacks-node --bin stacks-inspect --bin clarity-cli --bin blockstack-cli \
     && mkdir -p /out \
     && cp -R ${BUILD_DIR}/target/${TARGET}/release/. /out
 
 FROM scratch AS export-stage
 COPY --from=build /out/stacks-inspect /out/blockstack-cli /out/clarity-cli /out/stacks-node /
+

build-scripts/Dockerfile.windows-x64

@@ -17,9 +17,9 @@ RUN --mount=type=tmpfs,target=${BUILD_DIR} cp -R /src/. ${BUILD_DIR}/ \
     && rustup target add ${TARGET} \
     && CC_x86_64_pc_windows_gnu=x86_64-w64-mingw32-gcc \
     CARGO_TARGET_X86_64_PC_WINDOWS_GNU_LINKER=x86_64-w64-mingw32-gcc \
-    cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} \
+    cargo build --features monitoring_prom,slog_json --release --workspace --target ${TARGET} --bin stacks-node --bin stacks-inspect --bin clarity-cli --bin blockstack-cli \
     && mkdir -p /out \
     && cp -R ${BUILD_DIR}/target/${TARGET}/release/. /out
 
 FROM scratch AS export-stage
-COPY --from=build /out/stacks-inspect.exe /out/blockstack-cli.exe /out/clarity-cli.exe /out/stacks-node.exe /
+COPY --from=build /out/stacks-inspect.exe /out/blockstack-cli.exe /out/clarity-cli.exe /out/stacks-node.exe /

clarity/src/libclarity.rs

@@ -70,11 +70,13 @@ pub mod boot_util {
     use crate::vm::representations::ContractName;
     use crate::vm::types::QualifiedContractIdentifier;
 
+    #[allow(clippy::expect_used)]
     pub fn boot_code_id(name: &str, mainnet: bool) -> QualifiedContractIdentifier {
         let addr = boot_code_addr(mainnet);
         QualifiedContractIdentifier::new(
             addr.into(),
-            ContractName::try_from(name.to_string()).unwrap(),
+            ContractName::try_from(name.to_string())
+                .expect("FATAL: boot contract name is not a legal ContractName"),
         )
     }
 

clarity/src/vm/analysis/analysis_db.rs

@@ -45,26 +45,33 @@ impl<'a> AnalysisDatabase<'a> {
     pub fn execute<F, T, E>(&mut self, f: F) -> Result<T, E>
     where
         F: FnOnce(&mut Self) -> Result<T, E>,
+        E: From<CheckErrors>,
     {
         self.begin();
-        let result = f(self).map_err(|e| {
-            self.roll_back();
-            e
+        let result = f(self).or_else(|e| {
+            self.roll_back()
+                .map_err(|e| CheckErrors::Expects(format!("{e:?}")).into())?;
+            Err(e)
         })?;
-        self.commit();
+        self.commit()
+            .map_err(|e| CheckErrors::Expects(format!("{e:?}")).into())?;
         Ok(result)
     }
 
     pub fn begin(&mut self) {
         self.store.nest();
     }
 
-    pub fn commit(&mut self) {
-        self.store.commit();
+    pub fn commit(&mut self) -> CheckResult<()> {
+        self.store
+            .commit()
+            .map_err(|e| CheckErrors::Expects(format!("{e:?}")).into())
     }
 
-    pub fn roll_back(&mut self) {
-        self.store.rollback();
+    pub fn roll_back(&mut self) -> CheckResult<()> {
+        self.store
+            .rollback()
+            .map_err(|e| CheckErrors::Expects(format!("{e:?}")).into())
     }
 
     pub fn storage_key() -> &'static str {
@@ -78,7 +85,8 @@ impl<'a> AnalysisDatabase<'a> {
     pub fn test_insert_contract_hash(&mut self, contract_identifier: &QualifiedContractIdentifier) {
         use stacks_common::util::hash::Sha512Trunc256Sum;
         self.store
-            .prepare_for_contract_metadata(contract_identifier, Sha512Trunc256Sum([0; 32]));
+            .prepare_for_contract_metadata(contract_identifier, Sha512Trunc256Sum([0; 32]))
+            .unwrap();
     }
 
     pub fn has_contract(&mut self, contract_identifier: &QualifiedContractIdentifier) -> bool {
@@ -90,30 +98,42 @@ impl<'a> AnalysisDatabase<'a> {
     pub fn load_contract_non_canonical(
         &mut self,
         contract_identifier: &QualifiedContractIdentifier,
-    ) -> Option<ContractAnalysis> {
+    ) -> CheckResult<Option<ContractAnalysis>> {
         self.store
             .get_metadata(contract_identifier, AnalysisDatabase::storage_key())
             // treat NoSuchContract error thrown by get_metadata as an Option::None --
             //    the analysis will propagate that as a CheckError anyways.
-            .ok()?
-            .map(|x| ContractAnalysis::deserialize(&x))
+            .ok()
+            .flatten()
+            .map(|x| {
+                ContractAnalysis::deserialize(&x).map_err(|_| {
+                    CheckErrors::Expects("Bad data deserialized from DB".into()).into()
+                })
+            })
+            .transpose()
     }
 
     pub fn load_contract(
         &mut self,
         contract_identifier: &QualifiedContractIdentifier,
         epoch: &StacksEpochId,
-    ) -> Option<ContractAnalysis> {
-        self.store
+    ) -> CheckResult<Option<ContractAnalysis>> {
+        Ok(self
+            .store
             .get_metadata(contract_identifier, AnalysisDatabase::storage_key())
             // treat NoSuchContract error thrown by get_metadata as an Option::None --
             //    the analysis will propagate that as a CheckError anyways.
-            .ok()?
-            .map(|x| ContractAnalysis::deserialize(&x))
-            .map(|mut x| {
-                x.canonicalize_types(epoch);
-                x
+            .ok()
+            .flatten()
+            .map(|x| {
+                ContractAnalysis::deserialize(&x)
+                    .map_err(|_| CheckErrors::Expects("Bad data deserialized from DB".into()))
             })
+            .transpose()?
+            .and_then(|mut x| {
+                x.canonicalize_types(epoch);
+                Some(x)
+            }))
     }
 
     pub fn insert_contract(
@@ -127,7 +147,8 @@ impl<'a> AnalysisDatabase<'a> {
         }
 
         self.store
-            .insert_metadata(contract_identifier, key, &contract.serialize());
+            .insert_metadata(contract_identifier, key, &contract.serialize())
+            .map_err(|e| CheckErrors::Expects(format!("{e:?}")))?;
         Ok(())
     }
 
@@ -140,7 +161,7 @@ impl<'a> AnalysisDatabase<'a> {
         //         stored as its own entry. the analysis cost tracking currently only
         //         charges based on the function type size.
         let contract = self
-            .load_contract_non_canonical(contract_identifier)
+            .load_contract_non_canonical(contract_identifier)?
             .ok_or(CheckErrors::NoSuchContract(contract_identifier.to_string()))?;
         Ok(contract.clarity_version)
     }
@@ -156,7 +177,7 @@ impl<'a> AnalysisDatabase<'a> {
         //         stored as its own entry. the analysis cost tracking currently only
         //         charges based on the function type size.
         let contract = self
-            .load_contract_non_canonical(contract_identifier)
+            .load_contract_non_canonical(contract_identifier)?
             .ok_or(CheckErrors::NoSuchContract(contract_identifier.to_string()))?;
         Ok(contract
             .get_public_function_type(function_name)
@@ -174,7 +195,7 @@ impl<'a> AnalysisDatabase<'a> {
         //         stored as its own entry. the analysis cost tracking currently only
         //         charges based on the function type size.
         let contract = self
-            .load_contract_non_canonical(contract_identifier)
+            .load_contract_non_canonical(contract_identifier)?
             .ok_or(CheckErrors::NoSuchContract(contract_identifier.to_string()))?;
         Ok(contract
             .get_read_only_function_type(function_name)
@@ -192,7 +213,7 @@ impl<'a> AnalysisDatabase<'a> {
         //         stored as its own entry. the analysis cost tracking currently only
         //         charges based on the function type size.
         let contract = self
-            .load_contract_non_canonical(contract_identifier)
+            .load_contract_non_canonical(contract_identifier)?
             .ok_or(CheckErrors::NoSuchContract(contract_identifier.to_string()))?;
         Ok(contract.get_defined_trait(trait_name).map(|trait_map| {
             trait_map
@@ -207,7 +228,7 @@ impl<'a> AnalysisDatabase<'a> {
         contract_identifier: &QualifiedContractIdentifier,
     ) -> CheckResult<BTreeSet<TraitIdentifier>> {
         let contract = self
-            .load_contract_non_canonical(contract_identifier)
+            .load_contract_non_canonical(contract_identifier)?
             .ok_or(CheckErrors::NoSuchContract(contract_identifier.to_string()))?;
         Ok(contract.implemented_traits)
     }