All payment tweaks #674

joshuef · 2023-08-28T09:25:13Z

Description

reviewpad:summary

sn_networking/src/lib.rs

        let mut retries = 0;
+        // TODO: Move this put retry loop up above store cost checks so we can re-put if storecost failed.


sn_node/src/api.rs

sn_protocol/src/storage/chunks.rs

@@ -76,5 +83,6 @@
 #[derive(Debug, Eq, PartialEq, Clone, Serialize, Deserialize)]
 pub struct ChunkWithPayment {
    pub chunk: Chunk,
-    pub payment: PaymentProof,
+    /// TODO: encrypt this or supply only encrypted derivation index


sn_node/src/put_validation.rs

-            addr_name,
-            reason: err.to_string(),
-        })?;
+    // TODO: properly verify which one of these was for this node, and check _that_


sn_node/src/put_validation.rs

-                assert!(matches!(
-                    verify_fee_output_and_proof(name, network_store_cost, &tx, audit_trail, &[]),
-                    Err(err) if err == ProtocolError::PaymentProofInvalidFeeOutput(invalid_fee_id)));
+                // TODO: sort out what is acceptable based on the network store cost


sn_node/src/put_validation.rs

                    inputs: vec![],
-                    outputs: vec![],
+                    outputs: tx_payments.clone(),
+                    // TODO: Clean this up when we remove FeeOutput


sn_node/src/put_validation.rs

-        return Err(ProtocolError::PaymentProofInsufficientAmount { paid, expected });
+    // the fee output the expected `acceptable_fee` nano per record.
+    for output in tx.outputs.iter() {
+        // TODO: Is an output


sn_networking/src/lib.rs

sn_node/src/put_validation.rs

+                        "Trust replictated spend for {:?}",
+                        PrettyPrintRecordKey::from(key.clone())
+                    );
+                    // TODO: may need to tweak the `signed_spend_validation` function,


sn_cli/src/subcommands/files.rs

sn_transfers/src/wallet/local_store.rs

sn_node/tests/storage_payments.rs

starting to simplify payments to a per chunk basis A first step in maidsafe#638

Sets us up to pay every node that will be storing the data. We GetStoreCost and pay to their key, and put the chunk with the DBCs This is not yet validated at the node side

This gives maximal resilience as a place to start. (This is required for data location verification tests eg)

… pending

Use semaphore to allow uploads in parallel over many files instead of limiting it to chunks within a given file

with DBC parsing/passing mem is higher at the moment. This should be reduce as we move to make use of UTXOs

reviewpad · 2023-08-30T07:36:47Z

Reviewpad Report

‼️ Errors

Unconventional commit detected: 'feat!(protocol): remove chunk merkletree to simplify payment

starting to simplify payments to a per chunk basis

A first step in #638' (321cd19)

Unconventional commit detected: 'feat!(protocol): get price and pay for each chunk individually' (f670c09)
Unconventional commit detected: 'feat!(protocol): gets keys with GetStoreCost' (9a9cb25)
Unconventional title detected: 'All payment tweaks' illegal 'A' character in commit message type: col=00

Prevent readding validated data to errd.

github-advanced-security bot found potential problems Aug 28, 2023

View reviewed changes

reviewpad bot added the Large Large sized PR label Aug 28, 2023

joshuef force-pushed the AllPaymentTweaks branch from 8dea972 to f10c322 Compare August 28, 2023 13:07

github-advanced-security bot found potential problems Aug 28, 2023

View reviewed changes

sn_transfers/src/wallet/local_store.rs Fixed Show fixed Hide fixed

sn_node/tests/storage_payments.rs Fixed Show fixed Hide fixed

joshuef force-pushed the AllPaymentTweaks branch 2 times, most recently from afdf92c to b3293f1 Compare August 28, 2023 14:15

RolandSherwin force-pushed the AllPaymentTweaks branch 2 times, most recently from 54cbfd1 to 32066db Compare August 29, 2023 08:26

maqi force-pushed the AllPaymentTweaks branch 2 times, most recently from 4ef7dd2 to 790bd46 Compare August 29, 2023 14:07

RolandSherwin force-pushed the AllPaymentTweaks branch from 790bd46 to 8c548d3 Compare August 30, 2023 06:44

joshuef and others added 19 commits August 30, 2023 12:15

feat!(protocol): remove chunk merkletree to simplify payment

321cd19

starting to simplify payments to a per chunk basis A first step in maidsafe#638

feat!(protocol): get price and pay for each chunk individually

f670c09

feat!(protocol): gets keys with GetStoreCost

9a9cb25

feat!: pay each chunk holder direct

38325c4

Sets us up to pay every node that will be storing the data. We GetStoreCost and pay to their key, and put the chunk with the DBCs This is not yet validated at the node side

feat(node): store data if the majority of CLOSE_GROUP will

9900db9

test(node): reenable payment fail check

faafb20

chore(node): clarify payment errors

419623e

chore(node): only store paid for data, ignore maj

7533454

chore(networking): return all GetStoreCost prices and use them

c0539ee

This gives maximal resilience as a place to start. (This is required for data location verification tests eg)

chore(client): error out early for invalid transfers

8ab67aa

feat: one transfer per data set, mapped dbcs to content addrs

940a885

test(node): data verification test refactors for readability

422137b

chore(node): data verification log tweaks

6cd33dc

fix: not check payment for relocated holder

5ac7129

fix: correct replicated spend validation

c3fbeca

fix(node): handling events should wait before connected to the network

7f21f29

fix(network): trigger bootstrap until we have enough peers

2ad6a61

ci: add missing package specification to dbc spend tests

27d4f0c

ci: increase node mem limit with increased dbc processing

dff6293

joshuef and others added 12 commits August 30, 2023 12:15

fix(protocol): avoid panics

97d5187

chore: increase concurrent fetches for replication data

330936a

test: parallelise churn data final query

3ed2589

chore(networking): ensure we're always driving forward replication if…

5051481

… pending

chore(client): reduce transferoutputs cloning

4f1237f

chore(client): pass around content payments map mut ref

3d73477

feat: refactor to allow greater upload parallelisation

334c1e6

Use semaphore to allow uploads in parallel over many files instead of limiting it to chunks within a given file

chore(client): refactor client wallet to reduce dbc clones

c466c8e

chore: mem_check test update

b70223c

chore(deps): bump tokio to 1.32.0

0dd0452

fix(tokio): remove tokio fs

6f23a27

chore: trival clean ups

d1329bb

RolandSherwin force-pushed the AllPaymentTweaks branch from 8c548d3 to d1329bb Compare August 30, 2023 06:45

joshuef marked this pull request as ready for review August 30, 2023 07:26

RolandSherwin and others added 3 commits August 30, 2023 13:04

feat(protocol): add logs for RecordHeader serde

9fa1968

chore(logs): add more spend PUT validation logs

f855706

ci: increase client mem usage limit

c02cf0f

with DBC parsing/passing mem is higher at the moment. This should be reduce as we move to make use of UTXOs

reviewpad bot requested a review from bochaco August 30, 2023 07:36

reviewpad bot added the waiting-for-review label Aug 30, 2023

joshuef added 2 commits August 30, 2023 16:09

test(node): refactor churn test order

f56694f

Prevent readding validated data to errd.

chore(networking): increase FETCH_TIMEOUT to 10s

d32eab4

maqi approved these changes Aug 30, 2023

View reviewed changes

joshuef added this pull request to the merge queue Aug 30, 2023

RolandSherwin mentioned this pull request Aug 30, 2023

PPC data location debug #672

Closed

Merged via the queue into maidsafe:main with commit 953d793 Aug 30, 2023
23 checks passed

joshuef deleted the AllPaymentTweaks branch August 30, 2023 17:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

All payment tweaks #674

All payment tweaks #674

joshuef commented Aug 28, 2023

reviewpad bot commented Aug 30, 2023

		let mut retries = 0;
		// TODO: Move this put retry loop up above store cost checks so we can re-put if storecost failed.

All payment tweaks #674

All payment tweaks #674

Conversation

joshuef commented Aug 28, 2023

Description

reviewpad bot commented Aug 30, 2023