Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix replication tree with sharding #668

Merged
merged 5 commits into from
Nov 17, 2022

Conversation

samuelmanzanera
Copy link
Member

Description

This PR aims to resolve a problem of serialization when the size of the replication tree is different from the chain replicas & beacon replicas

Fixes #663

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

You can apply this patch, with more than 5 running nodes

diff --git a/lib/archethic/election/constraints/storage.ex b/lib/archethic/election/constraints/storage.ex
index 02ed0edc..7f8fb879 100755
--- a/lib/archethic/election/constraints/storage.ex
+++ b/lib/archethic/election/constraints/storage.ex
@@ -13,8 +13,8 @@ defmodule Archethic.Election.StorageConstraints do
 
   alias Archethic.P2P.Node
 
-  @default_min_geo_patch 4
-  @default_min_geo_patch_avg_availability 0.8
+  @default_min_geo_patch 1
+  @default_min_geo_patch_avg_availability 0.0
 
   @type min_geo_patch_fun() :: (() -> non_neg_integer())
   @type min_geo_patch_avg_availability_fun() :: (() -> float())
@@ -108,9 +108,14 @@ defmodule Archethic.Election.StorageConstraints do
   """
   @spec hypergeometric_distribution(list(Node.t())) :: pos_integer()
   def hypergeometric_distribution(nodes)
-      when is_list(nodes) and length(nodes) >= 0 and length(nodes) <= 10,
-      do: nodes
+      when is_list(nodes) and length(nodes) >= 0 and length(nodes) <= 10 do
+    if length(nodes) > 5 do
+      length(nodes) - 2
+    else
+      length(nodes)
+    end
+  end
 
-  def hypergeometric_distribution(nodes) when is_list(nodes) and length(nodes) >= 0,
+  def hypergeometric_distribution(nodes) when is_list(nodes),
     do: HypergeometricDistribution.run_simulation(length(nodes))
 end

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

@samuelmanzanera samuelmanzanera added bug Something isn't working P2P Involve P2P networking serialization Involve message serialization labels Nov 4, 2022
@samuelmanzanera samuelmanzanera self-assigned this Nov 4, 2022
@samuelmanzanera samuelmanzanera force-pushed the fix_replication_tree_with_sharding branch from 945f553 to 5a32469 Compare November 4, 2022 15:28
@samuelmanzanera samuelmanzanera added the core team Assigned to the core team label Nov 4, 2022
@apoorv-2204
Copy link
Contributor

How to ?

@apoorv-2204
Copy link
Contributor

@default_min_geo_patch_avg_availability 0.0?

Copy link
Contributor

@apoorv-2204 apoorv-2204 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(develop branch latest commit -1) code + 5 Nodes + mix dev.lbench + Archethic.Utils.Regression.Benchmark.EndToEndValidation.plan(["127.0.0.1"],[]) + git patch
serialization errors

Screenshot_20221114_175838

Screenshot_20221114_175826

No serialization error, seen with
(develop branch latest commit -1) code + prcode+ 5 Nodes + mix dev.lbench + Archethic.Utils.Regression.Benchmark.EndToEndValidation.plan(["127.0.0.1"],[]) + git patch

Unit Test cases is required.

@samuelmanzanera samuelmanzanera force-pushed the fix_replication_tree_with_sharding branch from 1247950 to 5d6cd59 Compare November 14, 2022 22:02
@samuelmanzanera
Copy link
Member Author

samuelmanzanera commented Nov 14, 2022

@default_min_geo_patch_avg_availability 0.0?

This is used to say we don't expect minimal availability ratio per geo patch.

36, 248, 98, 108, 241, 114, 47, 102, 176, 212>>,
protocol_version: ArchethicCase.current_protocol_version()
},
replication_tree: %{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we include dynamic creation of replication tree?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean ?

Copy link
Member

@Neylix Neylix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In storage constraint if the number of nodes is between 0 and 10 we are returning the list of nodes instead of the number of nodes

@samuelmanzanera
Copy link
Member Author

In storage constraint if the number of nodes is between 0 and 10 we are returning the list of nodes instead of the number of nodes

Yes indeed , to do like in the patch

@samuelmanzanera samuelmanzanera force-pushed the fix_replication_tree_with_sharding branch from 807fdee to 5e5f8f1 Compare November 17, 2022 08:17
@samuelmanzanera samuelmanzanera merged commit 5466387 into develop Nov 17, 2022
@samuelmanzanera samuelmanzanera deleted the fix_replication_tree_with_sharding branch November 17, 2022 11:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working core team Assigned to the core team P2P Involve P2P networking serialization Involve message serialization
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix replication tree when sharding is applied
3 participants