Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: 13135 Added PlatformState protobuf representation #349

Open
wants to merge 21 commits into
base: main
Choose a base branch
from

Conversation

imalygin
Copy link
Member

@imalygin imalygin commented May 8, 2024

Description:

This PR adds representation of com.swirlds.platform.state.PlatformState object. It's a prerequisite for hashgraph/hedera-services#11771

Related issue(s):

Fixes # hashgraph/hedera-services#13135

Related # hashgraph/hedera-services#11771

@imalygin imalygin requested review from a team as code owners May 8, 2024 21:47
@imalygin imalygin force-pushed the 13135-add-platform-state branch 2 times, most recently from 145c149 to 75049c9 Compare May 8, 2024 21:58
@imalygin imalygin requested a review from a team as a code owner May 8, 2024 23:17
@imalygin imalygin requested a review from kfa-aguda May 8, 2024 23:17
@imalygin imalygin force-pushed the 13135-add-platform-state branch 2 times, most recently from f6f6194 to 8ead05f Compare May 8, 2024 23:24
iwsimon
iwsimon previously approved these changes May 9, 2024
Copy link
Contributor

@iwsimon iwsimon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @imalygin

@imalygin imalygin force-pushed the 13135-add-platform-state branch 5 times, most recently from 39010c2 to d208126 Compare May 9, 2024 22:19
Copy link
Member

@jsync-swirlds jsync-swirlds left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for the delay, this took a bit more thought and rework than expected; partly a lot of description was in pseudo-requirements so that took some thought to understand.

platform/state/platform_state.proto Outdated Show resolved Hide resolved
platform/state/platform_state.proto Outdated Show resolved Hide resolved
jsync-swirlds
jsync-swirlds previously approved these changes May 10, 2024
platform/state/platform_state.proto Outdated Show resolved Hide resolved
platform/state/platform_state.proto Outdated Show resolved Hide resolved
platform/state/platform_state.proto Outdated Show resolved Hide resolved
platform/state/platform_state.proto Outdated Show resolved Hide resolved
platform/state/platform_state.proto Outdated Show resolved Hide resolved
platform/state/platform_state.proto Outdated Show resolved Hide resolved
platform/state/platform_state.proto Outdated Show resolved Hide resolved
Comment on lines 326 to 343
/**
* A string network address.<br/>
* This value is the hostname assigned on the "internal" network
* interface behind any network firewalls, protocol gateways, or
* DNS translations.
* <p>
* This value SHALL be either a FQDN host name or an IPv4 address.
*/
string hostname_internal = 5;

/**
* Network port number.<br/>
* This value is the "translated" port number assigned on the "internal"
* network behind any network firewalls or other address translations.
* <p>
* This value SHALL be between 1 and 65535.
*/
uint32 port_internal = 6;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before committing to supporting these fields forever, it might be worth having another discussion about this. I never quite wrapped my head around the truly irreplaceable need for these to be present in the address book.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who should I have this conversation with?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think devops are going to be the important people to talk to on this one. They should be knowledgeable about whether or not this is in any way relevant. @nathanklick should also probably be included in the conversation.

Comment on lines 376 to 379
/**
* A string that provides additional information about this consensus node.
*/
string memo = 11;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also probably doesn't belong in the platform data structure in the long term. If we can't get rid of it in the short term, we should move it to a high index.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The field is updated with a high index.

platform/state/platform_state.proto Show resolved Hide resolved
platform/state/platform_state.proto Outdated Show resolved Hide resolved
platform/state/platform_state.proto Outdated Show resolved Hide resolved
Comment on lines 376 to 379
/**
* A string that provides additional information about this consensus node.
*/
string memo = 11;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed.

platform/state/platform_state.proto Outdated Show resolved Hide resolved
platform/state/platform_state.proto Outdated Show resolved Hide resolved
imalygin and others added 3 commits June 11, 2024 16:10
Signed-off-by: Ivan Malygin <ivan@swirldslabs.com>
Signed-off-by: Ivan Malygin <ivan@swirldslabs.com>
Co-authored-by: Joseph Sinclair <121976561+jsync-swirlds@users.noreply.github.com>
Updated the documentation according to the guidelines.

Moved platform_state.proto to platform/state directory.
Updated the documentation according to the guidelines.

Signed-off-by: Ivan Malygin <ivan@swirldslabs.com>
imalygin and others added 2 commits June 13, 2024 12:02
Signed-off-by: Ivan Malygin <ivan@swirldslabs.com>
Co-authored-by: Joseph Sinclair <121976561+jsync-swirlds@users.noreply.github.com>
…laced it with `SemanticVersion`

Signed-off-by: Ivan Malygin <ivan@swirldslabs.com>
jsync-swirlds
jsync-swirlds previously approved these changes Jun 13, 2024
* This SHALL be the consensus timestamp of the first transaction in
* the current round.
*/
proto.Timestamp consensus_timestamp = 2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This data is also duplicated inside of ConsensusSnapshot. Can it be removed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I can remove it if you can confirm that these two fields mean exactly the same thing in the context of persisted platform state

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cody-littley @lpetrovic05 Could you weigh in here too?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same data in both places, there is only a single concept of the term "round" with respect to the data stored in the state. (Consensus overloads the term "round", but that data should never be leaving the consensus black box.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, fixed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@imalygin The value in the platform state gets populated from the ConsensusSnapshot, so it must be removed from the platform state, not the snapshot

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lpetrovic05 okay, sounds good

message MinimumJudgeInfo {
/**
* A consensus round.<br/>
* The round this judge information applies to.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The round this judge information applies to.

I don't know how to interpret this. Does this mean the round here will match the round field in ConsensusSnapshot? Or something else?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @cody-littley have the answer.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will have a list of these objects, with one object corresponding to a particular round.

Now that I think about it, I wonder if we actually need to encode the round number in this object. We could interpret the first element in this list as having the current round, the next as having round N-1, and so on. In fact, we may be able to get away with not having a type for this... we could simply use list of long values.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though this is a reasonable idea, I suggest implementing it separately.

…d_mode` and updated its order.

Signed-off-by: Ivan Malygin <ivan@swirldslabs.com>
…ancient_threshold`.

Signed-off-by: Ivan Malygin <ivan@swirldslabs.com>
… from `ConsensusSnapshot`, as these fields duplicate fields in `PlatformState` and mean exactly the same thing.

Signed-off-by: Ivan Malygin <ivan@swirldslabs.com>
Signed-off-by: Ivan Malygin <ivan@swirldslabs.com>
Signed-off-by: Ivan Malygin <ivan@swirldslabs.com>
Co-authored-by: Joseph Sinclair <121976561+jsync-swirlds@users.noreply.github.com>
…put it back to `ConsensusSnapshot`

Signed-off-by: Ivan Malygin <ivan@swirldslabs.com>
…ormState` and put it back to `ConsensusSnapshot`

Signed-off-by: Ivan Malygin <ivan@swirldslabs.com>
Signed-off-by: Ivan Malygin <ivan@swirldslabs.com>
platform/state/platform_state.proto Outdated Show resolved Hide resolved
Comment on lines +111 to +120
uint64 lowest_judge_generation_before_birth_round_mode = 10001 [deprecated = true];

/**
* A consensus round.<br/>
* The last round before the birth round mode was enabled.
* Will be removed after the birth round migration.
* <p>
* This SHALL be `MAX_UNSIGNED` if birth round mode has not yet been enabled.
*/
uint64 last_round_before_birth_round_mode = 10002 [deprecated = true];

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if calling these fields deprecated is correct. We still very much need them up until we do the birth round migration. It's only after that migration that we should mark them as deprecated.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Joseph @jsync-swirlds proposed to mark all the fields that we're intending to remove as deprecated. That is, if a field has index 10000+, then it should be marked as deprecated. I don't have a strong opinion on that. I will remove it if Joseph agrees with your comment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In protobuf, deprecated should not be a "this field is no longer used" indicator; protobuf would say to remove the field at that point and reserve the number. For a protobuf, deprecated should be "we are working to stop using this field and will remove it as soon as we can", which is what it sounds like is happening with respect to birth round migration.

* This will be _removed_ and the field number reserved once the consensus
* event stream is retired.
*/
bytes legacy_running_event_hash = 10000 [deprecated = true];

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to delete this field, but we currently have a hard dependency on it. Should we mark it as deprecated if we are still using this field, or only after we are no longer reading/writing it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typically deprecated is aded when a field is still in use, but the plan is to remove it. When the field is no longer in use the typical action would be to remove it and reserve the field number.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is that if we can foresee the deletion of this field, then we should mark it as deprecated. And we clearly can tell that we're going to remove it relatively soon.

message MinimumJudgeInfo {
/**
* A consensus round.<br/>
* The round this judge information applies to.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will have a list of these objects, with one object corresponding to a particular round.

Now that I think about it, I wonder if we actually need to encode the round number in this object. We could interpret the first element in this list as having the current round, the next as having round N-1, and so on. In fact, we may be able to get away with not having a type for this... we could simply use list of long values.

…h_round_mode` to the set of deprecated fields.

Signed-off-by: Ivan Malygin <ivan@swirldslabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants