Skip to content

Executions stored in Database#560

Merged
krhubert merged 38 commits intodevfrom
feature/execution-databse
Nov 2, 2018
Merged

Executions stored in Database#560
krhubert merged 38 commits intodevfrom
feature/execution-databse

Conversation

@antho1404
Copy link
Copy Markdown
Member

I've created a database to store all executions. For now it is a leveldb database but this can be changed in the future if needed. The purpose of this PR is really to not store any execution state in memory to avoid huge memory consumption.

Comment thread interface/grpc/core/core_test.go Outdated
Comment thread execution/leveldb.go Outdated
Comment thread execution/execution.go Outdated
Comment thread execution/leveldb.go Outdated
Comment thread execution/execution_test.go Outdated
Comment thread execution/execution_test.go Outdated
Comment thread execution/leveldb.go Outdated
Comment thread api/api_test.go Outdated
Comment thread execution/execution_test.go Outdated
Comment thread execution/execution_test.go Outdated
@ilgooz
Copy link
Copy Markdown
Contributor

ilgooz commented Oct 29, 2018

this one has some conflicts that needs to be resolved

@antho1404
Copy link
Copy Markdown
Member Author

I've done some update like splitting the database part (and put it in the database package) and also have the execution package that only manage the business logic related to the execution object. The responsibility to save this execution is done directly in the api package.

Waiting more feedbacks on that

krhubert
krhubert previously approved these changes Oct 30, 2018
Comment thread database/execution_db.go Outdated
Comment thread database/execution_db.go Outdated
Comment thread database/execution_db.go Outdated
// Save an instance of executable in the database
// Returns an error if anything from marshaling to database saving goes wrong
func (db *LevelDBExecutionDB) Save(execution *execution.Execution) (*execution.Execution, error) {
id := fmt.Sprintf("%x", sha1.Sum(structhash.Dump(execution, 1)))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

id shouldn't be generated from hash calculation of execution struct. there is no a specific need for this so we can avoid doing it and use uuid. otherwise while updating something from execution, the ID also needs to be changed for no real reason.

Comment thread database/execution_db.go Outdated
Comment thread execution/execution.go Outdated
Comment thread execution/execution.go Outdated
Comment thread execution/execution.go Outdated
@antho1404
Copy link
Copy Markdown
Member Author

I changed slightly the current implementation, now the hash is not calculated on the execution date but based on an eventID, executions should always be executed based on an event and every events should have an unique identifier. For now we generate the unique Id only based on an UUID but with decentralized services this should be the ID of the event (like the transaction hash on ethereum for example)

@ilgooz
Copy link
Copy Markdown
Contributor

ilgooz commented Oct 31, 2018

I changed slightly the current implementation, now the hash is not calculated on the execution date but based on an eventID, executions should always be executed based on an event and every events should have an unique identifier. For now we generate the unique Id only based on an UUID but with decentralized services this should be the ID of the event (like the transaction hash on ethereum for example)

I don't get these changes. Why are we complicating unique id generation? To me, we can create and set execution.Execution.ID with a randomly generated uuid inside execution.New() or execution.Save(). Why do we need to generate a hash over execution.Execution data structure? We're not looking for uniqueness of executions. The only thing needed is a random unique id for identifying each execution.

@antho1404
Copy link
Copy Markdown
Member Author

We actually need "uniqueness of executions". When we have the distributed version of MESG, the execution will be created by one node but all other nodes should be able to reproduce the same id based on the same data to ensure that they are verifying/executing the exact same execution. For now it's not mandatory to use a hash but it will be and it doesn't add much complexity, we already have the exact same principle for services for the same reason.

Comment thread execution/execution_test.go Outdated
hasError bool
}{
{e.ID, false},
{"doesn't exists", true},
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm removing this because I think, it's forgotten here.

@ilgooz ilgooz force-pushed the feature/execution-databse branch from 7b685a4 to 40db360 Compare November 1, 2018 07:42
@ilgooz ilgooz force-pushed the feature/execution-databse branch from 40db360 to 572fe63 Compare November 1, 2018 07:48
krhubert
krhubert previously approved these changes Nov 1, 2018
@ilgooz ilgooz force-pushed the feature/execution-databse branch from 5367613 to 99eb632 Compare November 1, 2018 08:19
Copy link
Copy Markdown
Contributor

@ilgooz ilgooz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

manual tests are ok

@krhubert krhubert merged commit a940f93 into dev Nov 2, 2018
@ilgooz ilgooz deleted the feature/execution-databse branch November 2, 2018 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants