Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vinai/load data basic network handling. #54

Merged
merged 27 commits into from
Mar 1, 2021
Merged

Conversation

VinaiRachakonda
Copy link

@VinaiRachakonda VinaiRachakonda commented Feb 25, 2021

Basic steps for packet management. Have text with files up to 1gb.

Definitely some more work to be done here in the future. Is it optimal to write everything to a file or can we do something better? Regardless, I think this is a good enough start for Max to get unblocked while I continue to build more functionality.

Copy link
Member

@zachmu zachmu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable, couple comments

go/mysql/conn.go Outdated
c.recycleReadPacket()
query := c.parseComQuery(data)
//if strings.Contains(strings.ToLower(query), "local"){
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How are you actually going to check this? You have to parse the query, right?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea I move the actual call to go-mysql-server

go/mysql/conn.go Outdated

// Write the file to a temporary directory.
// TODO: Get the tmpdir config params from here
err = ioutil.WriteFile(fileName, fileData, 0644)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There can be more than one packet with file data, right?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Is the idea here to keep waiting with a timeout or something like that?

go/mysql/conn.go Outdated
if sequence != c.sequence {
return 0, vterrors.Errorf(vtrpc.Code_INTERNAL, "invalid sequence, expected %v got %v", c.sequence, sequence)
}
//sequence := uint8(header[3])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this about?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will remove this edit. Was having trouble with code earlier in the process...

@VinaiRachakonda VinaiRachakonda changed the title [Draft] Vinai/load data p2 Vinai/load data basic network handling. Feb 25, 2021
Copy link
Member

@zachmu zachmu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG

go/mysql/conn.go Outdated
fileData, err = c.readEphemeralPacket();
}

if len(fileData) != 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this is now impossible due to the above loop

@VinaiRachakonda VinaiRachakonda merged commit 8460b65 into master Mar 1, 2021
@VinaiRachakonda VinaiRachakonda deleted the vinai/load-data-p2 branch March 1, 2021 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants