Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tablet): add uncompressed size to pb.Tablet #7095

Merged
merged 6 commits into from Dec 10, 2020
Merged

Conversation

NamanJain8
Copy link
Contributor

@NamanJain8 NamanJain8 commented Dec 9, 2020

  • Add UncompressedBytes to pb.Tablet denoting the uncompressed size of tablet.
  • Rename pb.Tablet.Space -> pb.Tablet.OnDiskBytes
  • Simplify tablet size calculation as badger does not have internal keys now.

/state endpoint displays the tablet in format:

"post.body": {
          "groupId": 1,
          "predicate": "post.body",
          "force": false,
          "onDiskBytes": "40773239190",
          "remove": false,
          "readOnly": false,
          "moveTs": "0",
          "uncompressedBytes": "66824100606"
        },

Log in predicate move

Groups sorted by size: [{gid:2 size:0} {gid:1 size:73403748503}]

I1209 23:26:28.678858 2689798 tablet.go:215] size_diff 73403748503
I1209 23:26:28.679134 2689798 tablet.go:109] Going to move predicate: [comment.text], size: [ondisk: 13 GB, uncompressed: 19 GB] from group 1 to 2


This change is Reviewable

Copy link
Contributor

@manishrjain manishrjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 6 of 6 files at r1.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @NamanJain8 and @vvbalaji-dgraph)


worker/draft.go, line 1314 at r1 (raw file):

	var total int64
	tablets := make(map[string]*pb.Tablet)
	updateSize := func(pred string, onDiskSize int64, uncompressedSize int64) {

accept tinfo.


worker/draft.go, line 1349 at r1 (raw file):

		// Count the table only if it is occupied by a single predicate.
		if left.Attr == right.Attr {
			updateSize(left.Attr, int64(tinfo.OnDiskSize), int64(tinfo.UncompressedSize))

Just pass in tinfo.

msg := fmt.Sprintf("Going to move predicate: [%v], size: [%v] from group %d to %d\n", predicate,
humanize.Bytes(uint64(tab.Space)), srcGroup, dstGroup)
msg := fmt.Sprintf("Going to move predicate: [%v], size: [ondisk: %v, uncompressed: %v]"+
" from group %d to %d\n", predicate, humanize.Bytes(uint64(tab.OnDiskBytes)),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

humanize.IBytes

Copy link
Contributor

@danielmai danielmai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 5 of 5 files at r2.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @NamanJain8 and @vvbalaji-dgraph)

@NamanJain8 NamanJain8 merged commit 843fe1f into master Dec 10, 2020
@NamanJain8 NamanJain8 deleted the naman/tablet-size branch December 10, 2020 07:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants