Skip to content

Conversation

@Jefffrey
Copy link
Contributor

What changes were proposed in this pull request?

Update PatchedBase specification doc to include details about the behaviour of padding the patch gap + patch width bits to nearest fixed btis.

Why are the changes needed?

Ensure spec is accurate to implementation details

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

No

@github-actions github-actions bot added the DOCS label Mar 31, 2024
Comment on lines 837 to 851
(PGW + PW) | ceil(PGW + PW)
:------------ | :-------------
1 <= x <= 24 | x
25 | 26
26 | 26
27 | 28
28 | 28
29 | 30
30 | 30
31 | 32
32 | 32
33 <= x <= 40 | 40
41 <= x <= 48 | 48
49 <= x <= 56 | 56
57 <= x <= 64 | 64
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From

orc/c++/src/RLEV2Util.cc

Lines 29 to 33 in 9b79de9

// Map bit length i to closest fixed bit width that can contain i bits.
const uint8_t ClosestFixedBitsMap[65] = {
1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21,
22, 23, 24, 26, 26, 28, 28, 30, 30, 32, 32, 40, 40, 40, 40, 40, 40, 40, 40, 48, 48, 48,
48, 48, 48, 48, 48, 56, 56, 56, 56, 56, 56, 56, 56, 64, 64, 64, 64, 64, 64, 64, 64};

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it better to follow the java code to add 0 as well?

public int getClosestFixedBits(int n) {
if (n == 0) {
return 1;
}
if (n <= 24) {
return n;
}
if (n <= 26) {
return 26;
}
if (n <= 28) {
return 28;
}
if (n <= 30) {
return 30;
}
if (n <= 32) {
return 32;
}
if (n <= 40) {
return 40;
}
if (n <= 48) {
return 48;
}
if (n <= 56) {
return 56;
}
return 64;
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it makes sense to have 0 as the input is PGW + PW, and the spec states that PGW is 1 to 8 bits and PW is 1 to 64 bits, so it can never be 0 (or 1, for that matter, I guess 🤔 )

Comment on lines 834 to 835
64. (PGW + PW) is padded to the nearest fixed bit size according to the
below table before being encoded in the patch list.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to

uint32_t cfb = getClosestFixedBits(patchBitSize + pgw);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please rename ceil to closestFixedBits or cfb? ceil does not seems to have its common meaning here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing, done

Copy link
Member

@wgtmac wgtmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix! Could you please also add them to ORCv2.md as well?

Comment on lines 834 to 835
64. (PGW + PW) is padded to the nearest fixed bit size according to the
below table before being encoded in the patch list.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please rename ceil to closestFixedBits or cfb? ceil does not seems to have its common meaning here.

Comment on lines 837 to 851
(PGW + PW) | ceil(PGW + PW)
:------------ | :-------------
1 <= x <= 24 | x
25 | 26
26 | 26
27 | 28
28 | 28
29 | 30
30 | 30
31 | 32
32 | 32
33 <= x <= 40 | 40
41 <= x <= 48 | 48
49 <= x <= 56 | 56
57 <= x <= 64 | 64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it better to follow the java code to add 0 as well?

public int getClosestFixedBits(int n) {
if (n == 0) {
return 1;
}
if (n <= 24) {
return n;
}
if (n <= 26) {
return 26;
}
if (n <= 28) {
return 28;
}
if (n <= 30) {
return 30;
}
if (n <= 32) {
return 32;
}
if (n <= 40) {
return 40;
}
if (n <= 48) {
return 48;
}
if (n <= 56) {
return 56;
}
return 64;
}

Copy link
Member

@wgtmac wgtmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM +1

@wgtmac
Copy link
Member

wgtmac commented Apr 4, 2024

cc @dongjoon-hyun @deshanxiao

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM.

@dongjoon-hyun dongjoon-hyun added this to the 2.1.0 milestone Apr 9, 2024
@deshanxiao
Copy link
Contributor

Thanks @Jefffrey @wgtmac @dongjoon-hyun . Merged to main.

@Jefffrey Jefffrey deleted the ORC-642 branch April 10, 2024 10:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants