Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add default Huffman table if huffman table uninitialized #493

Merged
merged 9 commits into from
Apr 18, 2023
48 changes: 48 additions & 0 deletions pkg/frame/compressed.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,55 @@ import (
"image/jpeg"
)

// Thank you to https://github.com/filiptc/gorbit/blob/fa87ff39b68a6706306f34c318e0b9a5a3c97110/image/overlay.go#L37-L40 for addMotionDht, dhtMarker, dht, and sosMarker
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

licensing checks out 👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strictly speaking, when OSS licensed code is copied, we should include the full original license statements according to

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

in case of MIT.
https://github.com/filiptc/gorbit/blob/fa87ff39b68a6706306f34c318e0b9a5a3c97110/LICENSE


var (
dhtMarker = []byte{255, 196}
dht = []byte{1, 162, 0, 0, 1, 5, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 1, 0, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 16, 0, 2, 1, 3, 3, 2, 4, 3, 5, 5, 4, 4, 0, 0, 1, 125, 1, 2, 3, 0, 4, 17, 5, 18, 33, 49, 65, 6, 19, 81, 97, 7, 34, 113, 20, 50, 129, 145, 161, 8, 35, 66, 177, 193, 21, 82, 209, 240, 36, 51, 98, 114, 130, 9, 10, 22, 23, 24, 25, 26, 37, 38, 39, 40, 41, 42, 52, 53, 54, 55, 56, 57, 58, 67, 68, 69, 70, 71, 72, 73, 74, 83, 84, 85, 86, 87, 88, 89, 90, 99, 100, 101, 102, 103, 104, 105, 106, 115, 116, 117, 118, 119, 120, 121, 122, 131, 132, 133, 134, 135, 136, 137, 138, 146, 147, 148, 149, 150, 151, 152, 153, 154, 162, 163, 164, 165, 166, 167, 168, 169, 170, 178, 179, 180, 181, 182, 183, 184, 185, 186, 194, 195, 196, 197, 198, 199, 200, 201, 202, 210, 211, 212, 213, 214, 215, 216, 217, 218, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 17, 0, 2, 1, 2, 4, 4, 3, 4, 7, 5, 4, 4, 0, 1, 2, 119, 0, 1, 2, 3, 17, 4, 5, 33, 49, 6, 18, 65, 81, 7, 97, 113, 19, 34, 50, 129, 8, 20, 66, 145, 161, 177, 193, 9, 35, 51, 82, 240, 21, 98, 114, 209, 10, 22, 36, 52, 225, 37, 241, 23, 24, 25, 26, 38, 39, 40, 41, 42, 53, 54, 55, 56, 57, 58, 67, 68, 69, 70, 71, 72, 73, 74, 83, 84, 85, 86, 87, 88, 89, 90, 99, 100, 101, 102, 103, 104, 105, 106, 115, 116, 117, 118, 119, 120, 121, 122, 130, 131, 132, 133, 134, 135, 136, 137, 138, 146, 147, 148, 149, 150, 151, 152, 153, 154, 162, 163, 164, 165, 166, 167, 168, 169, 170, 178, 179, 180, 181, 182, 183, 184, 185, 186, 194, 195, 196, 197, 198, 199, 200, 201, 202, 210, 211, 212, 213, 214, 215, 216, 217, 218, 226, 227, 228, 229, 230, 231, 232, 233, 234, 242, 243, 244, 245, 246, 247, 248, 249, 250}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this data standardized or specific to some device of some vendor?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should work across devices in the case that the Huffman encoding info has been omitted for bandwidth optimization

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you point where this comes from though?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The JPEG wiki lists dhtMarker & sosMarker as dht & sos respectively.
This RFC discusses a "Standard Huffman Table" and location of the citation is behind a paywall. This thread is where I got the table from. It works on cameras I have tested with and has been reported to work on other cameras as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@at-wat is that cool with you?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This RFC discusses a "Standard Huffman Table" and location of the citation is behind a paywall.

https://www.w3.org/Graphics/JPEG/itu-t81.pdf
W3C seems putting it in public.

sosMarker = []byte{255, 218}
)

func decodeMJPEG(frame []byte, width, height int) (image.Image, func(), error) {
img, err := jpeg.Decode(bytes.NewReader(frame))

if err == nil || (err != nil && err.Error() != "invalid JPEG format: uninitialized Huffman table") {
return img, func() {}, err
}

img, err = jpeg.Decode(bytes.NewReader(addMotionDht(frame)))
return img, func() {}, err
}

func addMotionDht(frame []byte) []byte {
jpegParts := bytes.Split(frame, sosMarker)
if len(jpegParts) != 2 {
return frame
}
correctedFrame := make([]byte, len(jpegParts[0])+len(dhtMarker)+len(dht)+len(sosMarker)+len(jpegParts[1]))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

len(dhtMarker)+len(dht)+len(sosMarker) can be calculated once outside of this function which will simplify reading this

correctedFrameOffset := 0

for indx, item := range jpegParts[0] {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indx -> idx for each of these

correctedFrame[indx] = item
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of these loops I think you can do copy(correctedFrame[offset:], jpegParts[0]) for each of these. that should make things smaller and simple to read!

}
correctedFrameOffset += len(jpegParts[0])

for indx, item := range dhtMarker {
correctedFrame[indx+correctedFrameOffset] = item
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slightly more readable to have it be the offset + the idx for each of these

}
correctedFrameOffset += len(dhtMarker)

for indx, item := range dht {
correctedFrame[indx+correctedFrameOffset] = item
}
correctedFrameOffset += len(dht)

for indx, item := range sosMarker {
correctedFrame[indx+correctedFrameOffset] = item
}
correctedFrameOffset += len(sosMarker)

for indx, item := range jpegParts[1] {
correctedFrame[indx+correctedFrameOffset] = item
}
return correctedFrame
}
44 changes: 44 additions & 0 deletions pkg/frame/compressed_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
package frame

import (
"bytes"
"image/jpeg"
"testing"
)

func TestAddMotionDht(t *testing.T) {
uninitializedHuffmanTableFrame, err := jpeg.Decode(bytes.NewReader(UninitializedHuffmanTable))

// Decode fails with an uninitialized Huffman table error for sample input
expectedErrorMessage := "invalid JPEG format: uninitialized Huffman table"
if err.Error() != expectedErrorMessage {
t.Errorf("Wrong decode error result,\nexpected:\n%+v\ngot:\n%+v", expectedErrorMessage, err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe you need to use FailNow as well

}

// Decode passes after adding default Huffman table to
defaultHuffmanTableFrame, err := jpeg.Decode(bytes.NewReader(addMotionDht(UninitializedHuffmanTable)))
if err != nil {
t.Errorf("Expected decode function to pass after adding default Huffman table. Failed with %v\n", err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all of these need the failnow too but you can just do Fatalf instead

}

// Adding default Huffman table to a valid frame without a Huffman table changes the table
if uninitializedHuffmanTableFrame == defaultHuffmanTableFrame {
t.Errorf("Expected addMotionDht to update frame. Instead returned original frame")
}

// Check that an improperly constructed frame does not get updated by addMotionDht
randomBytes := []byte{1, 2, 3, 4}
frame1, err := jpeg.Decode(bytes.NewReader(randomBytes))
if err == nil {
t.Errorf("Expected decode function to fail with random bytes but passed.")
}

frame2, err := jpeg.Decode(bytes.NewReader(addMotionDht(randomBytes)))
if err == nil {
t.Errorf("Expected decode function to fail with random bytes but passed.")
}

if frame1 != frame2 {
t.Errorf("addMotionDht updated the frame despite being improperly constructed")
}
}
3 changes: 3 additions & 0 deletions pkg/frame/sampleFrames.go

Large diffs are not rendered by default.