Fix Issue #428 #430
Fix Issue #428 #430
Conversation
Codecov Report
@@ Coverage Diff @@
## master #430 +/- ##
==========================================
+ Coverage 62.02% 62.09% +0.07%
==========================================
Files 149 149
Lines 20456 20465 +9
==========================================
+ Hits 12687 12708 +21
+ Misses 7769 7757 -12
Continue to review full report at Codecov.
|
|
Apologies for not getting to this on Monday, Luis. I was exhausted (and delighted) by solving the MinGW/setMsg/GCC8.2.0 issue (#421). I've reproduced #428 and will work on it later today (Tuesday). First priority is to review this PR. Thanks for working on this. My old friend Gaylord lives in California and yesterday was his 80th birthday. I've known him for 40 years and went to visit him 2 weeks ago. We didn't go to California for his birthday party yesterday, however we had a little cake and candles on the patio. Life is good when retired! https://clanmills.com/2018/September/
|
|
There's something not quite right about this. I've download the POCs: When I apply your patch, I see: Your patch is a step in the right direction. The code is cavalier in galloping forward without checking that it is reading within the buffer. Having decoded the length, the code gallops decoding hex/nibbles without bothering to test if the buffer has 2*length bytes. I've added Like you, I think the code needs a major review and the schedule doesn't permit this at the moment. I've put my modified version of PngChunk::readRawProfile() which includes the fixes that I've mentioned. This seems to fix the 6 POCs provided in #428 |
|
Great! Thanks @clanmills for the help. I'll include your changes and push them to the branch, so that we can see if the CI is also happy with the changes ;) |
|
Ups, it seems that those changes break other tests @clanmills. You can see the issues here: I'll revert the latest commits to make the CI green again. |
|
Oh, that's a surprise. The test suite is passing on my machine. You've done the right thing to revert this. I will revisit this and update you tomorrow. |
|
Apologies. I built and tested the wrong code on my machine yesterday. I've reproduced the test errors being reported. Somehow this has broken icc-test.sh. I'll debug and fix it tomorrow. Curiously, test the ASAN build has revealed something in stringto-test.sh and preview-test.sh and I'll also investigate that tomorrow and if necessary open new issues to address those matters. |
|
I know what's wrong with this and I'll fix it later. The option -iC (insert ICC profile) is causing the loss of Exif and IPTC metadata. |
|
I'm making steady progress with this. More difficult than expected. I've identified what's causing stringto-test to fail (when built for ASAN) and I'll open a different issue (with patch). I want to continue painting the summer house today and hope to finish this issue this evening. |
|
More challenges:
If I build without I've found this article (although I don't know if this is true) https://stackoverflow.com/questions/44320465/whats-the-proper-way-to-enable-addresssanitizer-in-cmake-that-works-in-xcode I suspect we'll need something similar to the following which was added to |
|
@clanmills we added the If the investigation you are doing is going to take long, I propose to merge the branch in the current status so that we have at least few fixes for 0.27 RC1. We can investigate the other issues later. |
|
Oh, I know you added I'll fix the ASAN issues in I'll open a new issue about |
There was a problem hiding this comment.
I've reviewed PngChunk::readRawProfile() again. My previous comments stand. The code is galloping along a buffer without checking for the end.
I don't know the Specification for this code. From the comments, this has been specified by ImageMagick. It seems that an Exif (&Iptc) binary metadata block is encoded as hex nibbles which are decoded. If the decoding fails, the chunk is returned as raw binary. These raw chunks can contain an ICC profile or XMP/xml.
DataBuf PngChunk::readRawProfile(const DataBuf& text,bool iTXt)
{
DataBuf info;
unsigned char unhex[103]={0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,1, 2,3,4,5,6,7,8,9,0,0,
0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,10,11,12,
13,14,15};
if (text.size_ == 0) {
return DataBuf();
}
if ( iTXt ) {
info.alloc(text.size_);
::memcpy(info.pData_,text.pData_,text.size_);
return info;
}
const char *sp = (char*) text.pData_+1; // current byte (space pointer)
const char *eot = (char*) text.pData_+text.size_; // end of text
// Look for newline
while (*sp != '\n' && sp < eot )
{
sp++;
if ( sp == eot )
{
return DataBuf();
}
}
sp++ ; // step over '\n'
// Look for length
while ( (*sp == '\0' || *sp == ' ' || *sp == '\n') && sp < eot )
{
sp++;
if (sp == eot )
{
return DataBuf();
}
}
const char* startOfLength = sp;
while ( ('0' <= *sp && *sp <= '9') && sp < eot)
{
sp++;
if (sp == eot )
{
return DataBuf();
}
}
sp++ ; // step over '\n'
long length = (long) atol(startOfLength);
// Allocate space
if (length == 0)
{
#ifdef DEBUG
std::cerr << "Exiv2::PngChunk::readRawProfile: Unable To Copy Raw Profile: invalid profile length\n";
#endif
}
info.alloc(length);
if (info.size_ != length)
{
#ifdef DEBUG
std::cerr << "Exiv2::PngChunk::readRawProfile: Unable To Copy Raw Profile: cannot allocate memory\n";
#endif
return DataBuf();
}
// Copy profile, skipping white space and column 1 "=" signs
unsigned char *dp = (unsigned char*)info.pData_; // decode pointer
unsigned int nibbles = length * 2;
for (long i = 0; i < (long) nibbles; i++)
{
while (*sp < '0' || (*sp > '9' && *sp < 'a') || *sp > 'f')
{
if (*sp == '\0')
{
#ifdef DEBUG
std::cerr << "Exiv2::PngChunk::readRawProfile: Unable To Copy Raw Profile: ran out of data\n";
#endif
return DataBuf();
}
sp++;
}
if (i%2 == 0)
*dp = (unsigned char) (16*unhex[(int) *sp++]);
else
(*dp++) += unhex[(int) *sp++];
}
return info;
} // PngChunk::readRawProfile
- Now it takes into account text.size_ when searching for a newline char.
|
Thanks for your help! 👍 |
This should fix the problem described in #428. It also adds a new regression test to reproduce the issue.