corrupt attachments: contentHandler issue?

[alexh8]

Hello,

I've just compiled the latest source of vmime with g++ 4.9.0 (i686-win32-dwarf-rev2, Built by MinGW-W64 project).

With this simple code:

messageBuilder mb
shared_ptr«contentHandler» cth = make_shared«stringContentHandler»( "8MB string here");
shared_ptr«attachment» att1 = make_shared«defaultAttachment»(cth, mediaType(mediaTypes::APPLICATION_OCTET_STREAM, mediaTypes::APPLICATION_OCTET_STREAM), text("toto"), word("toto.txt") ); mb.appendAttachment(att1);

I get like 2 modified characters inside the attachment, each 2MB or so... (didn't really find any scheme in that).
When I write the string in a file with std::ofstream, the file content has no problem.

Any clue?

Thanks for the help :)

[vincent-richard]

Hi!

I didn't really understood the problem. Could you please provide a minimal working example code which reproduces the problem, the result that is expected, and the result that is actually returned.

By the way, does the problem also occur with smaller contents?

Thanks,

Vincent

[alexh8]

Hello,

Thanks for your quick answer.

Here is the full code:

#include <iostream>
#include <fstream>

// vmime
#include <vmime/vmime.hpp>
#include <vmime/platforms/windows/windowsHandler.hpp>


using namespace std;
using namespace vmime;


int main()
{
    messageBuilder mb;

    // construct 10MB dummy file
    string my_file("");
    for (int i=0; i<(10000*1024); ++i)
        my_file+="1";

    // and save it with ofstream
    std::ofstream sortie("c:\\my_file_ofstream.dat", ios::binary);
    sortie << my_file << flush;

    // and attach it to email
    shared_ptr<contentHandler> cth = make_shared<stringContentHandler>( my_file);
    shared_ptr<attachment> att1 = make_shared<defaultAttachment>(cth, mediaType(mediaTypes::APPLICATION_OCTET_STREAM, mediaTypes::APPLICATION_OCTET_STREAM), text("my_file"), word("my_file_email.dat") );
    mb.appendAttachment(att1);

    // send the email
    mb.getRecipients().appendAddress( make_shared<mailbox>("alex@free.fr") );
    mb.setSubject( text("my file test") );
    mb.setExpeditor( mailbox(text("test"), "test@test.com"));

    shared_ptr<message> msg = mb.construct();

    utility::url url("smtp://192.168.0.1");
    shared_ptr<net::session> sess = make_shared<net::session>();
    shared_ptr<net::transport> tr = sess->getTransport(url) ;
    tr->connect();
    tr->send(msg);
    tr->disconnect();

    return 0;
}

And this is the diff I get between the 2 files:

Apparently, somehow vmime adds 1 or 2 characters regularly into the attachment, problem I didn't have before with VC++2005 and vmime 0.9.1.

What am I doing wrong?
Thanks for your help.

[vincent-richard]

Hi!

Please apologize for the late reply.

One more question: at which moment do you create the file "my_file_email.dat"? This is not in the example code you provided. Is it the result of a msg->generate(), or the attachment you get after the message has been sent over SMTP and received in your mailbox?

In the later case, are you sure the extra chars are not added by a gateway during the transfer of the message?

Please send me the exact code you used to produce the two files, so that I can reproduce the problem in the same conditions.

Thanks,

Vincent

[alexh8]

Hi Vincent,

Thank you for your answer. I appreciate it.

The full code is in my last message. I get "my_file_email.dat" from my email client Outlook 2003 SP3.
What do you mean by "gateway"? The SMTP server? It has been used for years with more than a hundred a day with no problem.

I did some more tests and:

• The 2 files sizes are always identical. So no character is added, but some are modified.
• Surprisingly, when I fill the string with 0x00, the ofstream file and the attachment are purely identical. When I fill with "1" the problem comes back.

I guess the bug is somewhere in the "charset conv". I compiled vmime with VMIME_CHARSETCONV_LIB_IS_WIN=1, perhaps I should try with iconv.

Thanks.

[vincent-richard]

Hi!

Finally, did you try with another charset conversion library? The result of testing this should be a good hint to find out where the issue occurs.

Thanks,

Vincent

[alexh8]

Hi Vincent,

I tested with iconv today (didn't have time to do it before), and result is the same. The attachment is corrupt. But the modified characters seem to more slightly more regular (sometimes 3 characters modified, sometimes 1, see capture).

Any clue?

Thanks.

[vincent-richard]

Hi!

Could you try the following program? (I used "/tmp" instead of "c:" on Linux)

#include <iostream>
#include <fstream>

// vmime
#include <vmime/vmime.hpp>


using namespace std;
using namespace vmime;


int main()
{
    try
    {
       messageBuilder mb;

        // construct 10MB dummy file
        string my_file("");
        for (int i=0; i<(10000*1024); ++i)
            my_file+="1";

        // and save it with ofstream
        std::ofstream sortie("/tmp/my_file_ofstream.dat", ios::binary);
        sortie << my_file << flush;

        // and attach it to email
        shared_ptr<contentHandler> cth = make_shared<stringContentHandler>( my_file);
        shared_ptr<attachment> att1 = make_shared<defaultAttachment>(cth, mediaType(mediaTypes::APPLICATION_OCTET_STREAM, mediaTypes::APPLICATION_OCTET_STREAM), text("my_file"), word("my_file_email.dat") );
        mb.appendAttachment(att1);

        // send the email
        mb.getRecipients().appendAddress( make_shared<mailbox>("test@example.com") );
        mb.setSubject( text("my file test") );
        mb.setExpeditor( mailbox(text("test"), "test@test.com"));

        shared_ptr<message> msg = mb.construct();

        vmime::string msgData = msg->generate();

        vmime::messageParser mp(msgData);

        for (int i = 0 ; i < mp.getAttachmentCount() ; ++i)
        {
            const vmime::attachment& att = *mp.getAttachmentAt(i);

            shared_ptr <const contentHandler> data = att.getData();

            std::ofstream sortie("/tmp/my_file_ofstream2.dat", ios::binary);
            vmime::utility::outputStreamAdapter osa(sortie);

            data->extract(osa);
        }
    }
    // VMime exception
    catch (vmime::exception& e)
    {
        std::cout << "vmime::exception: " << e.what() << std::endl;
        throw;
    }
    // Standard exception
    catch (std::exception& e)
    {
        std::cout << "std::exception: " << e.what() << std::endl;
        throw;
    }

    std::cout << std::endl;

    return 0;
}

Then, do a "diff" on the two files:

vincent:/tmp$ diff /tmp/my_file_ofstream.dat /tmp/my_file_ofstream2.dat

On my host, it found no diff between the files. I am using the latest version of VMime from Git. Could you tell me if there are on yours?

I will try some other tests to see whether there could be a problem when sending data over the SMTP stream.

[alexh8]

Hi,

With your code on my machine, files are identical.

[vincent-richard]

Hello!

Thank you for testing. Finally, I found the issue. It was a bad pointer offsetting when copying data to the actual send buffer, when using SMTP BDAT chunking (the problem did not occur with normal DATA command).

Here is the fix I committed: 07ae82f