Produces corrupt PDFs

[sandorvasas]

Hey,
I was using your library to convert html to pdf, and sometimes it produces corrupt files. Most probably you're not waiting for a promise somewhere. Please check the code.
I made direct requests to the API and that works well, so I highly suspect the bug is in your code.
Cheers

[tomasr78]

Do you use plain NodeJS or some kind of framework like Electron?

[sandorvasas]

plain node

[laurynas-convertapi]

Hi @sandorvasas, are you able to reproduce the issue?

Could you provide more details - OS version, Node version? Also code enable of how you are calling the API would be great.

[sandorvasas]

OS: Ubuntu 16.04 LTS
Node 8.x

    const finalHtml = ""; /** a complete HTML page. redacted for the example **/

    const savedHtmlPath = path.resolve(path.join(tmpFolder, uuid()+'.html'));
    fs.writeFileSync(savedHtmlPath, finalHtml);

    const result = await convertapi.convert('pdf', { File: savedHtmlPath })
    fs.unlinkSync(savedHtmlPath);

    await result.file.save(path.join(tmpFolder, uuid() + '.pdf'))

[laurynas-convertapi]

Thanks, @sandorvasas, we will investigate it. Do you get corrupt results from this example every time, or only occasionally?

[sandorvasas]

Sorry for the latency. Around 50% of the time. e.g. 4 times it works, 3 times doesn't, then works again, doesn't again, etc.

[laurynas-convertapi]

@sandorvasas maybe you could try downloading result URL as a stream and see if you get different results? Here is an example how to read converted file stream. You could save it to the filesystem instead of string variable:
https://github.com/ConvertAPI/convertapi-node/blob/master/examples/get_result_file_stream.js

[tomasr78]

@sandorvasas Have you found where was the issue?

[sandorvasas]

@tomasr78 I didn't debug further, just ended up using ConvertAPI directly:

import axios, { AxiosRequestConfig, AxiosResponse } from 'axios';
import { Logger } from 'utils/Logger';
import uuid from 'uuid';
const {
  CONVERTAPI_SECRET
} = process.env;

// https://www.convertapi.com/doc/content-types
// POST https://v2.convertapi.com/convert/html/to/pdf?Secret=XXX
// {
//     "Parameters": [
//         {
//             "Name": "File",
//             "FileValue": {
//                 "Name": "my_file.doc",
//                 "Data": "--Base64 encoded file content--"
//             }
//         }
//     ]
// }

export interface PDFOptions {
  footer?        : string
  marginBottomMm?: string
  orientation?   : "portrait"|"landscape"
  marginMm?      : string
}

class CConvertAPIService {

  private log = Logger.get("ConvertAPIService");

  /**
   * docs @ https://www.convertapi.com/html-to-pdf
   */
  public async html2pdf_Base64(html: string, opts?: PDFOptions): Promise<string> {
    const params = {
      Parameters: [
        {
          "Name": "File",
          "FileValue": {
            "Name": uuid() + '.html',
            "Data": Buffer.from(html).toString('base64')
          }
        },
        {
            "Name": "PageOrientation",
            "Value": (opts && opts.orientation) || "portrait",
        },
        {
            "Name": "PageSize",
            "Value": "a4"
        },
        {
            "Name": "MarginTop",
            "Value": (opts && opts.marginMm) || "10"
        },
        {
            "Name": "MarginRight",
            "Value": (opts && opts.marginMm) || "10"
        },
        {
            "Name": "MarginBottom",
            "Value": (opts && opts.marginBottomMm) || (opts && opts.marginMm) || "10"
        },
        {
            "Name": "MarginLeft",
            "Value": (opts && opts.marginMm) || "10"
        },
        {
            "Name": "ConversionDelay",
            "Value": "2" // wait secs so fonts load
        }
      ]
    }

    if (opts && opts.footer) {
      params.Parameters.push({
          "Name": "Footer",
          "Value": opts.footer
      })
    }

    const data = JSON.stringify(params);

    const options : AxiosRequestConfig = {
      url         : 'https://v2.convertapi.com/convert/html/to/pdf',
      params      : {
        Secret: CONVERTAPI_SECRET
      },
      timeout     : 180 * 1000,
      responseType: "json",
      method      : 'POST',
      headers     : {
        'User-Agent'    : 'API',
        'Content-Type'  : 'application/json',
        'Content-Length': Buffer.from(data).length,
        'Cache-Control' : 'no-cache',
      },
      data: data
    };

    try {
      const res : AxiosResponse = await axios(options);
      return res.data.Files[0].FileData;
    }
    catch (e) {
      this.log.error(JSON.stringify(e.response.data, null, 4));
      return null;
    }
  }
}


export const ConvertAPIService = new CConvertAPIService();

[tomasr78]

We can't reproduce the issue, closing this issue for now.

[p3v9d5ui]

@tomasr78 I didn't debug further, just ended up using ConvertAPI directly:

import axios, { AxiosRequestConfig, AxiosResponse } from 'axios';
import { Logger } from 'utils/Logger';
import uuid from 'uuid';
const {
  CONVERTAPI_SECRET
} = process.env;

// https://www.convertapi.com/doc/content-types
// POST https://v2.convertapi.com/convert/html/to/pdf?Secret=XXX
// {
//     "Parameters": [
//         {
//             "Name": "File",
//             "FileValue": {
//                 "Name": "my_file.doc",
//                 "Data": "--Base64 encoded file content--"
//             }
//         }
//     ]
// }

export interface PDFOptions {
  footer?        : string
  marginBottomMm?: string
  orientation?   : "portrait"|"landscape"
  marginMm?      : string
}

class CConvertAPIService {

  private log = Logger.get("ConvertAPIService");

  /**
   * docs @ https://www.convertapi.com/html-to-pdf
   */
  public async html2pdf_Base64(html: string, opts?: PDFOptions): Promise<string> {
    const params = {
      Parameters: [
        {
          "Name": "File",
          "FileValue": {
            "Name": uuid() + '.html',
            "Data": Buffer.from(html).toString('base64')
          }
        },
        {
            "Name": "PageOrientation",
            "Value": (opts && opts.orientation) || "portrait",
        },
        {
            "Name": "PageSize",
            "Value": "a4"
        },
        {
            "Name": "MarginTop",
            "Value": (opts && opts.marginMm) || "10"
        },
        {
            "Name": "MarginRight",
            "Value": (opts && opts.marginMm) || "10"
        },
        {
            "Name": "MarginBottom",
            "Value": (opts && opts.marginBottomMm) || (opts && opts.marginMm) || "10"
        },
        {
            "Name": "MarginLeft",
            "Value": (opts && opts.marginMm) || "10"
        },
        {
            "Name": "ConversionDelay",
            "Value": "2" // wait secs so fonts load
        }
      ]
    }

    if (opts && opts.footer) {
      params.Parameters.push({
          "Name": "Footer",
          "Value": opts.footer
      })
    }

    const data = JSON.stringify(params);

    const options : AxiosRequestConfig = {
      url         : 'https://v2.convertapi.com/convert/html/to/pdf',
      params      : {
        Secret: CONVERTAPI_SECRET
      },
      timeout     : 180 * 1000,
      responseType: "json",
      method      : 'POST',
      headers     : {
        'User-Agent'    : 'API',
        'Content-Type'  : 'application/json',
        'Content-Length': Buffer.from(data).length,
        'Cache-Control' : 'no-cache',
      },
      data: data
    };

    try {
      const res : AxiosResponse = await axios(options);
      return res.data.Files[0].FileData;
    }
    catch (e) {
      this.log.error(JSON.stringify(e.response.data, null, 4));
      return null;
    }
  }
}


export const ConvertAPIService = new CConvertAPIService();

Hi @sandorvasas , did this work better for you? I'm also experiencing something similar to what you mention. There must be a race condition somewhere, but I've pored over my code and there doesn't seem to be anything, so I wonder if it's in the node.js convertApi library.

[tomasr78]

@p3v9d5ui Do you have any chances to debug our library? You would really help us to find the bug if any. We just can't reproduce it.

[p3v9d5ui]

Hi @tomasr78 , it might be this:

  async download(url, path) {
    const options = {
      url,
      timeout: this.api.downloadTimeout * 1000,
      proxy: this.api.proxy,
      responseType: 'stream',
    };

    const response = await axios(options)
      .catch(error => Client.handleError(error));

    response.data.pipe(fs.createWriteStream(path));

    return new Promise((resolve, reject) => {
      response.data.on('end', () => {
        resolve(path);
      });

      response.data.on('error', (error) => {
        reject(new Error(error));
      });
    });
  }

If I'm reading this correctly, this resolves when the reader finishes: response.data.on('end', ..., but it should resolve when the writer finishes. Something like this:

    const writer = fs.createWriteStream(path)
    response.data.pipe(writer);

    return new Promise((resolve, reject) => {
      writer.on('finish', () => {
        resolve(path);
      });

      writer.on('error', (error) => {
        reject(new Error(error));
      });

I went ahead and created my own wrapper for the convertApi http api, and wrote the code this second way, and since then have had no more corrupt files.