How to detect the file type?

[gknpezgssb]

To detecting whether the uploaded file type is jpg / png or not,if not throw an ERROR.
which API should I use and how？

[tunnckoCore]

@gknpezgssb hi there. You can use the file event which is passed with the name and file arguments. Where file is an object, containing some properties. So you can use file.type which gives you the mime type of the file. Or the other way is getting it from the name argument.

Let us know if you need more help. We are looking to improve the docs.

edit: example

form.on('file', function (name, file) {
  console.log(name) // => string
  console.log(file) // => object
  console.log(file.type) // => mime type
})

Or the other variant is if you can check it from the .parse's method callback which is passed with 3 arguments - err, fields, files. Where fields and files are an arrays. Each entry in files array is an object same file as previously mentioned.

[DennisJames]

@olstenlarck the file event emits after storing the file into Disk. But how about detecting the file's type before the program storing it to disk? So that I can interrupt uploading if the file is in wrong type.

[DennisJames]

I have found my solution. Thanks

form.onPart = function(part) {
  if (!part.filename) {
    // let formidable handle all non-file parts
    form.handlePart(part);
  }
}

[tunnckoCore]

Good point. Hm, probably it's better to use the .on('part'). It is the same but I more like that api.

[DennisJames]

but there is no part event, right? I cann't find the source.

[tunnckoCore]

ha, strange! my fault haha

[xarguments]

Agree with @olstenlarck.

The correct place to do all validations (as soon as possible) is in form.onPart().
Here is a simplified example:

form.onPart = function (part) {
  if (part.filename) {
    if (isInvalidFileName(part.filename) || isInvalidMimeType(part.mime)) {
      res.sendStatus(400);
    }
  }
}

[xarguments]

@olstenlarck, you added there "docs" tag. Does it need something to document? Or it was back then when there was no "question" tag?

P.S. If there's no need to document it, then lets close it. Otherwise, lets create another issue for documenting, and then close this.

[tunnckoCore]

Re-reading the thread... and realizing that i still promote .on('part') API, haha.

the file event emits after storing the file into Disk.

@DennisJames that's totally another issues - probably the most talked - more functionality in the "before saving to disk" step.

@xarguments, yes, it was before the question label. And it's always good thing to add docs. Or at least add such code snippets in the readme. Or probably what about recipes (more trendy nowadays? i had such in koa-better-body) folder which will include examples and some actual words with explanation what, how and why.

And yea, probably may close and open one for the docs and link that issue for reminder. Actually there is one such issue that i opened #414 - there we can collect such things.

[droddy]

Should I be able to use form.onPart() along with form.parse() in order to

form.onPart(): test for valid file types and return validation message if that fails or continue on to...

form.parse(): parse the file if the validation in form.onPart() passes?

I can't seem to find the right way to do this.

CONTROLLER:

    this.router.post(
      '/file',
      this.uploadFile,
    );
  }
  formidableOptions = {
    encoding: 'utf-8',
    keepExtensions: true,
    // 3 mb for news image and attachments. override otherwise
    maxFileSize: 3 * 1024 * 1024,
  };

  uploadFile = (req: Request, res: Response) => {
    const typeValidationMessage = 'Please choose a JPEG, GIF, PDF or DOC file.';

    const form = formidable(this.formidableOptions);

    form.onPart = (part: { filename: any; mime: any }) => {
      let isValid = true;
      if (part.filename || part.mime) {
        if (part.filename) {
          isValid =
            isValid && !AttachmentValidator.isInvalidFileName(part.filename);
        }

        if (part.mime) {
          isValid =
            isValid && !AttachmentValidator.isInvalidFileType(part.mime);
        }
        if (!isValid) {
          return res.status(400).json({ message: typeValidationMessage });
        }
      }
    };

    try {
      UploadService.upload(
        req,
        // onSuccess callback
        async (fileLocation: string, originalName: string) => {
          try {
            await C3DistributionService.sendAttachment(
              fileLocation,
              originalName,
            );
          } catch (error) {
            this.logger.error(error);
            return res.status(500).json({ message: error.message });
          } finally {
            // try catch swallow in case only the delete tmp file (unlinkSync) fails
            try {
              fs.unlinkSync(fileLocation);
            } catch (error) {
              this.logger.error(error);
            }
          }
          return res.status(201).send();
        },
        // onFail callback
        () => {
          return res.status(500).json({
            message: 'unexpected error from upload service callback run',
          });
        },
      );
    } catch (error) {
      this.logger.error(error);
      return res.status(500).json({ message: error.message });
    }
  };
}

SERVICE/formidable parse wrapper:

  encoding: 'utf-8',
  uploadDir: UPLOAD_DIR,
  keepExtensions: true,
  // 3 mb for news image and attachments. override otherwise
  maxFileSize: 3 * 1024 * 1024,
};

const UploadService = {
  upload: (req: Request, onSuccess?: Function, onFail?: Function): void => {
    ensureTmpFilePath();

    const form = formidable(formidableOptions);

    form.parse(req, (err: any, fields: any, files: { files: File }) => {
      if (err) {
        logger.error('formidable err: %j', err);
        if (onFail) {
          onFail();
        }
      } else if (onSuccess) {
        onSuccess(files.files.path, files.files.name);
      }
    });
  },
};

[jithinlalpr]

you can use form.handlePart(part) inside the form.onPart callback

[men232]

This is how I solve it. In this way, I can keep using filter options and pause fileWriteStreamHandler till the file type is validated.

I don't test it how it works with base64 encoding, so use on own risk

import formidable from 'formidable';
import { fileTypeFromBuffer, supportedMimeTypes } from 'file-type';

export function formidableWithAnalyzing(options = {}) {
  const form = formidable(options);

  // The minimum to catch file header
  const minimumBytes = 4100;

  // Assign fieldName
  form.on('fileBegin', (fieldName, file) => {
    file.fieldName = fieldName;
  });

  form.onPart = function (part) {
    // Run analyzing only for supported types
    // just the way to exclude text/*
    part.mimetypeAnalyzing = (
      part.mimetype === 'application/octet-stream' ||
      supportedMimeTypes.has(part.mimetype)
    );

    part.mimetypeDetected = false;
    part.mimetypeOriginal = part.mimetype;
    part.mimetypeValid = false;

    if (!part.mimetypeAnalyzing) {
      form._handlePart(part);
      return;
    }

    const encoding = part.transferEncoding === 'utf-8' ? 'utf8' : 'binary';

    let cacheBuffer = Buffer.alloc(0, 0, encoding);

    let partEnd = false;
    let partAnalyzing = false;

    const catchUpEnd = () => {
      if (!partEnd) return;
      part.emit('end');
    };

    const catchUpBuffer = () => {
      if (cacheBuffer) {
        part.emit('data', cacheBuffer);
        cacheBuffer = undefined;
      }
    };

    part.on('end', onPartEnd);
    part.on('data', onPartData);

    function onPartEnd() {
      partEnd = true;

      // Not reached minimum bytes, so let's try to analyze what we have
      if (!partAnalyzing) {
        partAnalyzing = true;
        analyzeFileType();
      }
    }

    function onPartData(chunk) {
      if (chunk.length > 0) {
        cacheBuffer = Buffer.concat([cacheBuffer, chunk]);
      }

      if (!partAnalyzing && cacheBuffer.length >= minimumBytes) {
        partAnalyzing = true;
        analyzeFileType();
      }
    }

    function analyzeFileType() {
      form.pause();
      form._flushing += 1; // pause end trigger

      const analyzing = cacheBuffer
        ? fileTypeFromBuffer(cacheBuffer)
        : Promise.resolve(null);

      analyzing
        .then((r) => {
          if (r?.mime) {
            part.mimetypeDetected = true;
            part.mimetypeValid = part.mimetypeOriginal === r.mime;
            part.mimetype = r.mime; // replce it to keep use in filter options.
          }

          // De-attach our handlers
          part.off('data', onPartData);
          part.off('end', onPartEnd);

          // Setup native handlers
          form._flushing -= 1;
          form._handlePart(part);
          form.resume();

          // Catch-up data
          catchUpBuffer();

          // Catch-up end event
          catchUpEnd();
        })
        .catch((err) => {
          console.error('Error occurred while file mimetype analyzing', err);

          // Throw error on global level
          form._error(
            new FormidableError(
              'Failed to parse file content type',
              errors.pluginFailed,
              400,
            ),
          );
        });
    }
  };

  return form;
}

On my project, i need extra support of application/octet-stream you can remove this condition.
Now you able to use extra properties in filter function

function filter(part) {
  const { 
    mimetype,
    mimetypeValid,
    mimetypeOriginal,
    mimetypeDetected,
    mimetypeAnalyzing
  } = part;

  // Seems user uploaded the file with an incorrect file extension.
  if (mimetypeAnalyzing && !mimetypeValid) {
    return false;
  }

  return true;
}