Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to stream based API #399

Open
wants to merge 59 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
42a9e3b
Switch to stream based API
LinusU Sep 29, 2016
beeb22d
Cleanup tests
LinusU Sep 30, 2016
e5a0c4f
Pre-create all fields when in object mode
LinusU Sep 30, 2016
fff81bd
Bump standard to 8.3.0
LinusU Sep 30, 2016
c7d1487
Throw error when passing old options
LinusU Oct 1, 2016
3719e0f
2.0.0-alpha.1
LinusU Oct 1, 2016
4a5c707
Use LIMIT_FILE_COUNT when receiving too many files
LinusU Oct 2, 2016
2bda1c3
2.0.0-alpha.2
LinusU Oct 2, 2016
a862274
Backport test from #380
LinusU Dec 22, 2016
bdb8763
Unlink file as soon as it's opened
LinusU Dec 22, 2016
b1e5ea0
2.0.0-alpha.3
LinusU Dec 22, 2016
fd3e47d
Update changelog
LinusU Jan 9, 2017
1923ac2
Add file type detection
LinusU Jan 25, 2017
5dfa540
Merge pull request #451 from expressjs/mime-detection
LinusU Feb 11, 2017
267bb2b
Add 2.0.0-alpha.4 to changelog
LinusU Feb 14, 2017
3973e33
2.0.0-alpha.4
LinusU Feb 14, 2017
3a94500
Allow files without filename
LinusU Feb 14, 2017
e7f249e
Add 2.0.0-alpha.5 to changelog
LinusU Feb 14, 2017
ea47167
2.0.0-alpha.5
LinusU Feb 14, 2017
41f39ed
Handle client aborting request
LinusU Feb 18, 2017
f54762a
Add 2.0.0-alpha.6 to changelog
LinusU Feb 18, 2017
c4b223e
2.0.0-alpha.6
LinusU Feb 18, 2017
2b49f4f
Drop support for Node.js < 6.x
LinusU May 3, 2019
cc381d6
Add 2.0.0-alpha.7 to changelog
LinusU May 3, 2019
ac38af4
2.0.0-alpha.7
LinusU May 3, 2019
8d5219a
Update Travis CI Node.js versions
LinusU May 3, 2019
14b4ba0
Lower dependency on hasha for Node 6 compat
LinusU May 3, 2019
f3995ea
Disable package lock files
LinusU Nov 23, 2019
6001d34
Drop support for Node.js < 8.3.x
LinusU Nov 23, 2019
e82146d
Fix typo of "argument"
LinusU Nov 23, 2019
6c26419
Add 2.0.0-beta.1 to changelog
LinusU Nov 23, 2019
a415ee5
2.0.0-beta.1
LinusU Nov 23, 2019
ba6fd35
Add macOS & Windows to Travis CI
LinusU Nov 23, 2019
f3d9789
Merge pull request #807 from expressjs/test-win-macos
LinusU Nov 23, 2019
743b7a6
Tell Git to treat test files as binary
LinusU Feb 26, 2020
3b8fd94
Drop support for Node.js 8.x
LinusU Feb 26, 2020
db32edc
Add 100% code coverage
LinusU Feb 26, 2020
c569242
Remove deprecated "sudo" key
LinusU Feb 26, 2020
c15b6f2
Add 2.0.0-rc.1 to changelog
LinusU Feb 26, 2020
f94bdc2
2.0.0-rc.1
LinusU Feb 26, 2020
d762ab4
Bump Standard to 14.3.3
LinusU Mar 15, 2020
138e180
Hide limits behind a symbol
LinusU Mar 15, 2020
fb08b9f
Rework default limits
LinusU Mar 15, 2020
ee8b4ef
Add 2.0.0-rc.2 to changelog
LinusU Mar 15, 2020
639e359
2.0.0-rc.2
LinusU Mar 15, 2020
f181839
Improve documentation of limits
LinusU Mar 17, 2020
b5ca4b5
Bump version of stream-file-type
LinusU Mar 25, 2020
ed4ce55
Improve documentation on file properties
LinusU Mar 25, 2020
08d0c24
Convert package to ESM
LinusU Jul 26, 2021
fde725d
2.0.0-rc.3
LinusU Jul 29, 2021
ddc1727
Fix compatibility with Node.js <16
LinusU Aug 9, 2021
ba4e105
Switch from Travis CI to Github Actions
LinusU Aug 9, 2021
cb526fb
Run CI on pushes to explore-new-api branch
LinusU Sep 2, 2021
b13ce1d
Run CI tests on Windows as well
LinusU Sep 2, 2021
54061af
Update fs-temp
BlakeB415 Nov 3, 2021
690f703
Merge pull request #1043 from BlakeB415/explore-new-api
LinusU Nov 19, 2021
74456fc
Replace unsupported busboy version with fastify fork
kibertoad Dec 4, 2021
15f7e80
Merge pull request #1056 from kibertoad/chore/fastify-busboy
LinusU May 27, 2022
d983eb2
2.0.0-rc.4
LinusU May 27, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,6 @@ node_modules
*.log
*.gz


# Coveralls
coverage
# Code Coverage
/.nyc_output/
/coverage/
1 change: 1 addition & 0 deletions .npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
package-lock=false
15 changes: 8 additions & 7 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
sudo: false
language: node_js
node_js:
- "0.10"
- "0.12"
- "iojs-v1.8.4"
- "iojs-v2.5.0"
- "iojs-v3.3.0"
- "4"
- '13'
- '12'
- '10'
- '10.13.0'
os:
- linux
- osx
- windows
45 changes: 45 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,51 @@
All notable changes to this project will be documented in this file.
This project adheres to [Semantic Versioning](http://semver.org/).

## 2.0.0-rc.2 - 2020-03-15

- Allow limits to be passed as string, e.g. `'12MB'`
- Remove `parts` limit in favour of `fields` & `files`
- Set reasonable defaults for all limits

## 2.0.0-rc.1 - 2020-02-26

- Breaking: drop support for Node.js < 10.13.x
- Internal: achive 100% code coverage
- Internal: test on macOS & Windows

## 2.0.0-beta.1 - 2019-11-23

- Breaking: drop support for Node.js < 8.3.x

## 2.0.0-alpha.7 - 2019-05-03

- Breaking: drop support for Node.js < 6.x

## 2.0.0-alpha.6 - 2017-02-18

- Fix: handle client aborting request

## 2.0.0-alpha.5 - 2017-02-14

- Fix: allow files without filename

## 2.0.0-alpha.4 - 2017-02-14

- Feature: add file type detection

## 2.0.0-alpha.3 - 2016-12-22

- Feature: unlink file as soon as it's opened

## 2.0.0-alpha.2 - 2016-10-02

- Feature: use LIMIT_FILE_COUNT when receiving too many files

## 2.0.0-alpha.1 - 2016-10-01

- Feature: switch to stream based API
- Feature: throw error when passing old options

## 1.2.0 - 2016-08-04

- Feature: add .none() for accepting only fields
Expand Down
197 changes: 44 additions & 153 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on top of [busboy](https://github.com/mscdex/busboy) for maximum efficiency.
## Installation

```sh
$ npm install --save multer
npm install --save multer
```

## Usage
Expand All @@ -18,24 +18,24 @@ Multer adds a `body` object and a `file` or `files` object to the `request` obje
Basic usage example:

```javascript
var express = require('express')
var multer = require('multer')
var upload = multer({ dest: 'uploads/' })
const multer = require('multer')
const express = require('express')

var app = express()
const app = express()
const upload = multer()

app.post('/profile', upload.single('avatar'), function (req, res, next) {
app.post('/profile', upload.single('avatar'), (req, res, next) => {
// req.file is the `avatar` file
// req.body will hold the text fields, if there were any
})

app.post('/photos/upload', upload.array('photos', 12), function (req, res, next) {
app.post('/photos/upload', upload.array('photos', 12), (req, res, next) => {
// req.files is array of `photos` files
// req.body will contain the text fields, if there were any
})

var cpUpload = upload.fields([{ name: 'avatar', maxCount: 1 }, { name: 'gallery', maxCount: 8 }])
app.post('/cool-profile', cpUpload, function (req, res, next) {
const cpUpload = upload.fields([{ name: 'avatar', maxCount: 1 }, { name: 'gallery', maxCount: 8 }])
app.post('/cool-profile', cpUpload, (req, res, next) => {
// req.files is an object (String -> Array) where fieldname is the key, and the value is array of files
//
// e.g.
Expand All @@ -46,15 +46,16 @@ app.post('/cool-profile', cpUpload, function (req, res, next) {
})
```

In case you need to handle a text-only multipart form, you can use any of the multer methods (`.single()`, `.array()`, `fields()`). Here is an example using `.array()`:
In case you need to handle a text-only multipart form, you can use the `.none()` method, example:

```javascript
var express = require('express')
var app = express()
var multer = require('multer')
var upload = multer()
const multer = require('multer')
const express = require('express')

app.post('/profile', upload.array(), function (req, res, next) {
const app = express()
const upload = multer()

app.post('/profile', upload.none(), (req, res, next) => {
// req.body contains the text fields
})
```
Expand All @@ -65,45 +66,27 @@ app.post('/profile', upload.array(), function (req, res, next) {

Each file contains the following information:

Key | Description | Note
--- | --- | ---
`fieldname` | Field name specified in the form |
`originalname` | Name of the file on the user's computer |
`encoding` | Encoding type of the file |
`mimetype` | Mime type of the file |
`size` | Size of the file in bytes |
`destination` | The folder to which the file has been saved | `DiskStorage`
`filename` | The name of the file within the `destination` | `DiskStorage`
`path` | The full path to the uploaded file | `DiskStorage`
`buffer` | A `Buffer` of the entire file | `MemoryStorage`

### `multer(opts)`

Multer accepts an options object, the most basic of which is the `dest`
property, which tells Multer where to upload the files. In case you omit the
options object, the files will be kept in memory and never written to disk.

By default, Multer will rename the files so as to avoid naming conflicts. The
renaming function can be customized according to your needs.

The following are the options that can be passed to Multer.

Key | Description
--- | ---
`dest` or `storage` | Where to store the files
`fileFilter` | Function to control which files are accepted
`limits` | Limits of the uploaded data
`fieldName` | Field name specified in the form
`originalName` | Name of the file on the user's computer (`undefined` if no filename was supplied by the client)
`size` | Size of the file in bytes
LinusU marked this conversation as resolved.
Show resolved Hide resolved
`stream` | Stream of file
LinusU marked this conversation as resolved.
Show resolved Hide resolved
`detectedMimeType` | The detected mime-type, or null if we failed to detect
`detectedFileExtension` | The typical file extension for files of the detected type, or empty string if we failed to detect (with leading `.` to match `path.extname`)
`clientReportedMimeType` | The mime type reported by the client using the `Content-Type` header, or null<sup>1</sup> if the header was absent
`clientReportedFileExtension` | The extension of the file uploaded (as reported by `path.extname`)

In an average web app, only `dest` might be required, and configured as shown in
the following example.
<sup>1</sup> Currently returns `text/plain` if header is absent, this is a bug and it will be fixed in a patch release. Do not rely on this behavior.

```javascript
var upload = multer({ dest: 'uploads/' })
```
### `multer(opts)`

Multer accepts an options object, the following are the options that can be
passed to Multer.

If you want more control over your uploads, you'll want to use the `storage`
option instead of `dest`. Multer ships with storage engines `DiskStorage`
and `MemoryStorage`; More engines are available from third parties.
Key | Description
-------- | -----------
`limits` | Limits of the uploaded data [(full description)](#limits)

#### `.single(fieldname)`

Expand Down Expand Up @@ -146,107 +129,24 @@ Never add multer as a global middleware since a malicious user could upload
files to a route that you didn't anticipate. Only use this function on routes
where you are handling the uploaded files.

### `storage`

#### `DiskStorage`

The disk storage engine gives you full control on storing files to disk.

```javascript
var storage = multer.diskStorage({
destination: function (req, file, cb) {
cb(null, '/tmp/my-uploads')
},
filename: function (req, file, cb) {
cb(null, file.fieldname + '-' + Date.now())
}
})

var upload = multer({ storage: storage })
```

There are two options available, `destination` and `filename`. They are both
functions that determine where the file should be stored.

`destination` is used to determine within which folder the uploaded files should
be stored. This can also be given as a `string` (e.g. `'/tmp/uploads'`). If no
`destination` is given, the operating system's default directory for temporary
files is used.

**Note:** You are responsible for creating the directory when providing
`destination` as a function. When passing a string, multer will make sure that
the directory is created for you.

`filename` is used to determine what the file should be named inside the folder.
If no `filename` is given, each file will be given a random name that doesn't
include any file extension.

**Note:** Multer will not append any file extension for you, your function
should return a filename complete with an file extension.

Each function gets passed both the request (`req`) and some information about
the file (`file`) to aid with the decision.

Note that `req.body` might not have been fully populated yet. It depends on the
order that the client transmits fields and files to the server.

#### `MemoryStorage`

The memory storage engine stores the files in memory as `Buffer` objects. It
doesn't have any options.

```javascript
var storage = multer.memoryStorage()
var upload = multer({ storage: storage })
```

When using memory storage, the file info will contain a field called
`buffer` that contains the entire file.

**WARNING**: Uploading very large files, or relatively small files in large
numbers very quickly, can cause your application to run out of memory when
memory storage is used.

### `limits`

An object specifying the size limits of the following optional properties. Multer passes this object into busboy directly, and the details of the properties can be found on [busboy's page](https://github.com/mscdex/busboy#busboy-methods).

The following integer values are available:
The following limits are available:

Key | Description | Default
--- | --- | ---
`fieldNameSize` | Max field name size | 100 bytes
`fieldSize` | Max field value size | 1MB
`fields` | Max number of non-file fields | Infinity
`fileSize` | For multipart forms, the max file size (in bytes) | Infinity
`files` | For multipart forms, the max number of file fields | Infinity
`parts` | For multipart forms, the max number of parts (fields + files) | Infinity
`headerPairs` | For multipart forms, the max number of header key=>value pairs to parse | 2000

Specifying the limits can help protect your site against denial of service (DoS) attacks.

### `fileFilter`
`fieldNameSize` | Max field name size | `'100B'`
`fieldSize` | Max field value size | `'8KB'`
`fields` | Max number of non-file fields | `1000`
`fileSize` | The max file size | `'8MB'`
`files` | The max number of file fields | `10`
`headerPairs` | The max number of header key=>value pairs to parse | `2000` (same as Node's http)

Set this to a function to control which files should be uploaded and which
should be skipped. The function should look like this:
Bytes limits can be passed either as a number, or as a string with an appropriate prefix.

```javascript
function fileFilter (req, file, cb) {

// The function should call `cb` with a boolean
// to indicate if the file should be accepted

// To reject this file pass `false`, like so:
cb(null, false)

// To accept the file pass `true`, like so:
cb(null, true)

// You can always pass an error if something goes wrong:
cb(new Error('I don\'t have a clue!'))

}
```
Specifying the limits can help protect your site against denial of service (DoS) attacks.

## Error handling

Expand All @@ -257,10 +157,10 @@ If you want to catch errors specifically from multer, you can call the
middleware function by yourself.

```javascript
var upload = multer().single('avatar')
const upload = multer().single('avatar')

app.post('/profile', function (req, res) {
upload(req, res, function (err) {
app.post('/profile', (req, res) => {
upload(req, res, (err) => {
if (err) {
// An error occurred when uploading
return
Expand All @@ -270,12 +170,3 @@ app.post('/profile', function (req, res) {
})
})
```

## Custom storage engine

See [the documentation here](/StorageEngine.md) if you want to build your own
storage engine.

## License

[MIT](LICENSE)
Loading