Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOCS-6010: bulkWrite Concept + Reference Page #2438

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
171 changes: 104 additions & 67 deletions source/core/bulk-write-operations.txt
@@ -1,5 +1,3 @@
.. _write-operations-bulk-insert:

=====================
Bulk Write Operations
=====================
Expand All @@ -14,97 +12,134 @@ bulk. Bulk write operations affect a *single* collection. MongoDB
allows applications to determine the acceptable level of
acknowledgement required for bulk write operations.

New :method:`Bulk` methods provide the ability to perform bulk insert,
update, and remove operations. MongoDB also supports bulk insert
through passing an array of documents to the
:method:`db.collection.insert()` method.

.. versionchanged:: 2.6
.. versionadded:: 3.2

Previous versions of MongoDB provided the ability for bulk inserts
only. With previous versions, clients could perform bulk inserts by
passing an array of documents to the :v2.4:`db.collection.insert()
</core/bulk-inserts>` method. To see the documentation for earlier
versions, see :v2.4:`Bulk Inserts </core/bulk-inserts>`.
The :method:`db.collection.bulkWrite()` method provides the ability to
perform bulk insert, update, and remove operations.
MongoDB also supports bulk insert
through the :method:`db.collection.insertMany()`.

Ordered vs Unordered Operations
-------------------------------

Bulk write operations can be either *ordered* or *unordered*. With an
ordered list of operations, MongoDB executes the operations serially.
Bulk write operations can be either *ordered* or *unordered*.

With an ordered list of operations, MongoDB executes the operations serially.
If an error occurs during the processing of one of the write
operations, MongoDB will return without processing any remaining write
operations in the list.
operations in the list.
See :ref:`ordered Bulk Write<bulkwrite-example-bulk-write-operation>`

With an unordered list of operations, MongoDB can execute the
operations in parallel. If an error occurs during the processing of one
operations in parallel, but this behavior is not guaranteed.
If an error occurs during the processing of one
of the write operations, MongoDB will continue to process remaining
write operations in the list.
write operations in the list.
See :ref:`bulkwrite-example-unordered-bulk-write`.

Executing an ordered list of operations on a sharded collection will
generally be slower than executing an unordered list since with an
ordered list, each operation must wait for the previous operation to
finish.

``Bulk`` Methods
----------------
By default, :method:`~db.collection.bulkWrite()` performs ``ordered``
operations. To specify ``unordered`` write operations, set
``ordered : false`` in the options document.

To use the :method:`Bulk()` methods:
See :ref:`bulkwrite-write-operations-executionofoperations`

#. Initialize a list of operations using either
:method:`db.collection.initializeUnorderedBulkOp()` or
:method:`db.collection.initializeOrderedBulkOp()`.
bulkWrite() Methods
-------------------

#. Add write operations to the list using the following methods:
:method:`~db.collection.bulkWrite()` supports the following write operations:

- :method:`Bulk.insert()`
- :method:`Bulk.find()`
- :method:`Bulk.find.upsert()`
- :method:`Bulk.find.update()`
- :method:`Bulk.find.updateOne()`
- :method:`Bulk.find.replaceOne()`
- :method:`Bulk.find.remove()`
- :method:`Bulk.find.removeOne()`
- :ref:`bulkwrite-write-operations-insertOne`
- :ref:`updateOne <bulkwrite-write-operations-updateOneMany>`
- :ref:`updateMany <bulkwrite-write-operations-updateOneMany>`
- :ref:`bulkwrite-write-operations-replaceOne`
- :ref:`deleteOne <bulkwrite-write-operations-deleteOneMany>`
- :ref:`deleteMany <bulkwrite-write-operations-deleteOneMany>`

#. To execute the list of operations, use the :method:`Bulk.execute()`
method. You can specify the write concern for the list in the
:method:`Bulk.execute()` method.
Each write operation is passed to :method:`~db.collection.bulkWrite()` as a
document in an array.

Once executed, you cannot re-execute the list without
reinitializing.
For example, the following performs multiple write operations:

For example,
The ``characters`` collection contains the following documents:

.. code-block:: javascript

var bulk = db.items.initializeUnorderedBulkOp();
bulk.insert( { _id: 1, item: "abc123", status: "A", soldQty: 5000 } );
bulk.insert( { _id: 2, item: "abc456", status: "A", soldQty: 150 } );
bulk.insert( { _id: 3, item: "abc789", status: "P", soldQty: 0 } );
bulk.execute( { w: "majority", wtimeout: 5000 } );

For more examples, refer to the reference page for each
:doc:`/reference/method/js-bulk` method. For information and examples
on performing bulk insert using the :method:`db.collection.insert()`,
see :method:`db.collection.insert()`.
{ "_id" : 1, "char" : "Brisbane", "class" : "monk", "lvl" : 4 },
{ "_id" : 2, "char" : "Eldon", "class" : "alchemist", "lvl" : 3 },
{ "_id" : 3, "char" : "Meldane", "class" : "ranger", "lvl" : 3 }

The following :method:`~db.collection.bulkWrite()` performs multiple
operations on the collection:

.. seealso:: :ref:`rel-notes-write-operations`

``Bulk`` Execution Mechanics
----------------------------
.. code-block:: javascript

When executing an :method:`ordered
<db.collection.initializeOrderedBulkOp()>` list of operations, MongoDB
groups adjacent operations by the :data:`operation type <batchType>`.
When executing an :method:`unordered
<db.collection.initializeUnorderedBulkOp()>` list of operations,
MongoDB groups and may also reorder the operations to increase
performance. As such, when performing *unordered* bulk operations,
applications should not depend on the ordering.
try {
db.characters.bulkWrite(
[
{ insertOne :
{
"document" :
{
"_id" : 4, "char" : "Dithras", "class" : "barbarian", "lvl" : 4
}
}
},
{ insertOne :
{
"document" :
{
"_id" : 5, "char" : "Taeln", "class" : "fighter", "lvl" : 3
}
}
},
{ updateOne :
{
"filter" : { "char" : "Eldon" },
"update" : { $set : { "status" : "Critical Injury" } }
}
},
{ deleteOne :
{ "filter" : { "char" : "Brisbane"} }
},
{ replaceOne :
{
"filter" : { "char" : "Meldane" },
"replacement" : { "char" : "Tanys", "class" : "oracle", "lvl" : 4 }
}
}
]
);
}
catch (e) {
print(e);
}

The operation returns the following:

.. include:: /includes/fact-bulk-operation-batches.rst
.. code-block:: javascript

For more information, see :method:`Bulk.execute()`.
{
"acknowledged" : true,
"deletedCount" : 1,
"insertedCount" : 2,
"matchedCount" : 2,
"upsertedCount" : 0,
"insertedIds" : {
"0" : 4,
"1" : 5
},
"upsertedIds" : {

}
}

For more examples, see
:ref:`bulkWrite() Examples <bulkwrite-example-bulk-write-operation>`

Strategies for Bulk Inserts to a Sharded Collection
---------------------------------------------------
Expand All @@ -123,11 +158,13 @@ distribute the split chunks to the available shards. To avoid this
performance cost, you can pre-split the collection, as described in
:doc:`/tutorial/split-chunks-in-sharded-cluster`.

Insert to Multiple ``mongos``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Unordered Writes to ``mongos``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To parallelize import processes, send bulk insert or insert operations
to more than one :program:`mongos` instance. For *empty* collections,
To improve write performance to sharded clusters, use
:method:`~db.collection.bulkWrite()` with the optional parameter ``ordered``
set to ``false``. :program:`mongos` can attempt to send the writes to
multiple shards simultaneously. For *empty* collections,
first pre-split the collection as described in
:doc:`/tutorial/split-chunks-in-sharded-cluster`.

Expand Down
2 changes: 1 addition & 1 deletion source/core/distributed-write-operations.txt
Expand Up @@ -38,7 +38,7 @@ capacity of a single shard becomes the limit for the insert capacity
of the sharded cluster.

For more information, see :doc:`/administration/sharded-clusters` and
:ref:`write-operations-bulk-insert`.
:doc:`/core/bulk-write-operations`.

.. _write-operations-replica-sets:

Expand Down
55 changes: 55 additions & 0 deletions source/includes/apiargs-method-db.collection.bulkWrite-params.yaml
@@ -0,0 +1,55 @@
arg_name: param
description: |
An array of :method:`~db.collection.bulkWrite()` write
operations.

Valid operations are:

- :ref:`insertOne <bulkwrite-write-operations-insertOne>`

- :ref:`updateOne <bulkwrite-write-operations-updateOneMany>`

- :ref:`updateMany <bulkwrite-write-operations-updateOneMany>`

- :ref:`deleteOne <bulkwrite-write-operations-deleteOneMany>`

- :ref:`deleteMany <bulkwrite-write-operations-deleteOneMany>`

- :ref:`replaceOne <bulkwrite-write-operations-replaceOne>`

See :ref:`bulkwrite-write-operations` for usage of each operation.

interface: method
name: operations
operation: db.collection.bulkWrite
optional: false
position: 1
type:
- array
---
arg_name: param
description: |
A document expressing the :doc:`write concern
</core/write-concern>`. Omit to use the default write concern.

interface: method
name: writeConcern
operation: db.collection.bulkWrite
optional: true
position: 2
type: document
---
arg_name: param
description: |
A boolean specifying whether the :program:`mongod` instance should perform
an ordered or unordered operation execution. Defaults to ``true``.

See :ref:`bulkwrite-write-operations-executionofoperations`

interface: method
name: ordered
operation: db.collection.bulkWrite
optional: true
position: 3
type: boolean
...
4 changes: 4 additions & 0 deletions source/includes/ref-toc-method-collection.yaml
Expand Up @@ -2,6 +2,10 @@ name: ":method:`db.collection.aggregate()`"
file: /reference/method/db.collection.aggregate
description: "Provides access to the :doc:`aggregation pipeline </core/aggregation>`."
---
name: ":method:`db.collection.bulkWrite()`"
file: /reference/method/db.collection.bulkWrite
description: "Provides bulk write operation functionality."
---
name: ":method:`db.collection.count()`"
file: /reference/method/db.collection.count
description: "Wraps :dbcommand:`count` to return a count of the number of documents in a collection or matching a query."
Expand Down