-
Notifications
You must be signed in to change notification settings - Fork 1.7k
/
capped-collections.txt
213 lines (149 loc) · 6.54 KB
/
capped-collections.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
==================
Capped Collections
==================
.. default-domain:: mongodb
.. contents:: On this page
:local:
:backlinks: none
:depth: 1
:class: singlecol
Overview
--------
:term:`Capped collections <capped collection>` are fixed-size
collections that support high-throughput operations that insert
and retrieve documents based on insertion order. Capped
collections work in a way similar to circular buffers: once a
collection fills its allocated space, it makes room for new documents
by overwriting the oldest documents in the collection.
See :method:`~db.createCollection()` or :dbcommand:`create`
for more information on creating capped collections.
Behavior
--------
Insertion Order
~~~~~~~~~~~~~~~
Capped collections guarantee preservation of the insertion order. As a
result, queries do not need an index to return documents in insertion
order. Without this indexing overhead, capped collections can support
higher insertion throughput.
Automatic Removal of Oldest Documents
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To make room for new documents, capped collections automatically remove
the oldest documents in the collection without requiring scripts or
explicit remove operations.
For example, the :term:`oplog.rs <oplog>` collection that stores a log
of the operations in a :term:`replica set` uses a capped
collection. Consider the following potential use cases for capped
collections:
- Store log information generated by high-volume systems. Inserting
documents in a capped collection without an index is close to the
speed of writing log information directly to a file
system. Furthermore, the built-in *first-in-first-out* property
maintains the order of events, while managing storage use.
- Cache small amounts of data in a capped collections. Since caches
are read rather than write heavy, you would either need to ensure
that this collection *always* remains in the working set (i.e. in
RAM) *or* accept some write penalty for the required index or
indexes.
``_id`` Index
~~~~~~~~~~~~~
Capped collections have an ``_id`` field and an index on the ``_id``
field by default.
.. _capped-collections-recommendations-and-restrictions:
Restrictions and Recommendations
--------------------------------
Updates
~~~~~~~
If you plan to update documents in a capped collection, create an index
so that these update operations do not require a collection scan.
Document Size
~~~~~~~~~~~~~
.. versionchanged:: 3.2
.. include:: /includes/extracts/capped-collection-immutable-document-size.rst
Document Deletion
~~~~~~~~~~~~~~~~~
You cannot delete documents from a capped collection. To remove all
documents from a collection, use the :method:`~db.collection.drop()`
method to drop the collection and recreate the capped collection.
Sharding
~~~~~~~~
You cannot shard a capped collection.
Query Efficiency
~~~~~~~~~~~~~~~~
Use natural ordering to retrieve the most recently inserted elements
from the collection efficiently. This is (somewhat) analogous to tail
on a log file.
Aggregation ``$out``
~~~~~~~~~~~~~~~~~~~~
The aggregation pipeline operator :pipeline:`$out` cannot write results
to a capped collection.
.. include:: /includes/replacement-mms.rst
Procedures
----------
Create a Capped Collection
~~~~~~~~~~~~~~~~~~~~~~~~~~
You must create capped collections explicitly using the
:method:`db.createCollection()` method, which is a helper in the
:binary:`~bin.mongo` shell for the :dbcommand:`create` command. When
creating a capped collection you must specify the maximum size of the
collection in bytes, which MongoDB will pre-allocate for the collection.
The size of the capped collection includes a small amount of space for
internal overhead.
.. code-block:: javascript
db.createCollection( "log", { capped: true, size: 100000 } )
If the ``size`` field is less than or equal to 4096, then the collection will
have a cap of 4096 bytes. Otherwise, MongoDB will raise the provided size to
make it an integer multiple of 256.
Additionally, you may also specify a maximum number of documents for the
collection using the ``max`` field as in the following document:
.. code-block:: javascript
db.createCollection("log", { capped : true, size : 5242880, max : 5000 } )
.. important:: The ``size`` argument is *always* required, even when
you specify ``max`` number of documents. MongoDB will remove older
documents if a collection reaches the maximum size limit before it
reaches the maximum document count.
.. see:: :method:`db.createCollection()` and :dbcommand:`create`.
.. _capped-collections-options:
Query a Capped Collection
~~~~~~~~~~~~~~~~~~~~~~~~~
If you perform a :method:`~db.collection.find()` on a capped collection
with no ordering specified, MongoDB guarantees that the ordering of
results is the same as the insertion order.
To retrieve documents in reverse insertion order, issue
:method:`~db.collection.find()` along with the :method:`~cursor.sort()`
method with the :operator:`$natural` parameter set to ``-1``, as shown
in the following example:
.. code-block:: javascript
db.cappedCollection.find().sort( { $natural: -1 } )
Check if a Collection is Capped
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use the :method:`~db.collection.isCapped()` method to determine if a
collection is capped, as follows:
.. code-block:: javascript
db.collection.isCapped()
Convert a Collection to Capped
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can convert a non-capped collection to a capped collection with
the :dbcommand:`convertToCapped` command:
.. code-block:: javascript
db.runCommand({"convertToCapped": "mycoll", size: 100000});
The ``size`` parameter specifies the size of the capped collection in
bytes.
.. include:: /includes/warning-blocking-global.rst
Automatically Remove Data After a Specified Period of Time
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For additional flexibility when expiring data, consider MongoDB's
:term:`TTL` indexes, as described in
:doc:`/tutorial/expire-data`. These indexes allow you to expire and
remove data from normal collections using a special type, based on the
value of a date-typed field and a TTL value for the index.
:doc:`TTL Collections </tutorial/expire-data>` are not compatible with
capped collections.
Tailable Cursor
~~~~~~~~~~~~~~~
You can use a :term:`tailable cursor` with capped collections. Similar to the
Unix ``tail -f`` command, the tailable cursor "tails" the end of a
capped collection. As new documents are inserted into the capped
collection, you can use the tailable cursor to continue retrieving
documents.
See :doc:`/core/tailable-cursors` for information on creating
a tailable cursor.