Skip to content

Latest commit

 

History

History
1009 lines (857 loc) · 31.9 KB

bulk-api.rst

File metadata and controls

1009 lines (857 loc) · 31.9 KB

/{db}/_all_docs

/{db}/_design_docs

2.2

Sending multiple queries to a database

2.2

Request:

POST /db/_all_docs/queries HTTP/1.1
Content-Type: application/json
Accept: application/json
Host: localhost:5984

{
    "queries": [
        {
            "keys": [
                "meatballs",
                "spaghetti"
            ]
        },
        {
            "limit": 3,
            "skip": 2
        }
    ]
}

Response:

HTTP/1.1 200 OK
Cache-Control: must-revalidate
Content-Type: application/json
Date: Wed, 20 Dec 2017 11:17:07 GMT
ETag: "1H8RGBCK3ABY6ACDM7ZSC30QK"
Server: CouchDB (Erlang/OTP)
Transfer-Encoding: chunked

{
    "results" : [
        {
            "rows": [
                {
                    "id": "meatballs",
                    "key": "meatballs",
                    "value": 1
                },
                {
                    "id": "spaghetti",
                    "key": "spaghetti",
                    "value": 1
                }
            ],
            "total_rows": 3
        },
        {
            "offset" : 2,
            "rows" : [
                {
                    "id" : "Adukiandorangecasserole-microwave",
                    "key" : "Aduki and orange casserole - microwave",
                    "value" : [
                        null,
                        "Aduki and orange casserole - microwave"
                    ]
                },
                {
                    "id" : "Aioli-garlicmayonnaise",
                    "key" : "Aioli - garlic mayonnaise",
                    "value" : [
                        null,
                        "Aioli - garlic mayonnaise"
                    ]
                },
                {
                    "id" : "Alabamapeanutchicken",
                    "key" : "Alabama peanut chicken",
                    "value" : [
                        null,
                        "Alabama peanut chicken"
                    ]
                }
            ],
            "total_rows" : 2667
        }
    ]
}

Note

The multiple queries are also supported in /db/_local_docs/queries and /db/_design_docs/queries (similar to /db/_all_docs/queries).

/{db}/_bulk_get

/{db}/_bulk_docs

Inserting Documents in Bulk

Each time a document is stored or updated in CouchDB, the internal B-tree is updated. Bulk insertion provides efficiency gains in both storage space, and time, by consolidating many of the updates to intermediate B-tree nodes.

It is not intended as a way to perform ACID-like transactions in CouchDB, the only transaction boundary within CouchDB is a single update to a single database. The constraints are detailed in api/db/bulk_docs/semantics.

To insert documents in bulk into a database you need to supply a JSON structure with the array of documents that you want to add to the database. You can either include a document ID, or allow the document ID to be automatically generated.

For example, the following update inserts three new documents, two with the supplied document IDs, and one which will have a document ID generated:

POST /source/_bulk_docs HTTP/1.1
Accept: application/json
Content-Length: 323
Content-Type: application/json
Host: localhost:5984

{
    "docs": [
        {
            "_id": "FishStew",
            "servings": 4,
            "subtitle": "Delicious with freshly baked bread",
            "title": "FishStew"
        },
        {
            "_id": "LambStew",
            "servings": 6,
            "subtitle": "Serve with a whole meal scone topping",
            "title": "LambStew"
        },
        {
            "servings": 8,
            "subtitle": "Hand-made dumplings make a great accompaniment",
            "title": "BeefStew"
        }
    ]
}

The return type from a bulk insertion will be 201, with the content of the returned structure indicating specific success or otherwise messages on a per-document basis.

The return structure from the example above contains a list of the documents created, here with the combination and their revision IDs:

HTTP/1.1 201 Created
Cache-Control: must-revalidate
Content-Length: 215
Content-Type: application/json
Date: Sat, 26 Oct 2013 00:10:39 GMT
Server: CouchDB (Erlang OTP)

[
    {
        "id": "FishStew",
        "ok": true,
        "rev": "1-6a466d5dfda05e613ba97bd737829d67"
    },
    {
        "id": "LambStew",
        "ok": true,
        "rev": "1-648f1b989d52b8e43f05aa877092cc7c"
    },
    {
        "id": "00a271787f89c0ef2e10e88a0c0003f0",
        "ok": true,
        "rev": "1-e4602845fc4c99674f50b1d5a804fdfa"
    }
]

For details of the semantic content and structure of the returned JSON see api/db/bulk_docs/semantics. Conflicts and validation errors when updating documents in bulk must be handled separately; see api/db/bulk_docs/validation.

Updating Documents in Bulk

The bulk document update procedure is similar to the insertion procedure, except that you must specify the document ID and current revision for every document in the bulk update JSON string.

For example, you could send the following request:

POST /recipes/_bulk_docs HTTP/1.1
Accept: application/json
Content-Length: 464
Content-Type: application/json
Host: localhost:5984

{
    "docs": [
        {
            "_id": "FishStew",
            "_rev": "1-6a466d5dfda05e613ba97bd737829d67",
            "servings": 4,
            "subtitle": "Delicious with freshly baked bread",
            "title": "FishStew"
        },
        {
            "_id": "LambStew",
            "_rev": "1-648f1b989d52b8e43f05aa877092cc7c",
            "servings": 6,
            "subtitle": "Serve with a whole meal scone topping",
            "title": "LambStew"
        },
        {
            "_id": "BeefStew",
            "_rev": "1-e4602845fc4c99674f50b1d5a804fdfa",
            "servings": 8,
            "subtitle": "Hand-made dumplings make a great accompaniment",
            "title": "BeefStew"
        }
    ]
}

The return structure is the JSON of the updated documents, with the new revision and ID information:

HTTP/1.1 201 Created
Cache-Control: must-revalidate
Content-Length: 215
Content-Type: application/json
Date: Sat, 26 Oct 2013 00:10:39 GMT
Server: CouchDB (Erlang OTP)

[
    {
        "id": "FishStew",
        "ok": true,
        "rev": "2-2bff94179917f1dec7cd7f0209066fb8"
    },
    {
        "id": "LambStew",
        "ok": true,
        "rev": "2-6a7aae7ac481aa98a2042718d09843c4"
    },
    {
        "id": "BeefStew",
        "ok": true,
        "rev": "2-9801936a42f06a16f16c30027980d96f"
    }
]

You can optionally delete documents during a bulk update by adding the _deleted field with a value of true to each document ID/revision combination within the submitted JSON structure.

The return type from a bulk insertion will be 201, with the content of the returned structure indicating specific success or otherwise messages on a per-document basis.

The content and structure of the returned JSON will depend on the transaction semantics being used for the bulk update; see api/db/bulk_docs/semantics for more information. Conflicts and validation errors when updating documents in bulk must be handled separately; see api/db/bulk_docs/validation.

Bulk Documents Transaction Semantics

Bulk document operations are non-atomic. This means that CouchDB does not guarantee that any individual document included in the bulk update (or insert) will be saved when you send the request. The response will contain the list of documents successfully inserted or updated during the process. In the event of a crash, some of the documents may have been successfully saved, while others lost.

The response structure will indicate whether the document was updated by supplying the new _rev parameter indicating a new document revision was created. If the update failed, you will get an error of type conflict. For example:

[
    {
        "id" : "FishStew",
        "error" : "conflict",
        "reason" : "Document update conflict."
    },
    {
        "id" : "LambStew",
        "error" : "conflict",
        "reason" : "Document update conflict."
    },
    {
        "id" : "BeefStew",
        "error" : "conflict",
        "reason" : "Document update conflict."
    }
]

In this case no new revision has been created and you will need to submit the document update, with the correct revision tag, to update the document.

Replication of documents is independent of the type of insert or update. The documents and revisions created during a bulk insert or update are replicated in the same way as any other document.

Bulk Document Validation and Conflict Errors

The JSON returned by the _bulk_docs operation consists of an array of JSON structures, one for each document in the original submission. The returned JSON structure should be examined to ensure that all of the documents submitted in the original request were successfully added to the database.

When a document (or document revision) is not correctly committed to the database because of an error, you should check the error field to determine error type and course of action. Errors will be one of the following type:

  • conflict

    The document as submitted is in conflict. The new revision will not have been created and you will need to re-submit the document to the database.

    Conflict resolution of documents added using the bulk docs interface is identical to the resolution procedures used when resolving conflict errors during replication.

  • forbidden

    Entries with this error type indicate that the validation routine applied to the document during submission has returned an error.

    For example, if your validation routine <vdufun> includes the following:

    throw({forbidden: 'invalid recipe ingredient'});

    The error response returned will be:

    HTTP/1.1 201 Created
    Cache-Control: must-revalidate
    Content-Length: 80
    Content-Type: application/json
    Date: Sat, 26 Oct 2013 00:05:17 GMT
    Server: CouchDB (Erlang OTP)
    
    [
        {
            "id": "LambStew",
            "error": "forbidden",
            "reason": "invalid recipe ingredient"
        }
    ]