Attach files to Kinto records.
pip install kinto-attachment
In the Kinto project settings
kinto.includes = kinto_attachment
kinto.attachment.base_url = http://cdn.service.org/files/
kinto.attachment.folder = {bucket_id}/{collection_id}
kinto.attachment.keep_old_files = trueIf you want uploaded files to get gzipped when stored:
kinto.attachment.gzipped = trueStore files locally:
kinto.attachment.base_path = /tmpStore on Amazon S3:
kinto.attachment.aws.access_key = <AWS access key>
kinto.attachment.aws.secret_key = <AWS secret key>
kinto.attachment.aws.bucket_name = <bucket name>
kinto.attachment.aws.acl = <AWS ACL permissions|public-read>Note
access_key and secret_key may be omitted when using AWS Identity
and Access Management (IAM).
See Pyramid Storage.
In order to upload files on the default bucket, the built-in default bucket
plugin should be enabled before the kinto_attachment plugin.
In the configuration, this means adding it explicitly to includes:
kinto.includes = kinto.plugins.default_bucket
kinto_attachment- Make sure the
base_urlcan be reached (and points tobase_pathif files are stored locally) - Adjust the max size for uploaded files (e.g.
client_max_body_size 10m;for NGinx)
For example, with NGinx
server {
listen 80;
location /v1 {
...
}
location /files {
root /var/www/kinto;
}
}
POST /{record-url}/attachment
It will create the underlying record if it does not exist.
Required
attachment: a single multipart-encoded file
Optional
data: attributes to set on record (serialized JSON)permissions: permissions to set on record (serialized JSON)
DELETE /{record-url}/attachment
Deletes the attachement from the record.
By default, the server will randomize the name of the attached files. If you
don't want this behavior and prefer to keep the original file name, you can
pass ?randomize=false in the QueryString.
By default, the server won't gzip files unless you specifically used the
kinto.attachment.gzipped option if you want to force gzip to all
collections.
You can overwite that option by passing a ?gzipped=true in the QueryString
to specifically gzip some files.
When a file is attached, the related record is given an attachment attribute
with the following fields:
filename: the original filenamehash: a SHA-256 digestlocation: the URL of the attachmentmimetype: the media type of the filesize: size in bytes
{
"data": {
"attachment": {
"filename": "IMG_20150219_174559.jpg",
"hash": "hPME6i9avCf/LFaznYr+sHtwQEX7mXYHSu+vgtygpM8=",
"location": "http://cdn.service.org/files/ffa9c7b9-7561-406b-b7f9-e00ac94644ff.jpg",
"mimetype": "image/jpeg",
"size": 1481798
},
"id": "c2ce1975-0e52-4b2f-a5db-80166aeca688",
"last_modified": 1447834938251,
"theme": "orange",
"type": "wallpaper"
},
"permissions": {
"write": ["basicauth:6de355038fd943a2dc91405063b91018bb5dd97a08d1beb95713d23c2909748f"]
}
}If the file is gzipped by the server, an original key is added in the attachment
key, containing the file info before it's gzipped. The attachment keys are
in that case referring to the gzipped file:
{
"data": {
"attachment": {
"filename": "IMG_20150219_174559.jpg.gz",
"hash": "hPME6i9avCf/LFaznYr+sHtwQEX7mXYHSu+vgtygpM8=",
"location": "http://cdn.service.org/files/ffa9c7b9-7561-406b-b7f9-e00ac94644ff.jpg.gz",
"mimetype": "application/x-gzip",
"size": 14818,
"original": {
"filename": "IMG_20150219_174559.jpg",
"hash": "hPME6i9avCf/LFaznYr+sHtwQEX7mXYHSu+vgtygpM8=",
"mimetype": "image/jpeg",
"size": 1481798
}
},
"id": "c2ce1975-0e52-4b2f-a5db-80166aeca688",
"last_modified": 1447834938251,
"theme": "orange",
"type": "wallpaper"
},
"permissions": {
"write": ["basicauth:6de355038fd943a2dc91405063b91018bb5dd97a08d1beb95713d23c2909748f"]
}
}http --auth alice:passwd --form POST http://localhost:8888/v1/buckets/website/collections/assets/records/c2ce1975-0e52-4b2f-a5db-80166aeca689/attachment data='{"type": "wallpaper", "theme": "orange"}' "attachment@~/Pictures/background.jpg"HTTP/1.1 201 Created
Access-Control-Expose-Headers: Retry-After, Content-Length, Alert, Backoff
Content-Length: 209
Content-Type: application/json; charset=UTF-8
Date: Wed, 18 Nov 2015 08:22:18 GMT
Etag: "1447834938251"
Last-Modified: Wed, 18 Nov 2015 08:22:18 GMT
Location: http://localhost:8888/v1/buckets/website/collections/font/assets/c2ce1975-0e52-4b2f-a5db-80166aeca689
Server: waitress
{
"filename": "IMG_20150219_174559.jpg",
"hash": "hPME6i9avCf/LFaznYr+sHtwQEX7mXYHSu+vgtygpM8=",
"location": "http://cdn.service.org/files/ffa9c7b9-7561-406b-b7f9-e00ac94644ff.jpg",
"mimetype": "image/jpeg",
"size": 1481798
}auth = ("alice", "passwd")
attributes = {"type": "wallpaper", "theme": "orange"}
perms = {"read": ["system.Everyone"]}
files = [("attachment", ("background.jpg", open("Pictures/background.jpg", "rb"), "image/jpeg"))]
payload = {"data": json.dumps(attributes), "permissions": json.dumps(perms)}
response = requests.post(SERVER_URL + endpoint, data=payload, files=files, auth=auth)
response.raise_for_status()var headers = {Authorization: "Basic " + btoa("alice:passwd")};
var attributes = {"type": "wallpaper", "theme": "orange"};
var perms = {"read": ["system.Everyone"]};
// File object from input field
var file = form.elements.attachment.files[0];
// Build form data
var payload = new FormData();
// Multipart attachment
payload.append('attachment', file, "background.jpg");
// Record attributes and permissions JSON encoded
payload.append('data', JSON.stringify(attributes));
payload.append('permissions', JSON.stringify(perms));
// Post form using GlobalFetch API
var url = `${server}/buckets/${bucket}/collections/${collection}/records/${record}/attachment`;
fetch(url, {method: "POST", body: payload, headers: headers})
.then(function (result) {
console.log(result);
});Two scripts are provided in this repository.
They rely on the kinto-client Python package, which can be installed in a
virtualenv:
$ virtualenv env --python=python3 $ source env/bin/activate $ pip install kinto-client
Or globally on your system (not recommended):
$ sudo pip install kinto-client
upload.py takes a list of files and posts them on the specified server,
bucket and collection:
$ python3 scripts/upload.py --server=$SERVER --bucket=$BUCKET --collection=$COLLECTION --auth "token:mysecret" README.rst pictures/*
If the --gzip option is passed, the files are gzipped before upload.
Since the attachment attribute contains metadata of the compressed file
the original file metadata are stored in a original attribute.
See python3 scripts/upload.py --help for more details about options.
download.py downloads the attachments from the specified server, bucket and
collection and store them on disk:
$ python3 scripts/download.py --server=$SERVER --bucket=$BUCKET --collection=$COLLECTION --auth "token:mysecret"
If the record has an original attribute, the script decompresses the attachment
after downloading it.
Files are stored in the current folder by default.
See python3 scripts/download.py --help for more details about options.
- No support for chunk upload (#10)
- Files are not removed when server is purged with
POST /v1/__flush__
Currently the full URL is returned in records. This is very convenient for API consumers
which can access the attached file just using the value in the location attribute.
However, the way it is implemented has a limitation: the full URL is stored in each record
directly. This is annoying because changing the base_url setting
won't actually change the location attributes on existing records.
As workaround, it is possible to set the kinto.attachment.base_url to an empty
value. The location attribute in records will now contain a relative URL.
Using another setting kinto.attachment.extra.base_url, it is possible to advertise
the base URL that can be preprended by clients to obtain the full attachment URL.
If specified, it is going to be exposed in the capabilities of the root URL endpoint.
Run a fake Amazon S3 server in a separate terminal:
make moto
Run the tests suite:
make tests
- API design discussion about mixing up
attachmentand record fields.