Skip to content

Commit

Permalink
update filepad to accept conventional username/password and also upda…
Browse files Browse the repository at this point in the history
…te docs to represent label->identifier shift
  • Loading branch information
computron committed Nov 26, 2018
1 parent 8ae4e33 commit 0a3b33e
Show file tree
Hide file tree
Showing 3 changed files with 30 additions and 23 deletions.
6 changes: 3 additions & 3 deletions docs_rst/filepad_tasks.rst
Expand Up @@ -13,11 +13,11 @@ Required parameters
-------------------

* paths ([str]): list of paths to files to be added
* labels ([str]): list of labels, one for each file in the paths list.

Optional parameters
-------------------

* identifiers ([str]): list of identifiers, one for each file in the paths list.
* filepad_file (str): path to the filepad db config file
* compress (bool): whether or not to compress the file before inserting to gridfs
* metadata (dict): metadata to store along with the file, stored in 'metadata' key
Expand All @@ -31,7 +31,7 @@ them to the given destination directory.
Required parameters
-------------------

* labels ([str]): list of labels, one for each file in the paths list.
* identifiers ([str]): list of identifiers, one for each file in the paths list.

Optional parameters
-------------------
Expand All @@ -48,7 +48,7 @@ The *DeleteFilesTask* lets you delete one or more files from the filepad.
Required parameters
-------------------

* labels: ([str]) file labels to delete
* identifiers: ([str]) file identifiers to delete

Optional parameters
-------------------
Expand Down
39 changes: 23 additions & 16 deletions docs_rst/filepad_tutorial.rst
Expand Up @@ -5,40 +5,47 @@ Using FilePad for storing and retrieving files

FilePad utility provides the api to add and delete arbitrary files of arbitray sizes to MongoDB(filepad).
The is achieved by inserting the entire file contents to GridFS and storing the id returned by the
GridFS insertion, the user provided label and the metadata in a document in the filepad. In the following
GridFS insertion, the user provided identifier and the metadata in a document in the filepad. In the following
documentation, ``file contents`` refers to the file contents stored in GridFS and ``document`` refers to the
associated mongodb document that stores the ``file_id``, ``label`` and other miscellaneous information
associated mongodb document that stores the ``file_id``, ``identifier`` and other miscellaneous information
pertaining to the file.

Adding files
==============

Create a FilePad object::

fp = FilePad.auto_load()
fp = FilePad.from_db_file(<PATH_TO_DB_FILE>)

or:

fp = FilePad.auto_load() # if you have your configuration file paths set up for automatic loading

The db file needs to contain ``username`` and ``password`` keys. You should be able to reuse your existing ``my_launchpad.yaml`` file for this purpose.

To add a file::

file_id, label = fp.add(<path>, <label>, compress=True/False, metadata=<metadata>)
file_id, identifier = fp.add_file(<path>, <identifier>, compress=True/False, metadata=<metadata>)

where ``<path>`` is a string path to the file to be inserted, ``<label>`` is some
unique label that can be used to retrieve the file, the 'compress' argument value tells whether or not to compress
where ``<path>`` is a string path to the file to be inserted, ``<identifier>`` is some
unique identifier that can be used to retrieve the file, the 'compress' argument value tells whether or not to compress
the file contents before insertion, ``<metadata>`` is a python dictionary input that will stored in the key 'metadata'.
A bare minimum document in the filepad database consists of keys ``file_id``(used
to store the string representation of the object id returned by GridFS), ``label``(used to store the
user assigned label for the file), ``original_file_name`` , ``original_file_path`` and ``metadata``.
On successful insertion the ``file_id`` and the ``label`` are returned.
to store the string representation of the object id returned by GridFS), ``identifier``(used to store the
user assigned identifier for the file), ``original_file_name`` , ``original_file_path`` and ``metadata``.
On successful insertion the ``file_id`` and the ``identifier`` are returned.

Retrieving files
=================

Retrieve file contents and the associated document by the identifier::

Retrieve file contents and the associated document by the label::

file_contents, doc = fp.get_file(<label>)
file_contents, doc = fp.get_file(<identifier>)

where the returned values ``file_contents`` and ``doc`` are the contents of the file with label ``<label>``
and the document attached to it respectively.
where the returned values ``file_contents`` and ``doc`` are the contents of the file with identifier ``<identifier>``
and the document attached to it respectively. Note that if you had set the ``compress`` option to True when
inserting the file, it will automatically be decompressed at this stage so that you will retrieve the
original file contents as expected.

Retrieve file contents and the associated document by the file id::

Expand All @@ -57,9 +64,9 @@ tuples that match the query.
Deleting files
=================

To delete the contents of the file and the associated document by label::
To delete the contents of the file and the associated document by identifier::

fp.delete_file(<label>)
fp.delete_file(<identifier>)

To delete the file contents and the associated document by the file id::

Expand Down
8 changes: 4 additions & 4 deletions fireworks/utilities/filepad.py
Expand Up @@ -288,11 +288,11 @@ def from_db_file(cls, db_file, admin=True):
creds = loadfn(db_file)

if admin:
user = creds.get("admin_user")
password = creds.get("admin_password")
user = creds.get("admin_user", creds.get("username"))
password = creds.get("admin_password", creds.get("password"))
else:
user = creds.get("readonly_user")
password = creds.get("readonly_password")
user = creds.get("readonly_user", creds.get("username"))
password = creds.get("readonly_password", creds.get("password"))

coll_name = creds.get("filepad", "filepad")
gfs_name = creds.get("filepad_gridfs", "filepad_gfs")
Expand Down

0 comments on commit 0a3b33e

Please sign in to comment.