Nf dataset web interface #751
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
datalad ls -w results in filesystem traversal at mentioned location. stores data in list with a flat sublist of files, directories per git/annex submodule. can handle recursive case too. Next: format to json for web UI consumption
reduce code in fsmodel class by inheriting from dsmodel fix bugs in list flattening date not working, parent submodule incorrect after first, symlink unicode coercion error in non datalad modules
modify DsModel for usability in FsModel after inheritance work on issue datalad#520
prints json as it computes each directory node information. not storing json in variable/memory as expect a lot of files in annex. this json is to be consumed by the web UI for the datasets. refer issue datalad#520.
Add DOC-strings to _ls_web class, follow convention Based on comments on pull#541
use dicts to allow json.dumps to handle conversion to json refactor to traverse complete directory tree only once improve node names using leaf_node function remove redundant parent repo extraction code
Each subdirectory has a json containing the metadata on files etc available in that subdirectory only, no deeper standardizes json to simplify rendering dirs for website
adds search for file at each directory level, adds css styling based on file/folder type(i.e annex, git etc) uses datatables library for rendering json as table and dir traversal
no need to unwrap each submodule before passing to fs_traverse ignored index.html, ignore root's parent for web publish
handle GET parameters in URL to specify directory to load useful for adding permalink feature remove local assets, caching should take care of asset load speed
update browser url to reflect current directory being visualized use the dir argument passed in url to find and load dir metadata json allows accessing previous location in dataset with browser back button
passing file, display, delete option explicitly prevents users from inadvertently populating their dataset with json metadata files minor bug fix in column rendering of html
automatically update dataset metadata post-update scp web interface html from local datalad repository while initialising publication target
hardcoded path to datalad virtualenv on datalad.org server, so just works for our use-case currently, fix will require post-update hook to know location of datalad on server automagically/etc
datatable sorted by node.type, datatable doesn't support custom order need sorter to create custom ordering with directories at the top
Removes '../' parent row from top dataset in html instead of datalad ls.
…rely in the future We might not bother with those relative paths until we are actually printing those models and they might all get "relpath" property if they are provided "toppath"
…rface * origin/master: (82 commits) BF: Handle crazy cfg BF: Deal with symlinks TYPO: contraint -> constraint ENH: use @_where_reload decorator ENH+RF: remove_section, centralized duplicate docstring for where and reload ENH+TODO: test global gitconfig modifications with mocked HOME RF: swap logic - first dataset, and then git configs RF: move all invocations of git config into one method (+ few typos fixed) RF: Do not resolve symlinks to realpath TST: For the dataset integration of the config DOC: Small fixes DOC: for ConfigManager BF: Make test deterministic NF: Draft of our own config parser TST&RF: added more testing, placed exclude loop back but kept changes BF: allow unrelated merges while merging incoming-processed in balsa ENH: Pass call args to results renderers (closes: dataladgh-650) RF: compare all files with find_files BF+TST: Ignore excluded dirs before strip leading_dir DOC: typo ...
4 similar comments
Current coverage is 85.15% (diff: 76.42%)@@ master #751 diff @@
==========================================
Files 198 198
Lines 17890 18203 +313
Methods 0 0
Messages 0 0
Branches 0 0
==========================================
- Hits 15522 15500 -22
- Misses 2368 2703 +335
Partials 0 0
|
5 similar comments
|
the failure didn't disappear magically... |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Duplicate of #541 but with master merged -- may be it would resolve the failing test on travis