first steps towards global index, updated cloud-examples

Author: haraldschilly

.gitignore

@@ -0,0 +1 @@
+index.json

README.md

@@ -4,18 +4,24 @@ This mainly adds a categorization and meta-description layer on top of this and
 
 ## Indexing
 
-Nothing done yet, probably a simple `make` will call a script.
+Call `./index.py` or just do a simple `make` to call that script.
 
 ## Datamodel
 
-* Goal: JSON for the SMC UI, such that user can select templates -- no need to think about the specific representation, only the elements that are the actually moving parts for such a selector.
+* Goal: autogenerated JSON for the SMC UI, such that user can select templates -- no need to think about the specific representation, only the elements that are the actually moving parts for such a selector.
 
-* Parts
-  * Filename/Directoryname
-  * Title
-  * Description
-  * Tags for categorization
-  * Centralized description of tags (?)
+* YAML Document Types
+
+  * first document: metadata
+      * description of tags
+      * description of license
+      * references: point to another index description file
+  * remaining documents are for each indexed entry:
+      * Target: Filename/Directory
+      * Title
+      * Description
+      * Tags for categorization
+      * license
 
 ## Devnotes
 
@@ -29,6 +35,9 @@ If you want to update all submodules in one go, do something like
 
 ## License
 
-Unless otherwise noted, Apache 2.0
+Unless otherwise noted,
+[Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0) for code and
+[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)
+for content.
 
 

cloud-examples

@@ -0,0 +1,48 @@
+#!/usr/bin/env python3
+# Copyright: Harald Schilly <hsy@sagemath.com>
+# License: Apache 2.0
+from pprint import pprint
+import yaml
+import json
+import os
+os.chdir(os.path.dirname(os.path.abspath(__file__)))
+
+def update_meta(meta, new_meta):
+    '''
+    A simple dict update would overwrite/remove entries.
+    '''
+    if 'tags' in new_meta:
+        meta['tags'].update(new_meta['tags'])
+    if 'licenses' in new_meta:
+        meta['licenses'].update(new_meta['licenses'])
+
+def resolve_references(meta, docs):
+    # append new documents and merge meta
+    if 'references' in meta:
+        for ref in meta['references']:
+            new_meta, *new_docs = yaml.load_all(open(ref))
+            resolve_references(new_meta, new_docs)
+            update_meta(meta, new_meta)
+            docs.extend(new_docs)
+        del meta['references']
+    return meta, docs
+
+def debug(meta, docs):
+    print("META:")
+    pprint(meta)
+    print("DOCS:")
+    for doc in docs:
+        pprint(doc)
+
+def export_json(meta, docs):
+    with open('index.json', 'w') as out:
+        json.dump({'metadata': meta, 'documents': docs}, out, indent=1)
+
+def main(index_fn):
+    meta, *docs = yaml.load_all(open(index_fn))
+    resolve_references(meta, docs)
+    debug(meta, docs)
+    export_json(meta, docs)
+
+if __name__ == '__main__':
+    main(index_fn = 'index.yaml')

index.py

@@ -0,0 +1,19 @@
+---
+tags:
+    calc: Calculus
+    linalg: Linear Algebra
+    stats: Statistics
+licenses:
+    ccby: CC BY 4.0
+    asis: AS IS
+references:
+- cloud-examples/index.yaml
+---
+target: bayesian-methods-for-hackers/
+title: Bayesian Methods for Hackers
+description: |
+    Hands-on tutorial how Bayesian Methods work, based on Python's PyMC library.
+author:  Cameron Davidson-Pilon
+license: asis
+tags:
+- stats

index.yaml

@@ -1,5 +1,9 @@
 PHONY = update
 
+default: index
+
 update:
 	git submodule foreach "git checkout master; git pull origin master"
 
+index:
+	./index.py