Added delete_tileset manage command and tileset API route

README.md

@@ -1,8 +1,8 @@
 # HiGlass Server
 
-The HiGlass Server supports [HiGlass](https://github.com/hms-dbmi/higlass) and [HiPiler](https://github.com/flekschas/hipiler)
+The HiGlass Server supports [HiGlass](https://github.com/higlass/higlass) and [HiPiler](https://github.com/flekschas/hipiler)
 by providing APIs for accessing and uploading tiles generated by
-[Clodius](https://github.com/hms-dbmi/clodius).
+[Clodius](https://github.com/higlass/clodius).
 
 [![demo](https://img.shields.io/badge/higlass-👍-red.svg?colorB=0f5d92)](http://higlass.io)
 [![api](https://img.shields.io/badge/api-documentation-red.svg?colorB=0f5d92)](API.md)
@@ -19,14 +19,14 @@ _Note: that the HiGlass Server itself only provides an API, and does not serve a
 
 ### Docker
 
-The easiest way to run HiGlass with HiGlass Server is with Docker. More information is available at [higlass-docker](https://github.com/hms-dbmi/higlass-docker#readme) or check out the [Dockerfile](docker-context/Dockerfile).
+The easiest way to run HiGlass with HiGlass Server is with Docker. More information is available at [higlass-docker](https://github.com/higlass/higlass-docker#readme) or check out the [Dockerfile](docker-context/Dockerfile).
 
 ### Manually
 
 To install HiGlass Server manually follow the steps below. Note we strongly recommend to create a virtual environment using [Virtualenvwrapper](https://pypi.python.org/pypi/virtualenvwrapper) for example. Skip step 2 if you don't work with virtual environments.
 
 ```bash
-git clone https://github.com/hms-dbmi/higlass-server && cd higlass-server
+git clone https://github.com/higlass/higlass-server && cd higlass-server
 mkvirtualenv -a $(pwd) -p $(which python3) higlass-server && workon higlass-server
 pip install --upgrade -r ./requirements.txt
 pip install --upgrade -r ./requirements-secondary.txt
@@ -55,6 +55,34 @@ curl http://localhost:8000/api/v1/tileset_info/?d=hitile-demo
 curl http://localhost:8000/api/v1/tiles/?d=hitile-demo.0.0.0
 ```
 
+## Preparing files for ingest
+
+Genomics data needs to be aggregated across a range of scales before it can be used with HiGlass.
+
+### Cooler files
+
+**TODO**: Is this still accurate?
+
+[Cooler](https://github.com/mirnylab/cooler) files store Hi-C data. They need to be decorated with aggregated
+data at multiple resolutions in order to work with `higlass-server`. This is easily accomplished by simply
+installing the `cooler` python package and running the `recursive_agg_onefile.py` script. For now this has
+to come from a clone of the official cooler repository, but this will hopefully be merged into the main branch shortly.
+
+```
+
+git clone -b develop https://github.com/pkerpedjiev/cooler.git
+cd cooler
+python setup.py install
+
+recursive_agg_onefile.py file.cooler --out output.cooler
+```
+
+### BigWig files
+
+BigWig files can be aggregated with the [Clodius](https://github.com/higlass/clodius#bigwig-files).
+
+### **TODO**: Any other types?
+
 ---
 
 ## Development

docker-context/Dockerfile

@@ -4,7 +4,7 @@
 
 # Typical usage:
 #
-# git clone https://github.com/hms-dbmi/higlass-server.git
+# git clone https://github.com/higlass/higlass-server.git
 # docker build --cache-from higlass-server-image --tag higlass-server-image docker-context
 # docker run -dit --name higlass-server-container --volume `pwd`:/home/higlass/higlass-server higlass-server-image
 # docker exec higlass-server-container python manage.py migrate
@@ -25,7 +25,7 @@ ENV PATH="/home/higlass/miniconda/bin:${PATH}"
 RUN conda install --yes python=2.7.12 cython==0.25.2 numpy=1.11.2
 
 # This is not run with bash: Otherwise, the <( ) syntax would be useful here.
-ENV GITHUB=https://raw.githubusercontent.com/hms-dbmi/higlass-server/master
+ENV GITHUB=https://raw.githubusercontent.com/higlass/higlass-server/master
 RUN wget $GITHUB/requirements.txt           && pip install -r requirements.txt
 RUN wget $GITHUB/requirements-secondary.txt && pip install -r requirements-secondary.txt
 

package.json

@@ -3,7 +3,7 @@
   "description": "Server for HiGlass serving cooler files",
   "repository": {
     "type": "git",
-    "url": "https://github.com/hms-dbmi/higlass-server.git"
+    "url": "https://github.com/higlass/higlass-server.git"
   },
   "license": "MIT",
   "scripts": {

requirements-secondary.txt

@@ -1 +1 @@
-clodius==0.7.2
+clodius==0.9.3

requirements.txt

@@ -1,8 +1,8 @@
-Cython==0.25.2
+-e git+https://github.com/mirnylab/cooler.git@master#egg=cooler
 pybbi==0.2.0
-bumpversion==0.5.3
+pyBigWig==0.3.2
+Cython==0.27.2
 CacheControl==0.12.4
-cooler==0.7.10
 django-cors-headers
 django-guardian
 django-rest-swagger

tilesets/management/commands/delete_tileset.py

@@ -0,0 +1,29 @@
+from django.core.management.base import BaseCommand, CommandError
+from django.conf import settings
+import tilesets.models as tm
+import os
+
+class Command(BaseCommand):
+    def add_arguments(self, parser):
+        parser.add_argument('--uuid', type=str, required=True)
+
+    def handle(self, *args, **options):
+        uuid = options.get('uuid')
+
+        # search for Django object, remove associated file and record
+        instance = tm.Tileset.objects.get(uuid=uuid)
+        if not instance:
+          raise CommandError('Instance for specified uuid [%s] was not found' % (uuid))
+        else:
+          filename = instance.datafile.name
+          filepath = os.path.join(settings.MEDIA_ROOT, filename)
+          if not os.path.isfile(filepath):
+              raise CommandError('File does not exist under media root')
+          try:
+              os.remove(filepath)
+          except OSError:
+              raise CommandError('File under media root could not be removed')
+          try:
+              instance.delete()
+          except ProtectedError:
+              raise CommandError('Instance for specified uuid [%s] could not be deleted' % (uuid))

tilesets/management/commands/modify_tileset.py

@@ -0,0 +1,33 @@
+from django.core.management.base import BaseCommand, CommandError
+from django.conf import settings
+import tilesets.models as tm
+import os
+
+class Command(BaseCommand):
+    def add_arguments(self, parser):
+        parser.add_argument('--uuid', type=str, required=True)
+        parser.add_argument('--name', type=str)
+
+    def handle(self, *args, **options):
+        uuid = options.get('uuid')
+        name = options.get('name')
+
+        # search for Django object, modify associated record
+        instance = tm.Tileset.objects.get(uuid=uuid)
+        if not instance:
+          raise CommandError('Instance for specified uuid [%s] was not found' % (uuid))
+        else:
+          try:
+              instance_modified = False
+
+              # only change tileset name if specified, and if it is 
+              # different from the current instance name
+              if name and name != instance.name:
+                  instance.name = name
+                  instance_modified = True
+
+              # if any changes were applied, persist them
+              if instance_modified:
+                  instance.save()
+          except ProtectedError:
+              raise CommandError('Instance for specified uuid [%s] could not be renamed' % (uuid))

tilesets/models.py

@@ -49,4 +49,4 @@ def __str__(self):
         Get a string representation of this model. Hopefully useful for the
         admin interface.
         '''
-        return "Tileset [name: " + self.name + '] [ft: ' + self.filetype + ']'
+        return "Tileset [name: " + self.name + '] [ft: ' + self.filetype + '] [uuid: ' + self.uuid + ']'

tilesets/urls.py

@@ -18,6 +18,7 @@
     url(r'^viewconf', views.viewconfs),
     url(r'^uids_by_filename', views.uids_by_filename),
     url(r'^tiles/$', views.tiles),
+    url(r'^tileset/$', views.tileset),
     url(r'^tileset_info/$', views.tileset_info),
     url(r'^suggest/$', views.suggest),
     url(r'^', include(router.urls)),

tilesets/views.py

@@ -37,6 +37,7 @@
 import tilesets.serializers as tss
 import tilesets.suggestions as tsu
 
+import os
 import os.path as op
 
 import rest_framework.exceptions as rfe
@@ -58,8 +59,9 @@
 from django.views.decorators.gzip import gzip_page
 from rest_framework import generics
 from rest_framework import viewsets
-from rest_framework.decorators import api_view, authentication_classes
+from rest_framework.decorators import api_view, authentication_classes, permission_classes
 from rest_framework.authentication import BasicAuthentication
+from rest_framework.permissions import IsAuthenticated
 from fragments.drf_disable_csrf import CsrfExemptSessionAuthentication
 
 from higlass_server.utils import getRdb
@@ -492,6 +494,95 @@ def tiles(request):
         )
 
     return JsonResponse(tiles_to_return, safe=False)
+
+
+@api_view(['POST', 'DELETE'])
+@authentication_classes((CsrfExemptSessionAuthentication, BasicAuthentication))
+@permission_classes((IsAuthenticated,))
+def tileset(request):
+    '''
+    Delete or modify tileset with given uuid.
+
+    Args:
+        POST
+        request (django.http.HTTPRequest): The request object
+            containing the uuid (uuid=abcd1234) that identifies the
+            tileset, with additional (optional) parameters that update 
+            the identified object (name, at this time).
+
+        DELETE
+        request (django.http.HTTPRequest): The request object
+            containing the uuid (d=abcd1234) that identifies the
+            tileset.
+
+    Return:
+        django.http.JsonResponse: A JSON object containing
+            the tileset uuid or an error message detailing 
+            what step of the deletion or modification failed.
+    '''
+
+    if request.method == 'POST':
+        tileset_wrapper = json.loads(request.body.decode('utf-8'))
+        tileset_uuid = tileset_wrapper.get('uuid')
+        tileset_name = tileset_wrapper.get('name')
+        if not tileset_uuid:
+            return JsonResponse({
+                'error': 'Tileset uuid not specified'
+            }, status=400)
+        instance = tm.Tileset.objects.get(uuid=tileset_uuid)
+        if not instance:
+            return JsonResponse({
+                'error': 'Tileset instance for uuid {} does not exist'.format(tileset_uuid)
+            }, status=400)
+        try:
+            instance_modified = False
+
+            # only change tileset name if specified, and if it is 
+            # different from the current instance name
+            if tileset_name and tileset_name != instance.name:
+                instance.name = tileset_name
+                instance_modified = True
+
+            # if any changes were applied, persist them
+            if instance_modified:
+                instance.save()
+        except ProtectedError:
+            return JsonResponse({
+                'error': 'Tileset instance for uuid {} could not be modified'.format(tileset_uuid)
+            }, status=400)
+        return JsonResponse({'uuid': tileset_uuid})
+
+    elif request.method == 'DELETE':
+        if 'd' not in request.GET:
+            return JsonResponse({
+                'error': 'Tileset uuid not specified'
+            }, status=400)
+        tileset_uuid = request.GET['d']
+        instance = tm.Tileset.objects.get(uuid=tileset_uuid)
+        if not instance:
+            return JsonResponse({
+                'error': 'Tileset instance for uuid {} does not exist'.format(tileset_uuid)
+            }, status=400)
+        filename = instance.datafile.name
+        filepath = op.join(hss.MEDIA_ROOT, filename)
+        if not op.isfile(filepath):
+            return JsonResponse({
+                'error': 'Tileset instance for uuid {} does not have file at path {}'.format(tileset_uuid, filepath)
+            }, status=404)
+        try:
+            os.remove(filepath)
+        except OSError:
+            return JsonResponse({
+                'error': 'Tileset instance for uuid {} could not have file removed at path {}'.format(tileset_uuid, filepath)
+            }, status=400)
+        try:
+              instance.delete()
+        except ProtectedError:
+            return JsonResponse({
+                'error': 'Tileset instance for uuid {} could not be deleted'.format(tileset_uuid)
+            }, status=400)
+        return JsonResponse({'uuid': tileset_uuid})
+
 
 
 @api_view(['GET'])