Skip to content

Commit

Permalink
added geosites as contrib module
Browse files Browse the repository at this point in the history
  • Loading branch information
simod committed Jul 15, 2015
1 parent 809af26 commit 740af85
Show file tree
Hide file tree
Showing 43 changed files with 1,634 additions and 0 deletions.
4 changes: 4 additions & 0 deletions .gitignore
Expand Up @@ -82,3 +82,7 @@ angular-leaflet-directive.min.js
bootstrap.min.js
jquery.min.js
bootstrap.min.css
uploaded
development.db
master
slave
87 changes: 87 additions & 0 deletions docs/reference/developers/geosites.txt
@@ -0,0 +1,87 @@
===============================
GeoSites: GeoNode Multi-Tenancy
===============================

GeoSites is a way to run multiple websites with a single instance of GeoNode. Each GeoSite can have different templates, apps, and data permissions but share a single database (useful for sharing users and data layers), GeoServer, and CSW. This is useful when multiple websites are desired to support different sets of users, but with a similar set of data and overall look and feel of the sites. Users can be given permission to access multiple sites if needed, which also allows administrative groups can be set up to support all sites with one account.


Master Website
============================

A GeoSites installation uses a 'master' GeoNode website that has soms additional administrative pages for doing data management. Layers, Maps, Documents, Users, and Groups can all be added and removed from different sites. Users can be given access to any number of sites, and data may appear on only a single site, or all of them. Additionally, if desired, any or all of the django apps installed on the other sites can be added to the master site to provide a single administrative interface that gives full access to all apps. The master site need not be accessible from the outside so that it can be used as an internal tool to the organization.

Users created on a particular site are created with access to just that site. Data uploaded to a particular site is given permission on that site as well as the master site. Any further adjustments to site-based permissions must be done from the master site.

Database
=====================
The master site, and all of the individual GeoSites, share a single database. Objects, including users, groups, and data layers, all appear within the database but an additional sites table indicates which objects have access to which sites. The geospatial data served by GeoServer (e.g., from PostGIS) can exist in the database like normal, since GeoServer will authenticate against GeoNode, which will use it's database to determine permissions based on the object, current user, and site.


GeoServer
=====================
A single GeoServer instance is used to serve data to all of the GeoSites. To keep data organized each site specifies a default workspace (DEFAULT_WORKSPACE) that GeoServer will use to partition the data depending on which site uploaded the data. The workspaces themselves don't have any impact on permissions, since data can be added and removed from different sites, however it provides at least some organization of the data based on the initial site.

This comment has been minimized.

Copy link
@frippe12573

frippe12573 Apr 12, 2017

Contributor

I would remove "To keep data organized each site specifies a default workspace (DEFAULT_WORKSPACE) that GeoServer will use to partition the data depending on which site uploaded the data." it seems to be not working automatically.


Data that is common to all sites can be added to the master site which will appear in the generic 'geonode' workspace.


Settings Files and Templates
=============================

A key component in managing multiple sites is keeping data organized and using a structured series of settings files so that common settings can be shared and only site specific settings are separated out. It is also best to import the default GeoNode settings from the GeoNode installation. This prevents the settings from having to be manually upgraded if there is any default change the GeoNode settings.

Settings which are common to all GeoSites, but differ from the default GeoNode, are separated into a master_settings.py file. Then, each individual site has settings file which imports from the master site and will then only need to specify a small selection that make that site unique, such as:

This comment has been minimized.

Copy link
@frippe12573

frippe12573 Apr 12, 2017

Contributor

master_setting.py doesn't exist anymore. This is not clear which is the master setting file so far, the reader could guess setting.py in /contrib/geosites/


* SITE_ID: Each one is unique, the master site should have a SITE_ID of 1.
* SITENAME
* SITEURL
* ROOT_URLCONF: This may be optional. The master site url.conf can be configured to automatically import the urls.py of all SITE_APPS, so a different ROOT_URLCONF is only needed if there are further differences.
* SITE_APPS: Containing the site specific apps
* App settings: Any further settings required for the above sites
* Other site specific settings, such as REGISTRATION_OPEN

A GeoSite therefore has three layers of imports, which is used for settings as well as the search path for templates. First it uses the individual site files, then the master GeoSite, then default GeoNode. These are specified via variables defined in settings:

* SITE_ROOT: The directory where the site specific settings and files are located (templates, static)
* PROJECT_ROOT: The top-level directory of all the GeoSites which should include the global settings file as well as template and static files
* GEONODE_ROOT: The GeoNode directory.

The TEMPLATE_DIRS, and STATICFILES_DIRS will then include all three directories as shown::

TEMPLATE_DIRS = (
os.path.join(SITE_ROOT, 'templates/'),
os.path.join(PROJECT_ROOT,'templates/'), # files common to all sites
os.path.join(GEONODE_ROOT, 'templates/')
)

STATICFILES_DIRS = (
os.path.join(SITE_ROOT, 'static/'),
os.path.join(PROJECT_ROOT, 'static/'),
os.path.join(GEONODE_ROOT, 'static/')
)

At the end of the settings_global.py the following variables will be set based on site specific settings::

This comment has been minimized.

Copy link
@frippe12573

frippe12573 Apr 12, 2017

Contributor

setting_global.py doesn't exist anymore, it is replaced by post_settings.py


STATIC_URL = os.path.join(SITEURL,’static/’)
GEONODE_CLIENT_LOCATION = os.path.join(STATIC_URL,’geonode/’)
GEOSERVER_BASE_URL = SITEURL + ‘geoserver/’
if SITE_APPS:
INSTALLED_APPS += SITE_APPS

Templates and Static Files
==========================

As mentioned above for each website there will be three directories used for template and static files. The first template file found will be the one used so templates in the SITE_ROOT/templates directory will override those in PROJECT_ROOT/templates, which will override those in GEONODE_ROOT/templates.

Static files work differently because (at least on a production server) they are collected and stored in a single location. Because of this care must be taken to avoid clobbering of files between sites, so each site directory should contain all static files in a subdirectory with the name of the site (e.g., static/siteA/logo.png )

This comment has been minimized.

Copy link
@frippe12573

frippe12573 Apr 12, 2017

Contributor

In the end, it could be better clarified if for static directory we mean static_root as described for the single geonode site. That is ambiguous.


The location of the proper static directory can then be found in the templates syntax such as::

{{ STATIC_URL }}{{ SITENAME|lower }}/logo.png

Permissions by Site
================
By default GeoNode is publicly available. In the case of GeoSites, new data will be publicly available, but only for the site it was added to, and the master site (all data is added to the master site).

This comment has been minimized.

Copy link
@frippe12573

frippe12573 Apr 12, 2017

Contributor

I would add:

Activate geosites app

In order to use geonode in Multi-tenancy mode, follow these steps:

  1. check in settings.py if 'geonode.contrib.geosites' in GEONODE_CONTRIB_APPS is rightly uncommented
  2. add in settings.py 'geonode.contrib.geosites' in INSTALLED_APPS
  3. run python manage.py syncdb
Adding New Sites
===============
A management command exists to easily create a new site. This will create all the needed directories, as well as a site specific settings file. The command may also create a website configuration file.

This comment has been minimized.

Copy link
@frippe12573

frippe12573 Apr 12, 2017

Contributor

"A management command exists to easily create a new site. This will create all the needed directories, as well as a site specific settings file. The command may also create a website configuration file." It could replaced by:
To add a new site follow the following steps:

  1. copy the directory site_template in /geonode/geonode/contrib/geosites and give it a name
  2. from administration pannel add 'new site'
  3. create a virtualhost in the webserver related to the new created site. Remember to setup the WSGIDeamonProcess with the name you gave to the folder created at point 1. and the path to the geosites directory. WSGIProcessGroup have to be pointed to the name you choose for the folder you created at point 1.
    Eventually, WSGIScriptAlias have to be set to the wsgi.py you have in your site folder.
  4. check the configuration files: local_settings.py, pre_settings.py, post_settings.py in /geonode/geonode/contrib/geosites as well as local_settings.py and settings.py in your site folder
  5. in /geonode/geonode/contrib/geosites/local_settings.py set the variable SERVE_PATH. it have to point geosites folder.
  6. in the local_setting of the site folder insert the values to the following variables:
  • SITE_ID
  • SITE_NAME
  • SITE_URL
  1. run python manage.py collectstatics - Pay attention on the folder where you are running the command and the folder structure of your geosites/geonode project, in case pass the path of the settings file by using --setting to the python command
  2. you can customize the look and feel of your site working on the css and html file you find in your template site directory. After a change, run again collectstatics command.
3 changes: 3 additions & 0 deletions docs/reference/developers/index.txt
Expand Up @@ -10,12 +10,15 @@ Here you will find information about each and every component of Geonode, for ex

:ref:`settings`

:ref:`geosites`

.. toctree::
:hidden:
:maxdepth: 3

django-apps
javascript
settings
geosites


87 changes: 87 additions & 0 deletions geonode/contrib/geosites/README.md
@@ -0,0 +1,87 @@
===============================
GeoSites: GeoNode Multi-Tenancy
===============================

GeoSites is a way to run multiple websites with a single instance of GeoNode. Each GeoSite can have different templates, apps, and data permissions but share a single database (useful for sharing users and data layers), GeoServer, and CSW. This is useful when multiple websites are desired to support different sets of users, but with a similar set of data and overall look and feel of the sites. Users can be given permission to access multiple sites if needed, which also allows administrative groups can be set up to support all sites with one account.


Master Website
============================

A GeoSites installation uses a 'master' GeoNode website that has soms additional administrative pages for doing data management. Layers, Maps, Documents, Users, and Groups can all be added and removed from different sites. Users can be given access to any number of sites, and data may appear on only a single site, or all of them. Additionally, if desired, any or all of the django apps installed on the other sites can be added to the master site to provide a single administrative interface that gives full access to all apps. The master site need not be accessible from the outside so that it can be used as an internal tool to the organization.

Users created on a particular site are created with access to just that site. Data uploaded to a particular site is given permission on that site as well as the master site. Any further adjustments to site-based permissions must be done from the master site.

Database
=====================
The master site, and all of the individual GeoSites, share a single database. Objects, including users, groups, and data layers, all appear within the database but an additional sites table indicates which objects have access to which sites. The geospatial data served by GeoServer (e.g., from PostGIS) can exist in the database like normal, since GeoServer will authenticate against GeoNode, which will use it's database to determine permissions based on the object, current user, and site.


GeoServer
=====================
A single GeoServer instance is used to serve data to all of the GeoSites. To keep data organized each site specifies a default workspace (DEFAULT_WORKSPACE) that GeoServer will use to partition the data depending on which site uploaded the data. The workspaces themselves don't have any impact on permissions, since data can be added and removed from different sites, however it provides at least some organization of the data based on the initial site.

Data that is common to all sites can be added to the master site which will appear in the generic 'geonode' workspace.


Settings Files and Templates
=============================

A key component in managing multiple sites is keeping data organized and using a structured series of settings files so that common settings can be shared and only site specific settings are separated out. It is also best to import the default GeoNode settings from the GeoNode installation. This prevents the settings from having to be manually upgraded if there is any default change the GeoNode settings.

Settings which are common to all GeoSites, but differ from the default GeoNode, are separated into a master_settings.py file. Then, each individual site has settings file which imports from the master site and will then only need to specify a small selection that make that site unique, such as:

* SITE_ID: Each one is unique, the master site should have a SITE_ID of 1.
* SITENAME
* SITEURL
* ROOT_URLCONF: This may be optional. The master site url.conf can be configured to automatically import the urls.py of all SITE_APPS, so a different ROOT_URLCONF is only needed if there are further differences.
* SITE_APPS: Containing the site specific apps
* App settings: Any further settings required for the above sites
* Other site specific settings, such as REGISTRATION_OPEN

A GeoSite therefore has three layers of imports, which is used for settings as well as the search path for templates. First it uses the individual site files, then the master GeoSite, then default GeoNode. These are specified via variables defined in settings:

* SITE_ROOT: The directory where the site specific settings and files are located (templates, static)
* PROJECT_ROOT: The top-level directory of all the GeoSites which should include the global settings file as well as template and static files
* GEONODE_ROOT: The GeoNode directory.

The TEMPLATE_DIRS, and STATICFILES_DIRS will then include all three directories as shown::

TEMPLATE_DIRS = (
os.path.join(SITE_ROOT, 'templates/'),
os.path.join(PROJECT_ROOT,'templates/'), # files common to all sites
os.path.join(GEONODE_ROOT, 'templates/')
)

STATICFILES_DIRS = (
os.path.join(SITE_ROOT, 'static/'),
os.path.join(PROJECT_ROOT, 'static/'),
os.path.join(GEONODE_ROOT, 'static/')
)

At the end of the settings_global.py the following variables will be set based on site specific settings::

STATIC_URL = os.path.join(SITEURL,’static/’)
GEONODE_CLIENT_LOCATION = os.path.join(STATIC_URL,’geonode/’)
GEOSERVER_BASE_URL = SITEURL + ‘geoserver/’
if SITE_APPS:
INSTALLED_APPS += SITE_APPS

Templates and Static Files
==========================

As mentioned above for each website there will be three directories used for template and static files. The first template file found will be the one used so templates in the SITE_ROOT/templates directory will override those in PROJECT_ROOT/templates, which will override those in GEONODE_ROOT/templates.

Static files work differently because (at least on a production server) they are collected and stored in a single location. Because of this care must be taken to avoid clobbering of files between sites, so each site directory should contain all static files in a subdirectory with the name of the site (e.g., static/siteA/logo.png )

The location of the proper static directory can then be found in the templates syntax such as::

{{ STATIC_URL }}{{ SITENAME|lower }}/logo.png

Permissions by Site
================
By default GeoNode is publicly available. In the case of GeoSites, new data will be publicly available, but only for the site it was added to, and the master site (all data is added to the master site).

Adding New Sites
===============
A management command exists to easily create a new site. This will create all the needed directories, as well as a site specific settings file. The command may also create a website configuration file.
Empty file.
17 changes: 17 additions & 0 deletions geonode/contrib/geosites/admin.py
@@ -0,0 +1,17 @@
from django.contrib import admin

from .models import SiteResources, SitePeople


class SiteResourceAdmin(admin.ModelAdmin):
filter_horizontal = ('resources',)
readonly_fields = ('site',)


class SitePeopleAdmin(admin.ModelAdmin):
filter_horizontal = ('people',)
readonly_fields = ('site',)


admin.site.register(SiteResources, SiteResourceAdmin)
admin.site.register(SitePeople, SitePeopleAdmin)
135 changes: 135 additions & 0 deletions geonode/contrib/geosites/api.py
@@ -0,0 +1,135 @@
from django.contrib.sites.models import Site
from django.conf import settings
from django.db.models import Count
from django.contrib.auth import get_user_model

from tastypie.constants import ALL
from tastypie.resources import ModelResource
from tastypie.authorization import DjangoAuthorization
from guardian.shortcuts import get_objects_for_user

from geonode.api.resourcebase_api import CommonModelApi, LayerResource, MapResource, DocumentResource, \
ResourceBaseResource
from geonode.base.models import ResourceBase
from geonode.api.urls import api
from geonode.api.api import TagResource, TopicCategoryResource, RegionResource, CountJSONSerializer, \
ProfileResource

from .utils import resources_for_site, users_for_site


class CommonSiteModelApi(CommonModelApi):
"""Override the apply_filters method to respect the site"""

def apply_filters(self, request, applicable_filters):
filtered = super(CommonSiteModelApi, self).apply_filters(request, applicable_filters)

filtered = filtered.filter(id__in=resources_for_site())

return filtered


class SiteResourceBaseResource(CommonSiteModelApi):
"""Site aware ResourceBase api"""

class Meta(ResourceBaseResource.Meta):
pass


class SiteLayerResource(CommonSiteModelApi):
"""Site aware Layer API"""

class Meta(LayerResource.Meta):
pass


class SiteMapResource(CommonSiteModelApi):

"""Site aware Maps API"""

class Meta(MapResource.Meta):
pass


class SiteDocumentResource(CommonSiteModelApi):
"""Site aware Documents API"""

class Meta(DocumentResource.Meta):
pass


class SiteResource(ModelResource):
"""Sites API"""

class Meta:
queryset = Site.objects.all()
filtering = {
'name': ALL
}
resource_name = 'sites'
allowed_methods = ['get', 'delete', 'post']
authorization = DjangoAuthorization()


class SiteCountJSONSerializer(CountJSONSerializer):
"""Custom serializer to post process the api and add counts for site"""

def get_resources_counts(self, options):
if settings.SKIP_PERMS_FILTER:
resources = ResourceBase.objects.all()
else:
resources = get_objects_for_user(
options['user'],
'base.view_resourcebase'
)
if settings.RESOURCE_PUBLISHING:
resources = resources.filter(is_published=True)

if options['title_filter']:
resources = resources.filter(title__icontains=options['title_filter'])

if options['type_filter']:
resources = resources.instance_of(options['type_filter'])

resources = resources.filter(id__in=resources_for_site())
counts = list(resources.values(options['count_type']).annotate(count=Count(options['count_type'])))

return dict([(c[options['count_type']], c['count']) for c in counts])


class SiteTagResource(TagResource):
"""Site aware Tag API"""

class Meta(TagResource.Meta):
serializer = SiteCountJSONSerializer()


class SiteTopicCategoryResource(TopicCategoryResource):
"""Site aware Category API"""

class Meta(TopicCategoryResource.Meta):
serializer = SiteCountJSONSerializer()


class SiteRegionResource(RegionResource):
"""Site aware Region API"""

class Meta(RegionResource.Meta):
serializer = SiteCountJSONSerializer()


class SiteProfileResource(ProfileResource):
"""Site aware Profile API"""

class Meta(ProfileResource.Meta):
queryset = get_user_model().objects.exclude(username='AnonymousUser').filter(id__in=users_for_site())

api.register(SiteLayerResource())
api.register(SiteMapResource())
api.register(SiteDocumentResource())
api.register(SiteResourceBaseResource())
api.register(SiteResource())
api.register(SiteTagResource())
api.register(SiteTopicCategoryResource())
api.register(SiteRegionResource())
api.register(SiteProfileResource())
Empty file.
10 changes: 10 additions & 0 deletions geonode/contrib/geosites/fixtures/initial_data.json
@@ -0,0 +1,10 @@
[
{
"model": "sites.site",
"pk": 1,
"fields": {
"domain": "master.geonode.org",
"name": "Master"
}
}
]

0 comments on commit 740af85

Please sign in to comment.