| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,315 @@ | ||
| Monitoring: User Analytics | ||
| ========================== | ||
|
|
||
| Purpose | ||
| ------- | ||
|
|
||
| UA should provide information about GeoNode resources usage at user level (not request level, like plain monitoring). | ||
|
|
||
| Requests | ||
| -------- | ||
|
|
||
| 1. total number of unique visitors on GeoNode (excluding ows requests) per day. This gives a base view of the reach. | ||
|
|
||
| * requests from all sessions of all types, ows and non-ows | ||
|
|
||
| ``GET /monitoring/api/metric_data/request.users/?last=(x*86400)&interval=86400&group_by=label`` | ||
|
|
||
| * non-ows related | ||
|
|
||
| ``GET /monitoring/api/metric_data/request.users/?`` | ||
| ``last=(x*86400)&interval=86400&group_by=label&event_type=other`` | ||
|
|
||
| * only ows related | ||
|
|
||
| ``GET /monitoring/api/metric_data/request.users/?`` | ||
| ``last=(x*86400)&interval=86400&group_by=label&event_type=OWS:ALL`` | ||
|
|
||
| .. code-block:: json | ||
| { | ||
| "data": { | ||
| "input_valid_from": "2018-07-11T15:41:06.419Z", | ||
| "input_valid_to": "2018-07-12T15:41:06.419Z", | ||
| "data": [ | ||
| { | ||
| "valid_from": "2018-07-11T15:41:06.419Z", | ||
| "data": [ | ||
| { | ||
| "samples_count": 82, | ||
| "val": 9, | ||
| "min": "0.0000", | ||
| "max": "24.0000", | ||
| "sum": "82.0000", | ||
| "metric_count": 16 | ||
| } | ||
| ], | ||
| "valid_to": "2018-07-12T15:41:06.419Z" | ||
| } | ||
| ], | ||
| "metric": "request.users", | ||
| "interval": 86400.0, | ||
| "type": "value", | ||
| "axis_label": "Count", | ||
| "label": null | ||
| } | ||
| } | ||
| 2. total number of unique visitors per URL (excluding ows requests). Let me see how many users visits the layers page or the maps page | ||
|
|
||
| * get number of unique tracking ids for urls | ||
|
|
||
| ``GET /monitoring/api/metric_data/request.users/?`` | ||
| ``last=(x*86400)&interval=86400&group_by=resource_no_label&resource_type=url&event_type=other`` | ||
|
|
||
| .. code-block:: json | ||
| { | ||
| "data": { | ||
| "input_valid_from": "2018-07-11T15:39:25.126Z", | ||
| "input_valid_to": "2018-07-12T15:39:25.126Z", | ||
| "data": [ | ||
| { | ||
| "valid_from": "2018-07-11T15:39:25.126Z", | ||
| "data": [ | ||
| { | ||
| "resource": { | ||
| "type": "url", | ||
| "name": "/layers/", | ||
| "id": 15 | ||
| }, | ||
| "metric_count": 2, | ||
| "val": 2, | ||
| "min": "1.0000", | ||
| "max": "1.0000", | ||
| "sum": "2.0000", | ||
| "samples_count": 2 | ||
| }, | ||
| { | ||
| "resource": { | ||
| "type": "url", | ||
| "name": "/", | ||
| "id": 16 | ||
| }, | ||
| "metric_count": 2, | ||
| "val": 2, | ||
| "min": "1.0000", | ||
| "max": "1.0000", | ||
| "sum": "2.0000", | ||
| "samples_count": 2 | ||
| }, | ||
| { | ||
| "resource": { | ||
| "type": "url", | ||
| "name": "/documents/", | ||
| "id": 21 | ||
| }, | ||
| "metric_count": 1, | ||
| "val": 1, | ||
| "min": "1.0000", | ||
| "max": "1.0000", | ||
| "sum": "1.0000", | ||
| "samples_count": 1 | ||
| } | ||
| ], | ||
| "valid_to": "2018-07-12T15:39:25.126Z" | ||
| } | ||
| ], | ||
| "metric": "request.users", | ||
| "interval": 86400.0, | ||
| "type": "value", | ||
| "axis_label": "Count", | ||
| "label": null | ||
| } | ||
| } | ||
| 3. total number of unique visitors per event_type: for example total number of unique visits of resource pages (indipendently by resource type and id) | ||
|
|
||
| * to get number of requests | ||
|
|
||
| ``GET /monitoring/api/metric_data/request.users/?`` | ||
| ``last=86400&interval=86400&group_by=event_type`` | ||
|
|
||
| * to get number of unique tracking ids | ||
|
|
||
| ``GET /monitoring/api/metric_data/request.users/?`` | ||
| ``last=86400&interval=86400&group_by=event_type_on_label`` | ||
|
|
||
| * to get number of unique tracking ids for specific resource type | ||
|
|
||
| ``GET /monitoring/api/metric_data/request.users/?`` | ||
| ``last=86400&interval=86400&group_by=event_type_on_label&resource_type=url`` | ||
|
|
||
| .. code-block:: json | ||
| { | ||
| "data": { | ||
| "input_valid_from": "2018-07-11T17:54:41.467Z", | ||
| "input_valid_to": "2018-07-12T17:54:41.467Z", | ||
| "data": [ | ||
| { | ||
| "valid_from": "2018-07-11T17:54:41.467Z", | ||
| "data": [ | ||
| { | ||
| "samples_count": 5, | ||
| "event_type": "all", | ||
| "val": 2, | ||
| "min": "1.0000", | ||
| "max": "1.0000", | ||
| "sum": "5.0000", | ||
| "metric_count": 5 | ||
| }, | ||
| { | ||
| "samples_count": 5, | ||
| "event_type": "other", | ||
| "val": 2, | ||
| "min": "1.0000", | ||
| "max": "1.0000", | ||
| "sum": "5.0000", | ||
| "metric_count": 5 | ||
| }, | ||
| { | ||
| "samples_count": 5, | ||
| "event_type": "view", | ||
| "val": 2, | ||
| "min": "1.0000", | ||
| "max": "1.0000", | ||
| "sum": "5.0000", | ||
| "metric_count": 5 | ||
| } | ||
| ], | ||
| "valid_to": "2018-07-12T17:54:41.467Z" | ||
| } | ||
| ], | ||
| "metric": "request.users", | ||
| "interval": 86400.0, | ||
| "type": "value", | ||
| "axis_label": "Count", | ||
| "label": null | ||
| } | ||
| } | ||
| 4. total number of unique visitors per event_type and single resource: let me see what was the most visited map page in this day, or what was the most downloaded document, what was the most requested ows layer, etc. | ||
|
|
||
| * list of most visited resources of `url` type | ||
|
|
||
| ``GET /monitoring/api/metric_data/request.users/?`` | ||
| ``last=86400&interval=86400&group_by=resource_no_label&resource_type=url`` | ||
|
|
||
| * list of unique tracking ids for each resource (can be narrowed down to specific resource type with `resource_type` values). | ||
|
|
||
| ``GET /monitoring/api/metric_data/request.users/?`` | ||
| ``last=86400&interval=86400&group_by=resource_no_label`` | ||
|
|
||
| .. code-block:: json | ||
| { | ||
| "data": { | ||
| "input_valid_from": "2018-07-11T17:56:49.381Z", | ||
| "input_valid_to": "2018-07-12T17:56:49.381Z", | ||
| "data": [ | ||
| { | ||
| "valid_from": "2018-07-11T17:56:49.381Z", | ||
| "data": [ | ||
| { | ||
| "resource": { | ||
| "type": "", | ||
| "name": "", | ||
| "id": 1 | ||
| }, | ||
| "metric_count": 16, | ||
| "val": 9, | ||
| "min": "0.0000", | ||
| "max": "24.0000", | ||
| "sum": "82.0000", | ||
| "samples_count": 82 | ||
| }, | ||
| { | ||
| "resource": { | ||
| "type": "layer", | ||
| "name": "geonode:ne_50m_admin_0_countries_lakes", | ||
| "id": 2 | ||
| }, | ||
| "metric_count": 4, | ||
| "val": 3, | ||
| "min": "0.0000", | ||
| "max": "2.0000", | ||
| "sum": "3.0000", | ||
| "samples_count": 3 | ||
| }, | ||
| { | ||
| "resource": { | ||
| "type": "layer", | ||
| "name": "geonode:world_iso2", | ||
| "id": 12 | ||
| }, | ||
| "metric_count": 4, | ||
| "val": 2, | ||
| "min": "0.0000", | ||
| "max": "5.0000", | ||
| "sum": "8.0000", | ||
| "samples_count": 8 | ||
| }, | ||
| { | ||
| "resource": { | ||
| "type": "url", | ||
| "name": "/layers/", | ||
| "id": 15 | ||
| }, | ||
| "metric_count": 2, | ||
| "val": 2, | ||
| "min": "1.0000", | ||
| "max": "1.0000", | ||
| "sum": "2.0000", | ||
| "samples_count": 2 | ||
| }, | ||
| { | ||
| "resource": { | ||
| "type": "url", | ||
| "name": "/", | ||
| "id": 16 | ||
| }, | ||
| "metric_count": 2, | ||
| "val": 2, | ||
| "min": "1.0000", | ||
| "max": "1.0000", | ||
| "sum": "2.0000", | ||
| "samples_count": 2 | ||
| }, | ||
| { | ||
| "resource": { | ||
| "type": "url", | ||
| "name": "/documents/", | ||
| "id": 21 | ||
| }, | ||
| "metric_count": 1, | ||
| "val": 1, | ||
| "min": "1.0000", | ||
| "max": "1.0000", | ||
| "sum": "1.0000", | ||
| "samples_count": 1 | ||
| }, | ||
| { | ||
| "resource": { | ||
| "type": "document", | ||
| "name": "GeoServer Configuration.pdf", | ||
| "id": 22 | ||
| }, | ||
| "metric_count": 1, | ||
| "val": 1, | ||
| "min": "5.0000", | ||
| "max": "5.0000", | ||
| "sum": "5.0000", | ||
| "samples_count": 5 | ||
| } | ||
| ], | ||
| "valid_to": "2018-07-12T17:56:49.381Z" | ||
| } | ||
| ], | ||
| "metric": "request.users", | ||
| "interval": 86400.0, | ||
| "type": "value", | ||
| "axis_label": "Count", | ||
| "label": null | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| # -*- coding: utf-8 -*- | ||
| ######################################################################### | ||
| # | ||
| # Copyright (C) 2019 OSGeo | ||
| # | ||
| # This program is free software: you can redistribute it and/or modify | ||
| # it under the terms of the GNU General Public License as published by | ||
| # the Free Software Foundation, either version 3 of the License, or | ||
| # (at your option) any later version. | ||
| # | ||
| # This program is distributed in the hope that it will be useful, | ||
| # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| # GNU General Public License for more details. | ||
| # | ||
| # You should have received a copy of the GNU General Public License | ||
| # along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
| # | ||
| ######################################################################### | ||
|
|
||
| from __future__ import print_function | ||
| import logging | ||
|
|
||
| from django.core.management.base import BaseCommand | ||
|
|
||
| from geonode.monitoring.collector import CollectorAPI | ||
|
|
||
| log = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| class Command(BaseCommand): | ||
|
|
||
| def handle(self, *args, **kwargs): | ||
| c = CollectorAPI() | ||
| c.aggregate_past_periods() |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,302 @@ | ||
| # -*- coding: utf-8 -*- | ||
| ######################################################################### | ||
| # | ||
| # Copyright (C) 2018 OSGeo | ||
| # | ||
| # This program is free software: you can redistribute it and/or modify | ||
| # it under the terms of the GNU General Public License as published by | ||
| # the Free Software Foundation, either version 3 of the License, or | ||
| # (at your option) any later version. | ||
| # | ||
| # This program is distributed in the hope that it will be useful, | ||
| # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| # GNU General Public License for more details. | ||
| # | ||
| # You should have received a copy of the GNU General Public License | ||
| # along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
| # | ||
| ######################################################################### | ||
|
|
||
| from datetime import datetime, timedelta, time | ||
| from decimal import Decimal | ||
| import logging | ||
|
|
||
| import pytz | ||
|
|
||
| from django.conf import settings | ||
| from django.db.models import Sum, F | ||
| from django.core.exceptions import ObjectDoesNotExist, MultipleObjectsReturned | ||
|
|
||
| from geonode.monitoring.utils import generate_periods | ||
| from geonode.monitoring.models import (Metric, MetricValue, ServiceTypeMetric, | ||
| MonitoredResource, MetricLabel, EventType,) | ||
|
|
||
|
|
||
| log = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| def get_metric_names(): | ||
|
|
||
| """ | ||
| Returns list of tuples: (service type, list of metrics) | ||
| """ | ||
| q = ServiceTypeMetric.objects.all().select_related( | ||
| ).order_by('service_type', 'metric') | ||
|
|
||
| out = [] | ||
| current_service = None | ||
| current_set = [] | ||
| for item in q: | ||
| service, metric = item.service_type, item.metric | ||
| if current_service != service: | ||
| if current_service is not None: | ||
| out.append((current_service, current_set,)) | ||
| current_set = [] | ||
| current_service = service | ||
| current_set.append(metric) | ||
| if current_set: | ||
| out.append((current_service, current_set,)) | ||
|
|
||
| return out | ||
|
|
||
|
|
||
| def get_labels_for_metric(metric_name, resource=None): | ||
| mt = ServiceTypeMetric.objects.filter(metric__name=metric_name) | ||
| if not mt: | ||
| raise ValueError("No metric for {}".format(metric_name)) | ||
|
|
||
| qparams = {'metric_values__service_metric__in': mt} | ||
| if resource: | ||
| qparams['metricvalue__resource'] = resource | ||
| return list(MetricLabel.objects.filter( | ||
| **qparams).distinct().values_list('id', 'name')) | ||
|
|
||
|
|
||
| def get_resources_for_metric(metric_name): | ||
| mt = ServiceTypeMetric.objects.filter(metric__name=metric_name) | ||
| if not mt: | ||
| raise ValueError("No metric for {}".format(metric_name)) | ||
| return list(MonitoredResource.objects.filter(metric_values__service_metric__in=mt) | ||
| .exclude(name='', type='') | ||
| .distinct() | ||
| .order_by('type', 'name') | ||
| .values_list('type', 'name')) | ||
|
|
||
|
|
||
| def extract_resources(requests): | ||
| resources = MonitoredResource.objects.filter( | ||
| requests__in=requests).distinct() | ||
| out = [] | ||
| for res in resources: | ||
| out.append((res, requests.filter(resources=res).distinct(),)) | ||
| return out | ||
|
|
||
|
|
||
| def extract_event_type(requests): | ||
| q = requests.exclude(event_type__isnull=True).distinct( | ||
| 'event_type').values_list('event_type', flat=True) | ||
| try: | ||
| return q.get() | ||
| except (ObjectDoesNotExist, MultipleObjectsReturned,): | ||
| pass | ||
|
|
||
|
|
||
| def extract_event_types(requests): | ||
| event_types = requests.exclude(event_type__isnull=True)\ | ||
| .distinct('event_type')\ | ||
| .values_list('event_type', flat=True) | ||
| return [EventType.objects.get(id=evt_id) for evt_id in event_types] | ||
|
|
||
|
|
||
| def extract_special_event_types(requests): | ||
| """ | ||
| Return list of pairs (event_type, requests) | ||
| that should be registered as one of aggregating event types: ows:all, other, | ||
| """ | ||
| out = [] | ||
|
|
||
| ows_et = requests.exclude(event_type__isnull=True)\ | ||
| .filter(event_type__name__startswith='OWS:')\ | ||
| .exclude(event_type__name=EventType.EVENT_OWS)\ | ||
| .distinct('event_type')\ | ||
| .values_list('event_type', flat=True) | ||
| ows_rq = requests.filter(event_type__in=ows_et) | ||
| ows_all = EventType.get(EventType.EVENT_OWS) | ||
| out.append((ows_all, ows_rq,)) | ||
|
|
||
| nonows_et = requests.exclude(event_type__isnull=True)\ | ||
| .exclude(event_type__name__startswith='OWS:')\ | ||
| .exclude(event_type__name=EventType.EVENT_OTHER)\ | ||
| .distinct('event_type')\ | ||
| .values_list('event_type', flat=True) | ||
| nonows_rq = requests.filter(event_type__in=nonows_et) | ||
| nonows_all = EventType.get(EventType.EVENT_OTHER) | ||
| out.append((nonows_all, nonows_rq,)) | ||
|
|
||
| return out | ||
|
|
||
|
|
||
| def calculate_rate(metric_name, metric_label, | ||
| current_value, valid_to): | ||
| """ | ||
| Find previous network metric value and caclulate rate between them | ||
| """ | ||
| prev = MetricValue.objects.filter(service_metric__metric__name=metric_name, | ||
| label__name=metric_label, | ||
| valid_to__lt=valid_to)\ | ||
| .order_by('-valid_to').first() | ||
| if not prev: | ||
| return | ||
| prev_val = prev.value_num | ||
| valid_to = valid_to.replace(tzinfo=pytz.utc) | ||
| prev.valid_to = prev.valid_to.replace(tzinfo=pytz.utc) | ||
| interval = valid_to - prev.valid_to | ||
| if not isinstance(current_value, Decimal): | ||
| current_value = Decimal(current_value) | ||
|
|
||
| # this means counter was reset, don't want rates below 0 | ||
| if current_value < prev_val: | ||
| return | ||
| rate = float((current_value - prev_val)) / interval.total_seconds() | ||
| return rate | ||
|
|
||
|
|
||
| def calculate_percent( | ||
| metric_name, metric_label, current_value, valid_to): | ||
| """ | ||
| Find previous network metric value and caclulate percent | ||
| """ | ||
| rate = calculate_rate( | ||
| metric_name, metric_label, current_value, valid_to) | ||
| if rate is None: | ||
| return | ||
| return rate * 100 | ||
|
|
||
|
|
||
| def adjust_now_to_noon(now): | ||
| return pytz.utc.localize(datetime.combine(now.date(), time(0, 0, 0))) | ||
|
|
||
|
|
||
| def aggregate_past_periods(metric_data_q=None, periods=None, cleanup=True, now=None, max_since=None): | ||
| """ | ||
| Aggregate past metric data into longer periods | ||
| @param metric_data_q Query for metric data to use as input | ||
| (default: all MetricValues) | ||
| @param periods list of tuples (cutoff, aggregation) to be used | ||
| (default: settings.MONITORING_DATA_AGGREGATION) | ||
| @param cleanup flag if input data should be removed after aggregation | ||
| (default: True) | ||
| @param now arbitrary now moment to start calculation of cutoff | ||
| (default: current now) | ||
| @param max_since look for data no older than max_since | ||
| (default: 1 year) | ||
| """ | ||
| utc = pytz.utc | ||
| if now is None: | ||
| now = datetime.utcnow().replace(tzinfo=utc) | ||
| if metric_data_q is None: | ||
| metric_data_q = MetricValue.objects.all() | ||
| if periods is None: | ||
| periods = settings.MONITORING_DATA_AGGREGATION | ||
| max_since = max_since or now - timedelta(days=356) | ||
| previous_cutoff = None | ||
| counter = 0 | ||
| now = adjust_now_to_noon(now) | ||
| # start from the end, oldest one first | ||
| for cutoff_base, aggregation_period in reversed(periods): | ||
| since = previous_cutoff or max_since | ||
| until = now - cutoff_base | ||
|
|
||
| if since > until: | ||
| log.info("Wrong period boundaries, end %s is before start %s, agg: %s", | ||
| until, since, aggregation_period) | ||
| previous_cutoff = max(until, since) | ||
| continue | ||
|
|
||
| log.info("aggregation params: cutoff: %s agg period: %s" | ||
| "\n since: '%s' until '%s', but previous cutoff:" | ||
| " '%s', aggregate to '%s'", | ||
| cutoff_base, aggregation_period, since, until, previous_cutoff, aggregation_period) | ||
|
|
||
| periods = generate_periods(since, aggregation_period, end=until) | ||
|
|
||
| # for each target period we select mertic values within it | ||
| # and extract service, resource, event type and label combinations | ||
| # then, for each distinctive set, calculate per-metric aggregate values | ||
| for period_start, period_end in periods: | ||
| log.info('period %s - %s (%s s)', period_start, period_end, period_end-period_start) | ||
| ret = aggregate_period(period_start, period_end, metric_data_q, cleanup) | ||
| counter += ret | ||
| previous_cutoff = until | ||
| return counter | ||
|
|
||
|
|
||
| def aggregate_period(period_start, period_end, metric_data_q, cleanup=True): | ||
| counter = 0 | ||
| to_remove_data = {'remove_at': period_start.strftime("%Y%m%d%H%M%S")} | ||
| source_metric_data = metric_data_q.filter(valid_from__gte=period_start, | ||
| valid_to__lte=period_end)\ | ||
| .exclude(valid_from=period_start, | ||
| valid_to=period_end, | ||
| data={}) | ||
| r = source_metric_data.values_list('service_id', 'service_metric_id', 'resource_id', 'event_type_id', 'label_id',)\ | ||
| .distinct('service_id', 'service_metric_id', 'resource_id', 'event_type_id', 'label_id') | ||
| source_metric_data.update(data=to_remove_data) | ||
|
|
||
| for service_id, metric_id, resource_id, event_type_id, label_id in r: | ||
| m = Metric.objects.filter(service_type__id=metric_id).get() | ||
| f = m.get_aggregate_field() | ||
| per_metric_q = source_metric_data.filter(service_metric_id=metric_id, | ||
| service_id=service_id, | ||
| resource_id=resource_id, | ||
| event_type_id=event_type_id, | ||
| label_id=label_id) | ||
|
|
||
| try: | ||
| value_q = per_metric_q.aggregate(fvalue=f, | ||
| fsamples_count=Sum(F('samples_count'))) | ||
| except TypeError, err: | ||
| raise ValueError(f, m, err) | ||
| value = value_q['fvalue'] | ||
| samples_count = value_q['fsamples_count'] | ||
| if cleanup: | ||
| per_metric_q.delete() | ||
| log.info('Metric %s: %s - %s (value: %s, samples: %s)', | ||
| m, period_start, period_end, value, samples_count) | ||
| if not metric_data_q.filter(service_metric_id=metric_id, | ||
| service_id=service_id, | ||
| resource_id=resource_id, | ||
| event_type_id=event_type_id, | ||
| valid_from=period_start, | ||
| valid_to=period_end, | ||
| label_id=label_id).exists(): | ||
| MetricValue.objects.create(service_metric_id=metric_id, | ||
| service_id=service_id, | ||
| resource_id=resource_id, | ||
| event_type_id=event_type_id, | ||
| value=value, | ||
| value_num=value, | ||
| value_raw=value, | ||
| valid_from=period_start, | ||
| valid_to=period_end, | ||
| label_id=label_id, | ||
| samples_count=samples_count) | ||
| else: | ||
| metric_data_q.filter(service_metric_id=metric_id, | ||
| service_id=service_id, | ||
| resource_id=resource_id, | ||
| event_type_id=event_type_id, | ||
| valid_from=period_start, | ||
| valid_to=period_end, | ||
| label_id=label_id)\ | ||
| .update(value=value, | ||
| value_num=value, | ||
| value_raw=value, | ||
| data=None, | ||
| samples_count=samples_count) | ||
| counter += 1 | ||
|
|
||
| if cleanup: | ||
| source_metric_data.filter(data=to_remove_data).delete() | ||
| return counter |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| # -*- coding: utf-8 -*- | ||
| ######################################################################### | ||
| # | ||
| # Copyright (C) 2018 OSGeo | ||
| # | ||
| # This program is free software: you can redistribute it and/or modify | ||
| # it under the terms of the GNU General Public License as published by | ||
| # the Free Software Foundation, either version 3 of the License, or | ||
| # (at your option) any later version. | ||
| # | ||
| # This program is distributed in the hope that it will be useful, | ||
| # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| # GNU General Public License for more details. | ||
| # | ||
| # You should have received a copy of the GNU General Public License | ||
| # along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
| # | ||
| ######################################################################### | ||
|
|
||
| from __future__ import print_function | ||
| import logging | ||
|
|
||
| from django.core.management.base import BaseCommand | ||
|
|
||
| from geonode.monitoring.collector import CollectorAPI | ||
|
|
||
| log = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| class Command(BaseCommand): | ||
|
|
||
| def handle(self, *args, **kwargs): | ||
| c = CollectorAPI() | ||
| c.aggregate_past_periods() |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| # -*- coding: utf-8 -*- | ||
| # Generated by Django 1.11.22 on 2019-08-13 08:08 | ||
| from __future__ import unicode_literals | ||
|
|
||
| from django.db import migrations, models | ||
| import django.db.models.deletion | ||
|
|
||
|
|
||
| class Migration(migrations.Migration): | ||
|
|
||
| dependencies = [ | ||
| ('monitoring', '0024_auto_20190605_1619'), | ||
| ] | ||
|
|
||
| operations = [ | ||
| migrations.CreateModel( | ||
| name='EventType', | ||
| fields=[ | ||
| ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), | ||
| ('name', models.CharField(choices=[(b'OWS:TMS', b'TMS'), (b'OWS:WMS-C', b'WMS-C'), (b'OWS:WMTS', b'WMTS'), (b'OWS:WCS', b'WCS'), (b'OWS:WFS', b'WFS'), (b'OWS:WMS', b'WMS'), (b'OWS:WPS', b'WPS'), (b'other', b'Non-OWS'), (b'OWS:ALL', b'Any OWS'), (b'all', b'All'), (b'create', b'Create'), (b'upload', b'Upload'), (b'change', b'Change'), (b'change_metadata', b'Change Metadata'), (b'view_metadata', b'View Metadata'), (b'view', b'View'), (b'download', b'Download'), (b'publish', b'Publish'), (b'remove', b'Remove'), (b'geoserver', b'Geoserver event')], max_length=16, unique=True)), | ||
| ], | ||
| ), | ||
| migrations.RemoveField( | ||
| model_name='metricnotificationcheck', | ||
| name='ows_service', | ||
| ), | ||
| migrations.RemoveField( | ||
| model_name='notificationmetricdefinition', | ||
| name='use_ows_service', | ||
| ), | ||
| migrations.RemoveField( | ||
| model_name='requestevent', | ||
| name='ows_service', | ||
| ), | ||
| migrations.AddField( | ||
| model_name='notificationmetricdefinition', | ||
| name='use_event_type', | ||
| field=models.BooleanField(default=False), | ||
| ), | ||
| migrations.AddField( | ||
| model_name='requestevent', | ||
| name='user_anonymous', | ||
| field=models.BooleanField(default=True), | ||
| ), | ||
| migrations.AddField( | ||
| model_name='requestevent', | ||
| name='user_identifier', | ||
| field=models.CharField(blank=True, db_index=True, default=None, max_length=255, null=True), | ||
| ), | ||
| migrations.AlterField( | ||
| model_name='monitoredresource', | ||
| name='type', | ||
| field=models.CharField(choices=[(b'', b'No resource'), (b'layer', b'Layer'), (b'map', b'Map'), (b'resource base', b'Resource base'), (b'document', b'Document'), (b'style', b'Style'), (b'admin', b'Admin'), (b'url', b'URL'), (b'other', b'Other')], default=b'', max_length=255), | ||
| ), | ||
| migrations.AlterField( | ||
| model_name='requestevent', | ||
| name='client_ip', | ||
| field=models.GenericIPAddressField(blank=True, null=True), | ||
| ), | ||
| migrations.AddField( | ||
| model_name='metricnotificationcheck', | ||
| name='event_type', | ||
| field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to='monitoring.EventType'), | ||
| ), | ||
| migrations.AddField( | ||
| model_name='metricvalue', | ||
| name='event_type', | ||
| field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='metric_values', to='monitoring.EventType'), | ||
| ), | ||
| migrations.AddField( | ||
| model_name='requestevent', | ||
| name='event_type', | ||
| field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to='monitoring.EventType'), | ||
| ), | ||
| migrations.AlterUniqueTogether( | ||
| name='metricvalue', | ||
| unique_together=set( | ||
| [('valid_from', 'valid_to', 'service', 'service_metric', 'resource', 'label', 'event_type')]), | ||
| ), | ||
| migrations.RemoveField( | ||
| model_name='metricvalue', | ||
| name='ows_service', | ||
| ), | ||
| migrations.DeleteModel( | ||
| name='OWSService', | ||
| ), | ||
| ] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| # -*- coding: utf-8 -*- | ||
| # Generated by Django 1.11.23 on 2019-08-21 07:36 | ||
| from __future__ import unicode_literals | ||
|
|
||
| from django.db import migrations, models | ||
| import django.db.models.deletion | ||
|
|
||
|
|
||
| class Migration(migrations.Migration): | ||
|
|
||
| dependencies = [ | ||
| ('monitoring', '0025_auto_20190813_0808'), | ||
| ] | ||
|
|
||
| operations = [ | ||
| migrations.RemoveField( | ||
| model_name='requestevent', | ||
| name='user_anonymous', | ||
| ), | ||
| migrations.AddField( | ||
| model_name='requestevent', | ||
| name='user_username', | ||
| field=models.CharField(blank=True, default=None, max_length=150, null=True), | ||
| ), | ||
| migrations.AlterField( | ||
| model_name='metricvalue', | ||
| name='resource', | ||
| field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='metric_values', to='monitoring.MonitoredResource'), | ||
| ), | ||
| migrations.AlterField( | ||
| model_name='monitoredresource', | ||
| name='type', | ||
| field=models.CharField(choices=[(b'', b'No resource'), (b'layer', b'Layer'), (b'map', b'Map'), (b'resource_base', b'Resource base'), (b'document', b'Document'), (b'style', b'Style'), (b'admin', b'Admin'), (b'url', b'URL'), (b'other', b'Other')], default=b'', max_length=255), | ||
| ), | ||
| ] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # -*- coding: utf-8 -*- | ||
| ######################################################################### | ||
| # | ||
| # Copyright (C) 2016 OSGeo | ||
| # | ||
| # This program is free software: you can redistribute it and/or modify | ||
| # it under the terms of the GNU General Public License as published by | ||
| # the Free Software Foundation, either version 3 of the License, or | ||
| # (at your option) any later version. | ||
| # | ||
| # This program is distributed in the hope that it will be useful, | ||
| # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| # GNU General Public License for more details. | ||
| # | ||
| # You should have received a copy of the GNU General Public License | ||
| # along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
| # | ||
| ######################################################################### |
