From d93485dde31fb67515156777fa04639aa25f95a6 Mon Sep 17 00:00:00 2001 From: Ajay Tripathi Date: Thu, 11 Jun 2020 19:58:07 +0530 Subject: [PATCH] [end] Archive Notice #124 #130 Closes #124 Closes #130 --- .gitignore | 1 + README.rst | 663 ++--------------------------------------------------- 2 files changed, 17 insertions(+), 647 deletions(-) diff --git a/.gitignore b/.gitignore index b9879f3..cae5567 100644 --- a/.gitignore +++ b/.gitignore @@ -62,3 +62,4 @@ target/ *~ local_settings.py *.db +*.json diff --git a/README.rst b/README.rst index f3ce279..672d84f 100644 --- a/README.rst +++ b/README.rst @@ -1,662 +1,31 @@ django-netjsongraph =================== -.. image:: https://travis-ci.org/openwisp/django-netjsongraph.svg - :target: https://travis-ci.org/openwisp/django-netjsongraph +**WARNING**: The development of this project has moved to `openwisp-network-topology `_, we advise all users of *django-netjsongraph* to follow the tutorial below to migrate their existing database: -.. image:: https://coveralls.io/repos/openwisp/django-netjsongraph/badge.svg - :target: https://coveralls.io/r/openwisp/django-netjsongraph - -.. image:: https://requires.io/github/openwisp/django-netjsongraph/requirements.svg?branch=master - :target: https://requires.io/github/openwisp/django-netjsongraph/requirements/?branch=master - :alt: Requirements Status - -.. image:: https://badge.fury.io/py/django-netjsongraph.svg - :target: http://badge.fury.io/py/django-netjsongraph - -.. image:: https://img.shields.io/gitter/room/nwjs/nw.js.svg?style=flat-square - :target: https://gitter.im/openwisp/general - ------------- - -Reusable django app for collecting and visualizing network topology. - -.. image:: https://raw.githubusercontent.com/openwisp/django-netjsongraph/master/docs/images/visualizer.png - -.. image:: https://raw.githubusercontent.com/openwisp/django-netjsongraph/master/docs/images/admin.png - -.. contents:: **Table of Contents**: - :backlinks: none - :depth: 3 - ------------- - -Current features ----------------- - -* **network topology collector** supporting different formats: - - NetJSON NetworkGraph - - OLSR (jsoninfo/txtinfo) - - batman-adv (jsondoc/txtinfo) - - BMX6 (q6m) - - CNML 1.0 - - OpenVPN - - additional formats can be added by `specifying custom parsers <#netjsongraph-parsers>`_ -* **network topology visualizer** based on `netjsongraph.js `_ -* **simple HTTP API** that exposes data in `NetJSON `__ *NetworkGraph* format -* **admin interface** that allows to easily manage, audit, visualize and debug topologies and their relative data (nodes, links) -* **receive topology** from multiple nodes -* **topology history**: allows saving daily snapshots of each topology that can be viewed in the frontend - -Project goals -------------- - -* make it easy to visualize network topology data for the formats supported by `netdiff `_ -* expose topology data via RESTful resources in *NetJSON NetworkGraph* format -* make it easy to integrate in larger django projects to improve reusability -* make it easy to extend its models by providing abstract models (**needs improvement in this point**) -* provide ways to customize or replace the visualizer (**needs improvement in this point**) -* keep the core very simple -* provide ways to extend the default behaviour -* encourage new features to be published as extensions - -Deploy it in production ------------------------ - -An automated installer is provided by the `OpenWISP `_ project: -`ansible-openwisp2 `_. - -Ensure to follow the instructions explained in the following section: `Enabling the network topology -module `_. - -Install stable version from pypi --------------------------------- - -Install from pypi: - -.. code-block:: shell - - pip install django-netjsongraph - -Install development version ---------------------------- - -Install tarball: - -.. code-block:: shell - - pip install https://github.com/openwisp/django-netjsongraph/tarball/master - -Alternatively you can install via pip using git: - -.. code-block:: shell - - pip install -e git+git://github.com/openwisp/django-netjsongraph#egg=django-netjsongraph - -If you want to contribute, install your cloned fork: - -.. code-block:: shell - - git clone git@github.com:/django-netjsongraph.git - cd django-netjsongraph - python setup.py develop - -Setup (integrate in an existing django project) ------------------------------------------------ - -Add ``rest_framework`` and ``django_netjsongraph`` to ``INSTALLED_APPS``: - -.. code-block:: python - - INSTALLED_APPS = [ - # other apps - 'rest_framework', - 'openwisp_utils.admin_theme', - 'django_netjsongraph' - # ... - ] - -Include urls in your urlconf (you can change the prefixes -according to your needs): - -.. code-block:: python - - from django.conf.urls import include, url - - from django_netjsongraph.api import urls as netjsongraph_api - from django_netjsongraph.visualizer import urls as netjsongraph_visualizer - - urlpatterns = [ - # your URLs ... - url(r'^api/', include(netjsongraph_api)), - url(r'', include(netjsongraph_visualizer)), - ] - -Create database tables:: - - ./manage.py migrate - -Management Commands -------------------- - -``update_topology`` -^^^^^^^^^^^^^^^^^^^ - -After topology URLs (URLs exposing the files that the topology of the network) have been -added in the admin, the ``update_topology`` management command can be used to collect data -and start playing with the network graph:: - - ./manage.py update_topology - -The management command accepts a ``--label`` argument that will be used to search in -topology labels, eg:: - - ./manage.py update_topology --label mytopology - -``save_snapshot`` -^^^^^^^^^^^^^^^^^ - -The ``save_snapshot`` management command can be used to save the topology graph data which -could be used to view the network topology graph sometime in future:: - - ./manage.py save_snapshot - -The management command accepts a ``--label`` argument that will be used to search in -topology labels, eg:: - - ./manage.py save_snapshot --label mytopology - -Logging -------- - -The ``update_topology`` management command will automatically try to log errors. - -For a good default ``LOGGING`` configuration refer to the `test settings -`_. - -Strategies ----------- - -There are mainly two ways of collecting topology information: - -* **FETCH** strategy -* **RECEIVE** strategy - -Each ``Topology`` instance has a ``strategy`` field which can be set to the desired setting. - -FETCH strategy -^^^^^^^^^^^^^^ - -Topology data will be fetched from a URL. - -When some links are not detected anymore they will be flagged as "down" straightaway. - -RECEIVE strategy -^^^^^^^^^^^^^^^^ - -Topology data is sent directly from one or more nodes of the network. - -The collector waits to receive data in the payload of a POST HTTP request; -when such a request is received, a ``key`` parameter it's first checked against -the ``Topology`` key. - -If the request is authorized the collector proceeds to update the topology. - -If the data is sent from one node only, it's highly advised to set the -``expiration_time`` of the ``Topology`` instance to ``0`` (seconds), this way the -system works just like in the **FETCH strategy**, with the only difference that -the data is sent by one node instead of fetched by the collector. - -If the data is sent from multiple nodes, you **SHOULD** set the ``expiration_time`` -of the ``Topology`` instance to a value slightly higher than the interval used -by nodes to send the topology, this way links will be flagged as "down" only if -they haven't been detected for a while. This mechanism allows to visualize the -topology even if the network has been split in several parts, the disadvantage -is that it will take a bit more time to detect links that go offline. - -Settings --------- - -``NETJSONGRAPH_PARSERS`` -^^^^^^^^^^^^^^^^^^^^^^^^ - -+--------------+-------------+ -| **type**: | ``list`` | -+--------------+-------------+ -| **default**: | ``[]`` | -+--------------+-------------+ - -Additional custom `netdiff parsers `_. - -``NETJSONGRAPH_SIGNALS`` -^^^^^^^^^^^^^^^^^^^^^^^^ - -+--------------+-------------+ -| **type**: | ``str`` | -+--------------+-------------+ -| **default**: | ``None`` | -+--------------+-------------+ - -String representing python module to import on initialization. - -Useful for loading django signals or to define custom behaviour. - -``NETJSONGRAPH_TIMEOUT`` -^^^^^^^^^^^^^^^^^^^^^^^^ - -+--------------+-------------+ -| **type**: | ``int`` | -+--------------+-------------+ -| **default**: | ``8`` | -+--------------+-------------+ - -Timeout when fetching topology URLs. - -``NETJSONGRAPH_LINK_EXPIRATION`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -+--------------+-------------+ -| **type**: | ``int`` | -+--------------+-------------+ -| **default**: | ``60`` | -+--------------+-------------+ - -If a link is down for more days than this number, it will be deleted by the -``update_topology`` management command. - -Setting this to ``False`` will disable this feature. - -``NETJSONGRAPH_VISUALIZER_CSS`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -+--------------+--------------------------------+ -| **type**: | ``str`` | -+--------------+--------------------------------+ -| **default**: | ``netjsongraph/css/style.css`` | -+--------------+--------------------------------+ - -Path of the visualizer css file. Allows customization of css according to user's -preferences. - -``NETJSONGRAPH_NODE_EXPIRATION`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -+--------------+--------------------------------+ -| **type**: | ``int`` | -+--------------+--------------------------------+ -| **default**: | ``False`` | -+--------------+--------------------------------+ - -If a node has not been modified since the days specified and if it has no links, -it will be deleted by the ``update_topology`` management command. This depends on -``NETJSONGRAPH_LINK_EXPIRATION`` being enabled. -Replace ``False`` with an integer to enable the feature. - -``TOPOLOGY_API_URLCONF`` -^^^^^^^^^^^^^^^^^^^^^^^^ - -+--------------+---------------+ -| **type**: | ``string`` | -+--------------+---------------+ -| **default**: | ``None`` | -+--------------+---------------+ - -Use the ``urlconf`` option to change receive api url to point to -another module, example, ``myapp.urls``. - -``TOPOLOGY_API_BASEURL`` -^^^^^^^^^^^^^^^^^^^^^^^^ - -+--------------+---------------+ -| **type**: | ``string`` | -+--------------+---------------+ -| **default**: | ``None`` | -+--------------+---------------+ - -If you have a seperate instanse of django-netjsongraph on a -different domain, you can use this option to change the base -of the url, this will enable you to point all the API urls to -your django-netjsongraph API server's domain, -example value: ``https://mytopology.myapp.com``. - -Overriding visualizer templates -------------------------------- - -Follow these steps to override and customise the visualizer's default templates: - -* create a directory in your django project and put its full path in ``TEMPLATES['DIRS']``, - which can be found in the django ``settings.py`` file -* create a sub directory named ``netjsongraph`` and add all the templates which shall override - the default ``netjsongraph/*`` templates -* create a template file with the same name of the template file you want to override - -More information about the syntax used in django templates can be found in the `django templates -documentation `_. - -Example: overriding the `` - -Extending django-netjsongraph ------------------------------ - -*django-netjsongraph* provides a set of models, admin classes and generic views which can be imported, extended and reused by third party apps. - -To extend *django-netjsongraph*, **you MUST NOT** add it to ``settings.INSTALLED_APPS``, but you must create your own app (which goes into ``settings.INSTALLED_APPS``), import the base classes from django-netjsongraph and add your customizations. - -Extending models -^^^^^^^^^^^^^^^^ - -This example provides an example of how to extend the base models of -*django-netjsongraph*. - -.. code-block:: python - - # models.py of your custom ``network`` app - from django.db import models - - from django_netjsongraph.base.link import AbstractLink - from django_netjsongraph.base.node import AbstractNode - from django_netjsongraph.base.snapshot import AbstractSnapshot - from django_netjsongraph.base.topology import AbstractTopology - # the model ``organizations.Organization`` is omitted for brevity - # if you are curious to see a real implementation, check out django-organizations - # https://github.com/bennylope/django-organizations - - class OrganizationMixin(models.Model): - organization = models.ForeignKey('organization.Organization') - - class Meta: - abstract = True - - - class Topology(OrganizationMixin, AbstractTopology): - def clean(self): - # your own validation logic here - pass - - class Meta(AbstractTopology.Meta): - abstract = False - - - class Node(AbstractNode): - topology = models.ForeignKey('Topology') - - class Meta: - abstract = False - - - class Link(AbstractLink): - topology = models.ForeignKey('Topology') - source = models.ForeignKey('Node', - related_name='source_link_set') - target = models.ForeignKey('Node', - related_name='source_target_set') - - class Meta: - abstract = False - - class Snapshot(OrgMixin, AbstractSnapshot): - topology = models.ForeignKey('topology.Topology', on_delete=models.CASCADE) - - class Meta(AbstractSnapshot.Meta): - abstract = False - -Extending the admin -^^^^^^^^^^^^^^^^^^^ - -Following the above example, you can avoid duplicating the admin code by importing the base admin classes and registering your models with. - -.. code-block:: python - - # admin.py of your app - from django.contrib import admin - from django_netjsongraph.base.admin import (AbstractLinkAdmin, - AbstractNodeAdmin, - AbstractTopologyAdmin) - # these are you custom models - from .models import Link, Node, Topology - - - class TopologyAdmin(AbstractTopologyAdmin): - model = Topology - - - class NodeAdmin(AbstractNodeAdmin): - model = Node - - - class LinkAdmin(AbstractLinkAdmin): - model = Link - - - admin.site.register(Link, LinkAdmin) - admin.site.register(Node, NodeAdmin) - admin.site.register(Topology, TopologyAdmin) - -Extending API views -^^^^^^^^^^^^^^^^^^^ - -If your use case doesn't vary much from the base, you may also want to try to reuse the API views: - -.. code-block:: python - - # your app.api.views - from ..models import Snapshot, Topology - from django_netjsongraph.api.generics import (BaseNetworkCollectionView, BaseNetworkGraphHistoryView, - BaseNetworkGraphView, BaseReceiveTopologyView) - - - class NetworkCollectionView(BaseNetworkCollectionView): - queryset = Topology.objects.filter(published=True) - - - class NetworkGraphView(BaseNetworkGraphView): - queryset = Topology.objects.filter(published=True) - - - class ReceiveTopologyView(BaseReceiveTopologyView): - model = Topology - - - class NetworkGraphHistoryView(BaseNetworkGraphHistoryView): - topology_model = Topology - snapshot_model = Snapshot - - network_collection = NetworkCollectionView.as_view() - network_graph = NetworkGraphView.as_view() - network_graph_history = NetworkGraphHistoryView.as_view() - receive_topology = ReceiveTopologyView.as_view() - -API URLs -^^^^^^^^ - -If you are not making drastic changes to the api views, you can avoid duplicating the URL logic by using the ``get_api_urls`` function. Put this in your api ``urls.py``: - -.. code-block:: python - - # your app.api.urls - from django_netjsongraph.utils import get_api_urls - from . import views - - urlpatterns = get_api_urls(views) - -Extending Visualizer Views -^^^^^^^^^^^^^^^^^^^^^^^^^^ -If your use case doesn't vary much from the base, you may also want to try to reuse the Visualizer views: - -.. code-block:: python - - # your app.visualizer.views - from ..models import Topology - from .generics import BaseTopologyDetailView, BaseTopologyListView - - - class TopologyListView(BaseTopologyListView): - topology_model = Topology - - - class TopologyDetailView(BaseTopologyDetailView): - topology_model = Topology - - - topology_list = TopologyListView.as_view() - topology_detail = TopologyDetailView.as_view() - - -Visualizer URLs -^^^^^^^^^^^^^^^ -If you are not making any drastic changes to visualizer views, you can avoid duplicating the URL logic by using ``get_visualizer_urls`` function. Put this in your visualizer ``urls.py`` - -.. code-block:: python - - # your app.visualizer.urls - from django_netjsongraph.utils import get_visualizer_urls - from . import views - - urlpatterns = get_visualizer_urls(views) - -Extending AppConfig -^^^^^^^^^^^^^^^^^^^ - -You may want to reuse the ``AppConfig`` class of *django-netjsongraph* too: - -.. code-block:: python - - from django_netjsongraph.apps import DjangoNetjsongraphConfig - - class MyOwnConfig(DjangoNetjsongraphConfig): - name = 'yourapp' - label = 'yourapp' - -Installing for development --------------------------- - -Install sqlite: - -.. code-block:: shell - - sudo apt-get install sqlite3 libsqlite3-dev - -Install your forked repo: - -.. code-block:: shell - - git clone git://github.com//django-netjsongraph - cd django-netjsongraph/ - python setup.py develop - -Install test requirements: - -.. code-block:: shell - - pip install -r requirements-test.txt - -Create database: +1. Take a backup of your database. +2. Create JSON backup of data that's required for migration with the following command: .. code-block:: shell + # Go to the django-netjsongraph repository (or installed app) cd tests/ - ./manage.py migrate - ./manage.py createsuperuser + python manage.py dumpdata auth.user > user.json + python manage.py dumpdata auth.group > group.json + python manage.py dumpdata django_netjsongraph > netjsongraph.json + python manage.py dumpdata auth.permission > permission.json + python manage.py dumpdata contenttypes > contenttype.json + pwd # copy output of the command -Launch development server: +3. `Setup openwisp-network-topology `_ -.. code-block:: shell - - ./manage.py runserver - -You can access the visualizer at http://127.0.0.1:8000/ -and the admin interface at http://127.0.0.1:8000/admin/. - -Run tests with: +4. `Use the upgrader script `_: .. code-block:: shell - ./runtests.py - -Contributing ------------- - -First off, thanks for taking the time to read these guidelines. - -Trying to follow these guidelines is important in order to minimize waste and -avoid misunderstandings. - -1. Ensure your changes meet the `Project Goals`_ -2. If you found a bug please send a failing test with a patch -3. If you want to add a new feature, announce your intentions in the - `issue tracker `_ -4. Fork this repo and install it by following the instructions in - `Installing for development`_ -5. Follow `PEP8, Style Guide for Python Code`_ -6. Write code -7. Write tests for your code -8. Ensure all tests pass -9. Ensure test coverage is not under 90% -10. Document your changes -11. Send pull request - -.. _PEP8, Style Guide for Python Code: http://www.python.org/dev/peps/pep-0008/ - -Changelog ---------- - -See `CHANGES `_. - -License -------- - -See `LICENSE `_. + # In the openwisp-network-topology repository + python tests/manage.py upgrade_from_django_netjsongraph --backup -This projects bundles third-party javascript libraries in its source code: +For any support, please reach out to us on `the chat channel on gitter `_ or `use the mailing list `_. -- `D3.js (BSD-3-Clause) `_ +**The development of django-netjsongraph is discontinued and this repository is archived.**