diff --git a/docs/application-settings.rst b/docs/application-settings.rst index f10443c..2cec8f4 100644 --- a/docs/application-settings.rst +++ b/docs/application-settings.rst @@ -63,34 +63,6 @@ responsible to provide your own validation of the Host header. ALLOWED_HOSTS=* -.. _dejacode_settings_purldb: - -PURLDB ------- - -Provide the URL and API key of your `PurlDB `_ -instance. - -:: - - PURLDB_URL=https://your-purldb-domain/ - PURLDB_API_KEY=apikeyexample - -.. _dejacode_settings_vulnerablecode: - -VULNERABLECODE --------------- - -You can either run your own instance of -`VulnerableCode `_ -or connect to the public one. - -Authentication is provided using an API key that you can obtain by registering at -https://public.vulnerablecode.io/account/request_api_key/ :: - - VULNERABLECODE_URL=https://public.vulnerablecode.io/ - VULNERABLECODE_API_KEY=apikeyexample - EMAIL ----- @@ -187,11 +159,11 @@ to provide some progress about pipeline run execution. Default: ``INFO`` -The ``DEBUG`` value can be provided to this setting to see all ScanCode.io debug +The ``DEBUG`` value can be provided to this setting to see all DejaCode debug messages to help track down configuration issues for example. This mode can be enabled globally through the ``.env`` file:: - SCANCODEIO_LOG_LEVEL=DEBUG + DEJACODE_LOG_LEVEL=DEBUG .. _clamd-settings: @@ -222,6 +194,77 @@ default the ``US/Pacific`` time zone is used:: You can view a detailed list of time zones `here. `_ +.. _dejacode_settings_aboutcode_integrations: + +AboutCode integrations +====================== + +To **integrate DejaCode with other applications within the AboutCode stack**, +you have the flexibility to configure and set up integrations using the following +application settings. + +It's important to understand that employing application settings will make these +integrations **globally accessible across all Dataspaces** within your DejaCode +instance. + +Alternatively, if you wish to tailor the availability of these features to a specific +Dataspace, you can define and set those values directly within the :ref:`dataspace` +configuration. This can be done through the Dataspace admin UI, allowing you to scope +the availability of these integrations exclusively to the designated Dataspace. + +.. _dejacode_settings_scancodeio: + +SCANCODEIO +---------- + +Provide the URL and API key of your `ScanCode.io `_ +instance. + +.. code-block:: python + + SCANCODEIO_URL=https://your_scancodeio.url/ + SCANCODEIO_API_KEY=insert_your_api_key_here + +.. note:: + You have the option to define and set those settings directly on your Dataspace. + For detailed instructions, refer to :ref:`dejacode_dataspace_scancodeio`. + +.. _dejacode_settings_purldb: + +PURLDB +------ + +Provide the URL and API key of your `PurlDB `_ instance. + +.. code-block:: python + + PURLDB_URL=https://your-purldb.url/ + PURLDB_API_KEY=insert_your_api_key_here + +.. note:: + You have the option to define and set those settings directly on your Dataspace. + For detailed instructions, refer to :ref:`dejacode_dataspace_purldb`. + +.. _dejacode_settings_vulnerablecode: + +VULNERABLECODE +-------------- + +You can either run your own instance of +`VulnerableCode `_ +or connect to the public one https://public.vulnerablecode.io/. + +.. note:: Providing an API key is optional when using the public VulnerableCode instance. + +.. code-block:: python + + VULNERABLECODE_URL=https://public.vulnerablecode.io/ + VULNERABLECODE_API_KEY=insert_your_api_key_here + +.. note:: + You have the option to define and set those settings directly on your Dataspace. + For detailed instructions, refer to :ref:`dejacode_dataspace_vulnerablecode`. + LDAP Integration ================ diff --git a/docs/dataspace.rst b/docs/dataspace.rst index cc9d13f..e208595 100644 --- a/docs/dataspace.rst +++ b/docs/dataspace.rst @@ -1,7 +1,39 @@ +.. _dataspace: + ========= Dataspace ========= +The Dataspace serves as a pivotal mechanism within DejaCode, facilitating the +segregation of data for each organization while maintaining a unified storage +structure in the same database, schema, or table. +Within a given installation, multiple "Dataspace" organizations can be defined, +but there exists only one reference. + +This concept is a crucial element employed across DejaCode to effectively separate +**reference data provided by nexB** from the data utilized in a specific DJE +installation. +Essentially, it introduces the notion of a "tenant" within a DJE installation, +enabling the isolation of organization-specific and/or private records. +This segregation supports both multi-tenancy and the coexistence of nexB-provided +reference data and organization-specific or customized data. + +The key purposes of this separation include: + +1. **Orderly and Simplified Data Updates**: Facilitates smooth and streamlined updates + from nexB reference data, ensuring efficient data synchronization and exchange + across Dataspaces. +2. **Dataspace-Specific Customizations**: Allows for customization of Dataspace-specific + data, such as configurations for license tags or specific preferences, tailoring + the installation to the unique needs of each organization. +3. **Support for Multi-Tenancy**: Enables the sharing of the same DJE instance among + different organizations, each operating within its distinct Dataspace, + promoting multi-tenancy while maintaining data segregation. + +In summary, the Dataspace concept in DejaCode plays a pivotal role in maintaining data +integrity, enabling efficient updates, accommodating customization, and supporting +multi-tenancy for a diverse range of organizations within a DJE installation. + Setting up your License Library =============================== @@ -121,7 +153,7 @@ you do not need. Assigning Usage Policies to licenses ==================================== -There are more than **1,300 Licenses** in your License list from Reference Data, so +There are more than **2,000 Licenses** in your License list from Reference Data, so you will want to develop a strategy that works for your business requirements to assign Usage Policies efficiently. The two basic techniques are: @@ -130,7 +162,7 @@ assign Usage Policies efficiently. The two basic techniques are: 2. Mass update: Select a group of licenses on the **Browse License** form and use Mass Update to assign a Usage Policy to that group of licenses. -As an example of the first technique, locate and select the license with the Key +As an example of the first technique, locate and select the license with the key of **apache-2.0** in your list. Your organization probably already has a policy regarding this very common license; you may simply allow engineering to use components under that license or you may require additional review of how they @@ -278,6 +310,8 @@ The basic techniques to use in your own Dataspace are: **Set usage policy from licenses** option in the dropdown list in the lower right hand corner of the form and follow the prompts to complete that action. +.. _dejacode_dataspace_scancodeio: + Enable package scanning with your ScanCode.io server ==================================================== @@ -295,9 +329,16 @@ You can: * Download the scan results to a JSON-formatted file to integrate with other analysis and reporting tools. +Install and configure ScanCode.io +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + .. warning:: - The ScanCode.io server **should not be installed on the same server** (virtual or - physical) as the DejaCode one. + If you plan to run ScanCode.io **on the same server** (virtual or physical) as + the DejaCode instance, **ensure that the host machine has sufficient resources** + to handle both applications. + Also, you will have to provide custom network ports for the ScanCode.io application + as the ports 80 and 443 will be already used by the DejaCode application. + See https://scancodeio.readthedocs.io/en/latest/installation.html#use-alternative-http-ports 1. Install a ScanCode.io server following instructions at https://scancodeio.readthedocs.io/en/latest/installation.html @@ -321,29 +362,21 @@ You can: DejaCode instance: https://scancodeio.readthedocs.io/en/latest/command-line-interface.html#scanpipe-create-user-username -4. Set the ScanCode.io server URL and the API key in your local DejaCode settings file - ``.env``. - - .. code-block:: python - - SCANCODEIO_URL=https:/// - SCANCODEIO_API_KEY= - - **Restarting the services is required following any changes to .env:** +4. Set the ScanCode.io Server URL and API key in your Dataspace Configuration: - .. code-block:: bash - - docker compose restart web worker - -5. Enable package scanning on your Dataspace from the **Administration dashboard**. - - Select **Dataspaces** and open your Dataspace definition. - In the **Application Process Settings** section, check the - **Enable package scanning** option and save. + - Access your DejaCode web application **Administration dashboard**. + - Navigate to the **Dataspaces** section and select your Dataspace name. + - Within the **Application Process Settings** section, enable the + **Enable package scanning** option. + - Update the values for the **ScanCode.io URL** and **ScanCode.io API key** fields + located in the **Configuration** panel at the bottom of the form. + - Click the **Save** button. You can now access the **Scans** section from the **Tools** menu and request package scans from this view. +.. _dejacode_dataspace_purldb: + Enable PurlDB service ===================== @@ -359,47 +392,44 @@ and perform an asynchronous query of the PurlDB to find relevant data. You can: -* Browse and search from a list of over **14 millions Packages**. +* Browse and search from a list of over **21 millions Packages**. * Get extra information on your local Packages from the **"PurlDB" tab**. -* Create local Packages automatically from entries found in the PurlDB. +* **Create local Packages automatically** from entries found in the PurlDB. * Enhance the **Global search** results with Packages from the PurlDB. * Check for **new Package versions** from your Products inventory -1. Get in touch with nexB to request your credentials for the **PurlDB** service. - -2. Set those credentials in your local DejaCode configuration file ``.env``. - - See :doc:`/application-settings` for more details on the configuration system. +PurlDB service +^^^^^^^^^^^^^^ -.. code-block:: python +A public instance of **PurlDB** is available at +https://public.purldb.io/api/packages/ - PURLDB_URL=https://purldb.url/ - PURLDB_USER=PROVIDED_BY_NEXB - PURLDB_PASSWORD=PROVIDED_BY_NEXB +Alternatively, you have the option to run your instance of PurlDB by +following the documentation provided at https://purldb.readthedocs.io/ +Set the PurlDB Server URL and API key in your Dataspace Configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -**Restarting the services is required following any changes to .env:** - -.. code-block:: bash - - docker compose restart web worker - -3. Enable **PurlDB access** on your Dataspace from the **Administration dashboard**. - - Select **Dataspaces** and open your Dataspace definition. - In the **Application Process Settings** section, check the **Enable PurlDB access** - option and save. + - Access your DejaCode web application **Administration dashboard**. + - Navigate to the **Dataspaces** section and select your Dataspace name. + - Within the **Application Process Settings** section, enable the + **Enable PurlDB access** option. + - Update the values for the **PurlDB URL** and **PurlDB API key** fields + located in the **Configuration** panel at the bottom of the form. + - Click the **Save** button. You can now access the **PurlDB** section from the **Tools** menu and browse package from this view. +.. _dejacode_dataspace_vulnerablecode: + Enable VulnerableCodeDB service =============================== DejaCode integration with the **VulnerableCodeDB** service authorizes DejaCode to access -the VulnerableCodeDB using a Package URL (purl) to determine if there are any reported -vulnerabilities for a specific Package and return the Vulnerability ID and related URLs -to a Vulnerabilities tab in the Package details user view. +the VulnerableCodeDB using a Package URL (purl) to determine if there are any +**reported vulnerabilities for a specific Package** and return the Vulnerability ID +and related URLs to a **Vulnerabilities tab** in the **Package details** user view. DejaCode displays a Vulnerability icon next to the Package identifier in the user view list, and also in any Product Inventory list using that Package. @@ -414,31 +444,25 @@ You can: * Review and edit your Product Package assignments to record your analysis, the actions you have taken, and the current status of your usage of that Package. -1. Get in touch with nexB to request your credentials for the **VulnerableCodeDB** - service. - -2. Set those credentials in your local DejaCode configuration file ``.env``. - - See :doc:`/application-settings` for more details on the configuration system. - -.. code-block:: python - - VULNERABLECODE_URL=https://vulnerablecodedb.url/ - VULNERABLECODE_API_KEY=PROVIDED_BY_NEXB - -**Restarting the services is required following any changes to .env:** - -.. code-block:: bash +VulnerableCodeDB service +^^^^^^^^^^^^^^^^^^^^^^^^ - docker compose restart web worker +A public instance of **VulnerableCodeDB** is available at +https://public.vulnerablecode.io/api/ +Alternatively, you have the option to run your instance of VulnerableCodeDB by +following the documentation provided at https://vulnerablecode.readthedocs.io/ -3. Enable **VulnerableCodeDB access** on your Dataspace from the - **Administration dashboard**. +Set the VulnerableCodeDB Server URL and API key in your Dataspace Configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - Select **Dataspaces** and open your Dataspace definition. - In the **Application Process Settings** section, - check the **Enable VulnerableCodeDB access** option and save. + - Access your DejaCode web application **Administration dashboard**. + - Navigate to the **Dataspaces** section and select your Dataspace name. + - Within the **Application Process Settings** section, enable the + **Enable VulnerableCodeDB access** option. + - Update the values for the **VulnerableCode URL** and **VulnerableCode API key** + fields located in the **Configuration** panel at the bottom of the form. + - Click the **Save** button. You can now see Vulnerabilities in the Packages user view. The availability of the services can be checked by clicking on your user name in the diff --git a/docs/index.rst b/docs/index.rst index ce79b27..7f9fccf 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -9,8 +9,8 @@ Welcome to the very start of your DejaCode journey! :caption: Getting Started installation - application-settings dataspace + application-settings .. toctree:: :maxdepth: 1 diff --git a/docs/installation.rst b/docs/installation.rst index 1298740..728ac6c 100644 --- a/docs/installation.rst +++ b/docs/installation.rst @@ -33,7 +33,7 @@ Refer to Docker's documentation for the best installation path for your system: Get Docker -2. Build the Image +2. Build the image ------------------ DejaCode comes with the necessary ``Dockerfile`` and ``docker-compose.yml`` files to @@ -60,7 +60,7 @@ Create an **environment file**, and **build the Docker image** with:: cd dejacode && make envfile docker compose build -3. Run the App +3. Run the app -------------- To **run the DejaCode images as containers**, use the following command:: @@ -107,23 +107,50 @@ Use these credentials to access the application. For example, with Docker configured for 8 CPUs, allocate a minimum of 8 GB of memory. +5. Dataspace setup and AboutCode integrations +--------------------------------------------- + +Upon the initialization of the DejaCode application, the ``nexB`` reference +:ref:`dataspace` is created with a **default set of data**, including license and +organization libraries. + +Additionally, **AboutCode integrations are pre-configured** to connect to +**public instances** of the following AboutCode applications: + +- **ScanCode.io**: Facilitates package scanning. + Refer to :ref:`dejacode_dataspace_scancodeio`. +- **PurlDB**: Provides access to a database of scanned packages. + Refer to :ref:`dejacode_dataspace_purldb`. +- **VulnerableCodeDB**: Enables access to a database containing information on package + vulnerabilities. + Refer to :ref:`dejacode_dataspace_vulnerablecode`. + +.. warning:: + In the scenario of **deploying DejaCode as an enterprise service** within your + organization, it is **strongly recommended to review these configurations**. + Consideration should be given to **running your own instances** of these + applications to ensure that **sensitive or private data** is not inadvertently + submitted to public services. This strategic approach helps to safeguard + organizational data and privacy during package scanning and vulnerability + assessments. + Hardware requirements ===================== The minimum hardware/system requirements for running DejaCode as an enterprise server are: -+-----------+----------------------------------------------------------------+ -| Item | Minimum | -+===========+================================================================+ -| Processor | Modern X86 64 bits Intel Quad Core or better, or equivalent | -+-----------+----------------------------------------------------------------+ -| Memory | 64 GB or more (ECC preferred) | -+-----------+----------------------------------------------------------------+ -| Disk | 2 * 500GB SDD in RAID mirror setup (enterprise disk preferred) | -+-----------+----------------------------------------------------------------+ -| OS | Ubuntu 22.04 LTS 64-bit server clean installation | -+-----------+----------------------------------------------------------------+ ++-----------+------------------------------------------------------------------+ +| Item | Minimum | ++===========+==================================================================+ +| Processor | Modern X86 64 bit Multi Core, with at least **4 physical cores** | ++-----------+------------------------------------------------------------------+ +| Memory | **64 GB** or more (ECC preferred) | ++-----------+------------------------------------------------------------------+ +| Disk | 2 * 500GB SDD in RAID mirror setup (enterprise disk preferred) | ++-----------+------------------------------------------------------------------+ +| OS | **Ubuntu 22.04 LTS 64-bit** server clean installation | ++-----------+------------------------------------------------------------------+ .. _local_development_installation: