Pluralscan is a source code analysis software, that's combine the best features of open sources security tools into a single solution.

• Software Composition Analysis aka SCA
• Static Application Security Testing aka SAST
• Dynamic Application Security Testing aka DAST
• Code Quality Analysis with Linter's
• Active security testing with Fuzzing Tools
• Prevents unwanted law complication by checking open source license compliance
• Benchmark results provided by different tools
• ...

WARNING

Pluralscan should be currently considered as a POC/POW project that try to demonstrate how to realize a complexe business oriented software in Python by following Domain Driven Design concepts.

Features

• Fetch open-source projects from various locations (Git, Github, Gitlab, Disk...).
• Fetch software packages built with various packaging systems (pip, poetry, npm, pip, cargo, go...)
• Plan code analysis batch on a package with various analyzers (Roslyn, Sonar, Security Code Scan...).
• Centralize and persist analysis reports into a generic business model representation.
• Monitor and provide assistance to reduce technical debt.

Team Shared Vision

Domain Model

Ongoing work, so many changes should be expected.

Projects Management

Packages Management Domain

Security Tools Domain

Scan Proces Domain

Roadmap from June 2022 to December 2022

Getting started with Docker

docker-compose

Setup a complete stack with MongoDB, Redis and Pluralscan. (Known issue with analyzer execution, fix comming soon)

docker-compose up

Remove containers.

docker-compose down

Check Redis

• Navigate to http://localhost:8001
• Accept RedisInsight license.
• Login with the password defined inside .docker.env

Check Mongo

• Navigate to http://localhost:

Check Apache Solr

• Navigate to http://localhost:8983

Stack Overview

Core Developement

• Python 3.10 as language and runtime environment.
• asyncio to declare coroutines and execute concurrent code
• poetry for packaging and dependency management.
• pytest framework for testing.
• pytest-cov
• mypy for static type checking.

Recommendation

• Use pathlib for handling cross-platform file path.

Backend Web Application (API)

• Python 3.10* as language and runtime environment.
• FastApi is used as web framework used for API and serving SPA.
• asyncio is used to declare coroutines and execute concurrent code.
• esdbclient as gRPC client for EventStoreDB.
• Celery
• python-kafka
• grpcio
• grpcio-tools
• pymongo
• sse-starlette for performs Server Sent Event.
• uvicorn
• pydantic

Front-end Web Application

• NodeJS 16.13.0 as runtime environment.
• Typescript as main language.
• Svelte 3 as front-end framework.
• Carbon Design System for Svelte as design framework.
• jest as testing framework.
• ts-jest for writing tests in Typescript.
• svelte-jester for precompile svelte components before importing them in to tests.
• tailwindcss as utility css framework.
• PouchDB used to sync data from CouchDB Server on client device for offline usage.

Commandline CLI Application

• Python 3.10* as language and runtime environment
• asyncio to declare coroutines and execute concurrent code

Technologies Overview

Data Management & Persistence

• InMemory - only for tests purpose
• MongoDB
• Redis
• ElasticSearch
• Apache CouchDB used to persist data that's can be sync on client for offline usage.
• EventStoreDB
• Neo4j
• Apache Solr
• debezium

Messaging

• RabbitMQ
• Apache Kafka

Containerization

• Docker
• Kubernetes

Cloud Infrastructure

• Azure App Service
• Azure Container Registry

Static Code Analyzis Tools

Security Analysis

• KICS - Keeping Infrastructure as Code Secure [Cloud]
• Security Code Scan [C# | VB#]
• OWASP Dependency Check

Linters

• Staticcheck [Go]
• pylint [Python]
• cpplint [C | C++]
• Roslynator [C# | VB#]
• SQLFluff [SQL]
• Checkstyle [Java]
• ESLint [Javascript | Typescript]
• prettier [Javascript | Typescript]
• ktlint [Kotlin]
• php [PHP]
• rubucop [Ruby]
• clippy [Rust]
• SwiftLint [Swift]

Development

IDE

Visual Studio Code

Commandline

TODO.

Web Application

Pluralscan TS Api Client

cd pluralscan-api-client
npm install
npm run build

Svelte (Single Page Application)

Install

cd pluralscan-svelte
npm install

Run Svelte Rollup Dev Server (Hot Reloading)

cd pluralscan-svelte
npm run dev

Build and upgrade front-end libs (until npm)

Powershell

cd pluralscan-svelte
npm run build

The build process will output directly into the API project.

Check

To verifiy if the project is error free, you can use the CLI tool svelte-check. It acts like an editor asking for errors against all of .svelte files.

npx svelte-check

Debug

From VS Code Debug View, run the "Debug FastApi" profile for start a dev web service with debuging.

Tests

Usecases

• Shedule Package Scan
• Scan Package

Scan Package

Coverage

https://coverage.readthedocs.io/

Code coverage analysis with HTML report

py -m coverage html --skip-empty
cd htmlcov

References

Books

• Domain-Driven Design: Tackling Complexity in the Heart of Software by Eric Evans
• Implementing Domain-Driven Design by Vaughn Vernon
• Patterns, Principles, and Practices of Domain-Driven Design by Scott Millett
• Clean Architecture: A Craftsman's Guide to Software Structure and Design by Robert C. Martin aka "Uncle Bob"
• Living Documentation: Continuous Knowledge Sharing By Design by Cyrille Martraire

Python

• PEP 563 – Postponed Evaluation of Annotations
• ABC - Abstract Base Class

Microservices

• Microservices.io is edited by Chris Richardson an experienced software architect.

DDD

Methodology

• The Bounded Context Canvas

Events

• Handle events consistancy - Committing before dispatching
• Domain-Driven Design: Domain Events and Integration Events in .Net
• Should you publish Domain Events or Integration Events?

Patterns

CQRS

• What is CQRS?

Usefull resources

• SVG Repo
• PNG Repo