Architecture

IDP is built on an extensible microservices architecture. It leverages the orchestration capabilities of the widely popular Jenkins and provides a layer of abstraction over it through Angular 5 based responsive web front end and Spring Boot based RESTful services.

IDP satisfies the 15 factors guidelines from PIVOTAL, for a highly scalable and resilient Cloud App.

Below diagram depicts the high level architecture of IDP.

The table below explains the key components of IDP.

Module/File	Description
UI	This component is built on Angular 5 and TypeScript and supports seamless rendering in Mobile devices and tablets. All external API calls are routed through Zuul Filter from Netflix. Through Pre/Route/Post, Zuul will make sure only authenticated users are allowed to make API calls and for OAuth2 token renewal
Cloud Config Service	This component maintains all the config entries of IDP Components. This way, all configurations are externalized. Config files can be configured per environment and per client which gives greater flexibility When a service is booting up, it fetches the appropriate config files from this Service
Discovery Service	Eureka from Netflix is used. Services get registered following Self-Registration Pattern. The service registry provides a management API and a query API for registering/de-registering.
Auth Service	Auth Service's purpose is to authenticate users and manage OAuth2 tokens. Access token is generally short lived and refresh token would be valid for a long time. This supports Token Renewal and protects from MITM attacks. JSON Web Token(JWT) ensures the user’s session is handled in a stateless scalable way
IDP Rest	This service responsible for Application, Pipeline, User and Dashboard Management. It uses Swagger for API Documentation. Resources are protected using OAuth2 API First approach and Entities are defined using Google’s GSON. Uses Spring-boot for automating and simplifying and bootstrapping boiler plate.
IDP Orchestrator	This Java based component provides a layer of abstraction to decouple the underlying orchestration engine such as Jenkins, from the IDP REST component. The component subscribes to Kafka topics for processing of various pipeline management requests for the orchestration engine.
IDP Jenkins Connector	This Java based component implements pipeline, slave and user management functions for Jenkins OSS.Orchestrator component delegates the requests to respective Connector component depending on the orchestration engine defined in the Kafka message.
DSL	DSLs are the core for automating the Jenkins Job Creation. This is a hybrid approach for creating Jenkins jobs using Groovy. Seed Job gets created through REST service and this job in turn creates the child Jobs (SCM, Build, Deploy etc) . When the Seed Job is re-run, DSL is intelligent enough to identify the differences in configuration and would automatically create/update/delete job/configurations
Scheduler	This Java based component is for timed execution of pipelines.
Metrics Dashboard	Grafana is used for visualising the data collected by dashboard utilities (Java utilities that are invoked at the end of every job in the pipeline) through the dashboard "put" services. The "get" type of dashboard services query this data and offer it to Grafana for rendering and visualisation.
Message Queue	Apache Kafka is used for asynchronous communication between services
Continuous deployment	This components provides seamless ability to do cloud deployments. This is not yet part of the repository and will soon be released.
Subscription	This component maintains the license information for the platform and to enable subscription based access. This is not yet part of the repository and will soon be released.
Tracing	Tracing of REST Service calls is done using ZipKin, a distributed tracing system. This gathers timing data that is required to troubleshoot latency. This component matches, both the collection and lookup of this data. This is not yet part of the repository and will soon be released.
Monitoring	IDP components' monitoring is done using Prometheus. The Prometheus agent extracts logs and pushes to Elastic Search from which data is data is propagated to Grafana. This is not yet part of the repository and will soon be released.