Welcome to the repository of our .NET Web API, a specialized middleware application designed to streamline and optimize data integration processes for Microsoft Dynamics 365 (MS D365). This application plays a crucial role in transforming and loading data into connected MS D365 environments, facilitating seamless data management and synchronization.
The primary objective of this application is to act as an intermediary layer that transforms and loads data efficiently from a centralized data repository to the targeted MS D365 entity. It serves as a pivotal link in the data replication process, ensuring that any changes made to the data in the central repository are accurately reflected in the MS D365 environment. By doing so, it supports the continuity and integrity of data across systems.
Our application's API is consumed by a Data Replication application, which is responsible for pushing any changes in the data from the centralized repository to our middleware. This process ensures that the middleware application can perform its role in transforming and loading the data into the targeted MS D365 entity effectively.
In addition to data transformation and loading capabilities, our application provides an MS D365 metadata layer API. This API allows for the verification of certain items within the MS D365 environment, adding an extra layer of integrity and validation to the data integration process.
The .NET API layer of our application is deployed on OpenShift using Deployment Configurations. This deployment strategy ensures high availability, scalability, and security for the application, enabling it to handle data integration tasks efficiently and reliably.
To set up the middleware application for local development, please follow the steps outlined below. These steps will guide you through the process of setting up the development environment, cloning the repository, and running the application on your local machine.
- Microsoft Visual Studio 17.6 or above installed on your local machine.
-
Clone the Repository
Begin by cloning the repository to your local machine. Open your terminal or command prompt and run the following command:
git clone https://github.com/bcgov/ECC-IOSAS-INSTITUTE-API
-
Create
appsettings.jsonNavigate to the
{root}/SchoolInformationIntegrationdirectory within the cloned repository. Here, create a new file namedappsettings.json. -
Configure Application Settings
Access the GitHub repository's web interface and navigate to the following location to find the development environment configuration:
Settings > Security > Secrets and variables > Actions > Variables
Look for a variable named
APP_CONFIG_DEV. Copy the content of this variable and paste it into theappsettings.jsonfile you created in the previous step. -
Open the Solution in Visual Studio
Navigate to the
{repo_root}/ECC.Institute.CRM.IntegrationAPI.slnfile. Double-click on this file to open the solution in Microsoft Visual Studio. -
Run the Application
Inside Visual Studio, initiate the application by running it. Visual Studio automates the build process and launches the application.
-
Access Swagger UI
Once the application is running, Visual Studio will automatically open the default web browser and navigate to the Swagger API documentation page at:
https://localhost:7251/swagger/index.htmlThis page provides an interactive interface to test and explore the API endpoints offered by the middleware application.
This section outlines the steps for building and deploying the .NET Web API, which serves as a middleware for MS Dynamics 365, using OpenShift. The process utilizes a Makefile to automate tasks such as building the Docker image, creating certificates, and deploying the application to OpenShift.
- Access to an OpenShift cluster with appropriate permissions.
- The
ocCLI tool installed and configured to communicate with your OpenShift cluster. - A clone of the repository containing the
Makefile.
-
How to setup
appsettings.jsonSample
appsetings.json:{ "D365AppSettings": [ { "Name": "IOSAS", "BaseUrl": "https://app-env-url.crm3.dynamics.com", "WebApiUrl": "https://app-env-url.crm3.dynamics.com/api/data/v9.2/", "TenantId": "<id>", "ClientId": "<client-id>", "ClientSecret": "<secret>", "APIVersion": "v9.2", "AllowedFileUplaodTypes": "jpg,jpeg,pdf,png,doc,docx,heic,xls,xlsx,txt" }, { "Name": "ISFS", "BaseUrl": "https://app-env-url.crm3.dynamics.com", "WebApiUrl": "https://app-env-url.crm3.dynamics.com/api/data/v9.2/", "TenantId": "<tenant-id>", "ClientId": "<client-id>", "ClientSecret": "<secret>", "APIVersion": "v9.2", "AllowedFileUplaodTypes": "jpg,jpeg,pdf,png,doc,docx,heic,xls,xlsx,txt" } ], "Logging": { "LogLevel": { "Default": "Information", "Microsoft.AspNetCore": "Warning" } }, "AllowedHosts": "*" }Please get appropiate values for target env from MS Power Portal admistrator and saved as github variables in jsonstring non pretty formatting
Settings > Security > Secrets and variables > Actions > Variables > APP_CONFIG_{ENV_NAME}Note: Currently
APP_CONFIG_DEVis only configured with proper values. -
Set Up Environment Variables: Ensure that your
.envfile located in the project root directory contains all the necessary environment variables. This file is automatically included and exported by theMakefile. -
Build the API Container Image: Execute the following command to initiate the build process for the API container image. This step uses the
build-apitarget in theMakefileand relies on theopenshift/docker-build.ymltemplate for configurations.make oc-build-api
This command processes the Docker build template with the current git commit information and other environment variables, then starts a build in the OpenShift namespace defined by
BUILD_NAMESPACE.
-
Initialize API Configuration: Before deploying, ensure that the API's configuration is properly set up in a configMap within OpenShift. This is done using the
init-apitarget, which reads theAPP_CONFIGenvironment variable and creates anappsettings.jsonwithin the API container.make init-api
-
Deploy the API Container: Use the following command to deploy the API container using the deployment configuration. This step also handles the creation of a service certificate for the .NET API server.
make oc-deploy-api
The deployment uses the OpenShift
api-deploy.ymltemplate file and updates the deployment configuration with the latest image built in the previous step.
After successful deployment, the API will be accessible within the OpenShift cluster. You can create a port-forwarding tunnel to access the Swagger UI locally using the d365-api-tunnel target:
make d365-api-tunnelThis command sets up a port-forwarding tunnel to the service, allowing you to access the Swagger UI at https://localhost:7251/swagger/index.html (or a different port if specified).
To manage deployments, use the oc commands for rolling updates, scaling, or rollback as needed. For custom rollout and wait strategies, refer to the rollout_and_wait function defined in the Makefile.
- The
Makefileincludes targets for certificate creation (api-create-certificate), API configuration initialization (init-api), and deployment configuration updates (update-dc-api). - Environment-specific configurations are managed using conditional statements in the
Makefile, allowing for flexible deployments across development, test, and production environments.
By following these steps, you can build and deploy the .NET Web API middleware for MS Dynamics 365 on OpenShift, leveraging the automation provided by the Makefile.
This section outlines the Continuous Integration and Continuous Deployment (CI/CD) process for a web application, managed through GitHub Actions and deployed on OpenShift. The CI/CD pipeline is designed to automate the build and deployment process, ensuring that every change is seamlessly integrated and deployed to the development environment, with options for manual deployments and promotions.
The CI/CD pipeline utilizes GitHub Actions to automate the building and deploying of the application. It is configured to respond to push events to specific branches and manual triggers, with each OpenShift build tagged with the Git commit SHA for traceability.
- Development Workflow: The
devworkflow is triggered on a push event to themainbranch. It automatically builds and deploys the entire application to the development environment. This process is defined under.github/workflows/dev-api.yml.
API Deployment: To manually build and deploy the API component in the development environment, the dev-api workflow is used. This can be triggered manually through GitHub Actions and is defined in .github/workflows/dev-api.yml.
- Promotion Workflow: The
dev-anyworkflow facilitates the promotion of a build to any target environment. It can use either a specific build tag (commit SHA) if the Git branch head points to the same commit SHA, or it can use the latest tag if the Git branch head points to a SHA whose build tag is not available. This workflow is defined under.github/workflows/dev-any.ymland must be triggered manually.
Each build deployed to OpenShift is tagged with the Git commit SHA. This tagging strategy ensures traceability and allows for precise promotions between environments.
To trigger manual deployments or promotions:
- Go to the repository on GitHub.
- Navigate to the Actions tab.
- Select the desired workflow (e.g.,
dev-api,dev-any). - Click the Run workflow dropdown.
- Choose the branch where the workflow is defined.
- Click Run workflow to start the process.
This CI/CD pipeline automates the process of integrating, building, and deploying applications to various environments. It leverages GitHub Actions for both automatic and manual workflows, ensuring that the application remains up-to-date and stable across all development stages.
Please find detailed documentaion and static swagger-json
After running application locally we can access live swgger docuemenation on
https://localhost:7251/swagger/index.html
We can access the api swagger deployed in openshift after port forwarding
To report bugs/issues/features requests, please file an issue.
Copyright 2020 Province of British Columbia
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
