Skip to content
This repository has been archived by the owner on Oct 12, 2023. It is now read-only.

Developer Reference Guide

Jump to bottom
Avneet Singh edited this page Nov 12, 2018 · 60 revisions

This article provides a list of frequently used tasks in the day-to-day development process in the Remote monitoring solution.

For troubleshooting help to identify and resolve commonly encountered issues, please visit our Developer Troubleshooting Guide

Contents

Managing a Solution

Managing Deployed Containers

Deployment

Local Development

Customization

For other customization options, please read the Customize the solution tutorial on docs.microsoft.com

Managing a Solution

Add a user to a solution

In order to allow accounts that are not part of your subscription to have access, you must give them access through the Azure portal under your Azure Active Directory Application Settings.

Connect a physical device

For detailed instructions on how to connect a physical devices, please read the Connect physical device tutorial on docs.microsoft.com

Managing Deployed Containers

SSH into a VM

Prerequisites

  • You will need to enable SSH for your Network Security Group. Go to your resource group in the Azure Portal > Network Security Group > Inbound security rules > SSH. Change "Action" to "Allow" and save.
  • Sign into the VM (you may need to create a username/password)
  • Find your VM's SSH address from the Azure Portal.
    • Navigate to the VM in the Azure Portal (portal.azure.com > Resource Groups > Select Resource Group for your PCS > Select Virtual Machine)
    • You can also select "Connect" from your VM's portal page which will display your ssh command.

Sign into a VM

  • To sign into a VM using bash, use your username followed by the IP address of the VM and enter your sign in credentials. 
    ssh myusername@12.34.567.890

Disable authentication

In order to test HTTP requests from tools like Postman you will need to disable authentication in the application.

  1. Open a console and SSH into your Azure VM (ssh {user}@{host}) using the password chosen during deployment.
    • You can also find the VM connection information in the Azure Portal at
      portal.azure.com > Resource Groups > Select Resource Group for your PCS > Select Virtual Machine > Connect
  2. Restart the service by running sudo /app/start.sh --unsafe. If you want to restart again in safe mode, run sudo /app/start.sh

Common Docker Commands

From the VM you can accomplish many common tasks with the following commands. For more information see the Docker command line documentation

Pre-requisites

List all images that are locally stored with the Docker engine

docker images

List all running containers to check status of a service

docker ps -a

Example output:

CONTAINER ID        IMAGE                                            COMMAND                  CREATED             STATUS              PORTS                                      NAMES
b84eeb26c07a        azureiotpcs/remote-monitoring-nginx:latest       "/bin/bash /app/ru..."   12 minutes ago      Up 12 minutes       0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp   app_reverseproxy_1
c66af1abbafb        azureiotpcs/pcs-remote-monitoring-webui:latest   "/bin/sh /app/run.sh"    12 minutes ago      Up 12 minutes       80/tcp, 443/tcp                            app_webui_1
8cb29cd3902d        azureiotpcs/telemetry-agent-dotnet:latest        "/bin/bash /app/ru..."   13 minutes ago      Up 13 minutes                                                  app_telemetryagent_1
d96b506acf7a        azureiotpcs/pcs-config-dotnet:latest             "/bin/bash /app/ru..."   13 minutes ago      Up 13 minutes                                                  app_config_1
6c6802aacd91        azureiotpcs/device-simulation-dotnet:latest      "/bin/bash /app/ru..."   13 minutes ago      Up 13 minutes                                                  app_devicesimulation_1
47ec9f594920        azureiotpcs/telemetry-dotnet:latest              "/bin/bash /app/ru..."   13 minutes ago      Up 13 minutes                                                  app_telemetry_1
3531b19916be        azureiotpcs/pcs-storage-adapter-dotnet:latest    "/bin/bash /app/ru..."   13 minutes ago      Up 13 minutes                                                  app_storageadapter_1
d84d888d55e3        azureiotpcs/pcs-auth-dotnet                      "/bin/bash /app/ru..."   13 minutes ago      Up 13 minutes                                                  app_auth_1
67dbea599512        azureiotpcs/iothub-manager-dotnet                "/bin/bash /app/ru..."   14 minutes ago      Up 14 minutes                                                  app_iothubmanager_1

Get the logs for a running container

docker logs {Container ID}
docker logs --tail 100 {Container ID} (lists last 100 lines of log file)

Pull the latest image from Docker Hub

docker pull {docker-hub-user}/{docker-hub-container}:latest
Ex: docker pull azureiotpcs/iothub-manager-dotnet:latest

Stop a running container

docker stop {container-id}

Start simulations from VM

  1. Sign into the vm with the instructions above

  2. navigate to the app folder from the root directory: cd /app/

  3. run the simulation script from the terminal (root access required)

    sudo ./simulate.sh

    This will create the default simulations with the following command

    POST https://localhost/devicesimulation/v1/simulations?template=default

Pull the logs to a local machine for a container

  1. Open a bash terminal.
  2. SSH into the VM using the instructions above in "SSH into a VM".
  3. Find your container name or id with docker ps.
  4. Write the logs for a container to disk: docker logs <containerid> >& simlog.log
  5. Ensure the log data is present: sudo nano container.log
  6. Open a second bash terminal on your local machine.
  7. Copy the file local: scp <username>@<vm ip>:container.log c:/temp/container.log

Deployment

Deploy a PCS from the command line

  1. Clone the pcs-cli repo 
    git clone https://github.com/Azure/pcs-cli.git
  2. Follow the instructions in the README.md for How to use the CLI

Replace a running container with the testing images from PCS Docker Hub

These instructions are for pulling the testing images (the images in development) from the PCS Docker Hub repository from https://hub.docker.com/r/azureiotpcs/, to add your own images, please see Deploying containers from custom docker-hub account

Basic Deployments

  1. SSH into your deployment's VM using the instructions above
  2. navigate to the app folder from the root directory: cd /app
  3. Change to the /app directory: 
    cd /app
  4. Edit the docker-compose.yml file: 
    sudo nano docker-compose.yml
  5. Change your containers to use the :testing tag, e.g. 
    services:
  reverseproxy:
    image: azureiotpcs/remote-monitoring-nginx:testing
...
  webui:
    image: azureiotpcs/pcs-remote-monitoring-webui:testing
...
  auth:
    image: azureiotpcs/pcs-auth-dotnet:testing
  6. Save the file: Ctrl-X, Ctrl-Y
  7. Execute the start.sh command. sudo ./start.sh

To update all containers and environment variables

To reset all of the environment variables needed for your containers and redeploy the services based on the images specified in dockerhub (defined in your docker-compose.yml file).

run the start.sh script from the command line (root access required) sudo ./start.sh

Standard

Prerequisites

  1. Clone the pcs-cli repo
  2. Install az
  3. Install kubectl
  4. Access to the Subscription that created the deployment

One-time setup to create a container service for Kubernetes

  1. az login
  2. az account set --subscriptions {your-azure-subscription-id}
  3. Output .ssh credentials for Kubernetes
    az acs create -n {myClusterName} -d {myDNSPrefix} -g {resouceGroup} -t kubernetes --generate-ssh-keys
  4. Set credentials for Kubernetes
    az acs kubernetes get-credentials -g {myResorceGroupName} -n {myClusterName} --ssh-key-file ~\.ssh\{path-to-ssh-key}

Deploy Docker images through Kubernetes

Verify that you have access with kubectl get nodes

  1. kubectl create -f .\remotemonitoring\scripts\nginx-ingress-controller.yaml
  2. Go to your resource group on portal.azure.com and set up friendly DNS name for Public IP address that got created in (One-time setup to create a container service for Kubernetes)[#one-time-setup-to-create-a-container-service-for-kubernetes]. It will start with {myClusterName}. To confirm, match the IP address with "LoadBalancer Ingress" by running kubectl describe svc nginx-ingress
  3. Add actual values in the ConfigMap section in file all-in-one.yaml and deployment-configmap.yaml. Values to replace will be of format "{...}". Some examples below.
    • {DNS} with value from step 2
    • {IoT Hub connection string}
    • {DocumentDB connection string}
  4. kubectl create -f .\remotemonitoring\scripts\all-in-one.yaml

Important

If your account doesn't have the Azure Active Directory (AAD) and subscription permissions to create a service principal, then the command generates an error similar to Insufficient privileges to complete the operation.
Also, when using --generate-ssh-keys, if one key already exists at ~/.ssh/id_rsa then it will be used.

View the Kubernetes Dashboard for Standard Deployments

To view Kubernetes dashboard, run the following command, which will start a local web proxy for your cluster (it will start a local server at http://127.0.0.1:8001/ui):

az acs kubernetes browse -g {myResourceGroupName} -n {myClusterName} --ssh-key-file {path to ssh file}

Deploying containers from custom docker-hub account

In order to deploy your own changes to a pcs deployment, you will need to edit the build and deployment scripts in {repo}/scripts/docker to upload the containers to a docker-hub account

Prerequisites

  1. Clone the pcs-cli repo
  2. Create a Docker Hub account
  3. Create a Docker Hub repo for each microservice that you would like to redepoly. For example, if you would like to make changes to the Iot Hub Manager .NET project, you would create a docker hub repo for {your-username}/iothub-manager-dotnet

Instructions

Update project with custom changes

  1. Make changes to whatever microservices you would like
  2. When you are satisfied with your results, increment the version number on your project
    • Version file located in...
      • {repo}/version for .NET projects
      • build.sbt file for java projects

Modify the docker scripts with your docker hub repo info

In every repository there is a /scripts/docker folder with scripts to build, run, and publish Docker images. For the repositories that you have made changes to, update this script with your own Docker Hub repository info:

  • Update the following lines with your own information in the files listed below:
    DOCKER_IMAGE="{your-docker-hub-username/{your-docker-hub-repo}"

    • build
    • build.cmd
    • publish
    • publish.cmd
    • run
    • run.cmd

    image: {your-docker-hub-username/{your-docker-hub-repo}

    • docker-compose.yml

Generate new Docker images and push to Docker Hub

For each project that you have made changes to you will need to publish new Docker images to Docker Hub using your docker-hub account.

  1. cd {your-repo}/scripts/docker

  2. Build Docker image \

    • Windows cmd prompt:build
    • bash: ./build

  3. .NET repos only - Tag with "latest"
    docker tag {image-id-from-build} {your-docker-hub-username/{your-docker-hub-repo}:latest

  4. Publish docker image:

    • Windows cmd prompt:publish
    • bash: ./publish

  5. In the pcs-cli project, locate the file all-in-one.yaml

  6. For each container that you would like to deploy with custom changes, also update your Docker Hub repo info located in all-in-one.yaml for the changed microservice

  7. Update any references to Docker Hub in the individual service scripts in the remotemonitoring/individual folder that you would like to update

  • For example in iothub-manager.yaml, modify the following line to your own Docker Hub image repo
  • image: azureiotpcs/iothub-manager-dotnet:latest -> image: {your-docker-hub-username/{your-docker-hub-repo}:latest

Follow these steps to update containers

Update a deployed container via manual copy

If you would like to push your changes without creating a Docker Hub account you can follow these steps.
Note: all of the setup scripts on the VM reply on docker hub repositories, so you will need to manually start the service every time you wish to update.

  1. Generate a new docker image for your modified project
  2. cd {your-repo}/scripts/docker
  3. Build Docker image \
    • Windows cmd prompt:build
    • bash: ./build
  4. Copy your docker image to your VM using SCP. (more info on SCP to an Azure VM here)
    1. Get image id with docker images 
    docker save <image-id> > <path-to-output-file>/savedImage.tar
scp savedImage.tar <vm-username>@<vm-ip>:savedImage.tar
  5. SSH into the VM
  6. from vm, check to make sure simulation.tar exists with ls
  7. docker load < simulation.tar
  8. Find the new image id with docker images
  9. Edit the docker-compose.yml file to replace the docker-hub image with your image ID
    1. sudo nano /app/docker-compose.yml
    2. modify the desired container with your container id
      devicesimulation:
image: <your-image-id-here>
depends_on:
  - storageadapter
environment:
  - PCS_IOTHUB_CONNSTRING
  - PCS_STORAGEADAPTER_WEBSERVICE_URL=http://storageadapter:9022/v1
  - PCS_AUTH_ISSUER
  - PCS_AUTH_AUDIENCE
  - PCS_AUTH_REQUIRED
  - PCS_CORS_WHITELIST
  - PCS_APPLICATION_SECRET
#volumes:
#  - ./sample-volume:/app/data:ro
  10. run the start script to start the new container with the proper environment variables using the newly edited docker-compose.yml file. 
    ./start.sh
  11. verify your new image is running with docker ps -a

Local Development

Running the PCS Web UI against deployed services

If you have your pcs up and running and would like to customize the UI, you can start implementing against deployed services by updating the endpoint listed in the .env file. You will also need to disable authentication.

REACT_APP_BASE_SERVICE_URL={your-endpoint-url-from-pcs-cli}

More info in the README.MD for the pcs-remote-monitoring-webui

Running all PCS microservices locally

Running services with Docker

Please follow the detailed instructions for how to run all microservices locally inside a Docker container

Running services locally with Visual Studio

Please follow the detailed instructions for how to run all microservices locally via Visual Studio

Customization

Changing container names

If you would like to change the name of your containers, you will need to modify the name of the container in the pcs-cli files for:

  • all-in-one.yaml
  • the remotemonitoring/scripts/individual file for the container
  • ingress.yaml
  • deployment-configmap.yaml

After you have renamed the containter in the pcs-cli, you will need to redeploy with the pcs-cli

Upgrade map key to see devices on a dynamic map

You will need a Azure Maps key. You can follow these instructions: Getting a Azure Maps Key

  1. Open a command prompt/bash window and SSH into your VM
  2. Navigate to the /app folder cd /app
  3. Edit the env-vars file with sudo nano env-vars
  4. Edit the following line: 
    export PCS_AZUREMAPS_KEY="<your key here>"
  5. Press control+x enter.
  6. Restart the service with the start.sh script. 
    sudo ./start.sh

Switch to a static map

Follow the instructions for Upgrade map key to see devices on a dynamic map instead changing the environment variable to PCS_AZUREMAPS_KEY="static"

Firmware Update

The default solution uses simulated devices to generate data. Since they are simulated, there is no real firmware to update. Therefore, firmware update simply updates the version reported to the device twin and marks the job as complete.

In order to implement firmware update for your devices you will need to do some work to ensure your devices know which methods to execute when it receives the update method from the Remote Monitoring Solution front end.

For more information on what code you could run on the device see the tutorial (here)[https://docs.microsoft.com/azure/iot-hub/iot-hub-csharp-csharp-firmware-update#trigger-a-remote-firmware-update-on-the-device-using-a-direct-method].

If you are interested in reporting a specific status for that device you could consider adding a "MethodStatus" object to the device twin that the device would use to report the current status of your firmware update method. Useful states could include "Pending" | "Downloading" | "Applying" | "Rebooting" | "Complete".

   "reported": {
      "Protocol": "MQTT",
      "SupportedMethods": "Reboot,FirmwareUpdate,EmergencyValveRelease,IncreasePressure",
      },
      "MethodStatus": {
          "FirmwareUpdate": "Downloading",
      },
      "Firmware": "1.0",
      "Type": "Chiller",
      "Model": "CH101",
      "Location": "Building 2",
      "Latitude": 47.640792,
      "Longitude": -122.126258,
      "$metadata": {

      ...
Clone this wiki locally