This SDK has several entry points:
-
You can run the application using the CLI.
-
You can build a container containing the WebAPI and connect via REST requests.
-
(Future) You can install the Nuget package and make requests directly. (tracked here)
Each of these has configuration requirements which are detailed below. You can also visit the quickstart.md guide for a step-by-step process for running the CLI locally, deploying the Web API locally, polling the API via HTTP requests or generating and using client libraries (Python example).
Make sure you have installed the following pre-requisites:
- dotnet core SDK https://dotnet.microsoft.com/en-us/download
- WattTime account - See instruction on WattTime for details (or use our python samples as described here).
We intend to support multiple data sources for carbon data. At this time, only a JSON file and WattTime are supported. To use WattTime data, you'll need to acquire a license from them and set the appropriate configuration information.
This project uses standard Microsoft.Extensions.Configuration mechanisms.
The WebAPI project uses standard configuration sources provided by ASPNetCore. Please review this link to understand how configuration is loaded and the priority of that configuration.
Please note that configuration is hierarchical. The last configuration source loaded that contains a configuration value will be the value that's used. This means that if the same configuration value is found in both appsettings.json and as an environment variable, the value from the environment variable will be the value that's applied.
When adding values via environment variables, we recommend that you use the double underscore form, rather than the colon form. Colons won't work in non-windows environment. For example:
CarbonAwareVars__CarbonIntensityDataSource="WattTime"Note that double underscores are used to represent dotted notation or child elements that you see in the JSON below. For example, to set proxy information using environment variables, you'd do this:
CarbonAwareVars__Proxy__UseProxyFor local-only settings you can use environment variables, the Secret Manager tool, or an untracked Development appsettings file to override the default project settings.
To use the settings file, rename a copy of the template called appsettings.Development.json.template to appsettings.Development.json and remove the first line of (invalid) comments. Then update any settings according to your preferences.
Wherever possible, the projects leverage the default .NET configuration expectations. Thus, they can be configured using any file matching the format:
appsettings.<ENV>.json. Where<ENV>is the value of theASPNETCORE_ENVIRONMENTenvironment variable. By convention projects tend to use the provided HostEnvironment constantsDevelopment,Staging, andProduction.
Used to configure specific values that affect how the application gets data and the routes exposed. The configuration looks like this:
{
"carbonAwareVars": {
"carbonIntensityDataSource": "",
"webApiRoutePrefix": "",
"proxy": {
"useProxy": false,
"url": "",
"username": "",
"password": ""
}
}
}Must be one of the following: None, JSON, WattTime.
If set to WattTime, WattTime configuration must also be supplied.
None is the default, and if this value is supplied, an exception will be thrown at startup.
JSON will result in the data being loaded from a json file compiled into the project. You should not use these values in production, since they are static and don't represent carbon intensity accurately.
Used to add a prefix to all routes in the WebApi project. Must start with a /. Invalid paths will cause an exception to be thrown at startup.
By default, all controllers are off of the root path. For example:
http://localhost/emissionsIf this prefix is set, it will allow calls to controllers using the prefix, which can be helpful for cross cluster calls, or when proxies strip out information from headers. For example, if this value is set to:
/mydepartment/myappThen calls can be made that look like this:
http://localhost/mydepartment/myapp/emissionsNote that the controllers still respond off of the root path.
This value is used to set proxy information in situations where internet egress requires a proxy. For proxy values to be used useProxy must be set to true. Other values should be set as needed for your environment.
If using the WattTime datasource, WattTime configuration is required.
{
"wattTimeClient":{
"username": "",
"password": "",
"baseUrl": "https://api2.watttime.org/v2/"
}
}Sign up for a test account: To create an account, follow these steps : https://www.watttime.org/api-documentation/#best-practices-for-api-usage
The username you receive from WattTime. This value is required when using a WattTime datasource.
The WattTime password for the username supplied. This value is required when using a WattTime datasource.
The url to use when connecting to WattTime. Defaults to https://api2.watttime.org/v2/.
In normal use, you shouldn't need to set this value, but this value can be used to enable integration testing scenarios or if the WattTime url should change in the future.
This project is using standard Microsoft.Extensions.Logging. To configure different log levels, please see the documentation at this link.
Application monitoring and tracing can be configured using the TelemetryProvider variable in the application configuration.
CarbonAwareVars__TelemetryProvider="ApplicationInsights"This application is integrated with Application Insights for monitoring purposes. The telemetry collected in the app is pushed to AppInsights and can be tracked for logs, exceptions, traces and more. To connect to your Application Insights instance, configure the ApplicationInsights_Connection_String variable.
ApplicationInsights_Connection_String="AppInsightsConnectionString"You can alternatively configure using Instrumentation Key by setting the AppInsights_InstrumentationKey variable. However, Microsoft is ending technical support for instrumentation key–based configuration of the Application Insights feature soon. ConnectionString-based configuration should be used over InstrumentationKey. For more details, please refer to https://docs.microsoft.com/en-us/azure/azure-monitor/app/sdk-connection-string?tabs=net.
AppInsights_InstrumentationKey="AppInsightsInstrumentationKey"You can configure the verbosity of the application error messages by setting the 'VerboseApi' enviroment variable. Typically, you would set this value to 'true' in the development or staging regions. When set to 'true', a detailed stack trace would be presented for any errors in the request.
CarbonAwareVars__VerboseApi="true"By setting JsonDataSourceConfiguration__DataFileLocation=newdataset.json property when CarbonAwareVars__CarbonIntensityDataSource=JSON is set or there is not data source defined (JSON is by default), the user can specify a file that can contains custom EmissionsData sets. The file should be located under the <user's repo>/src/data/data-sources/ directory that is part of the repository. At build time, all the JSON files under <user's repo>/src/data/data-sources/ are copied over the destination directory <user's repo>/src/CarbonAware.WebApi/src/bin/[Debug|Publish]/net6.0/data-sources/json that is part of the CarbonAware.WebApi assembly. Also the file can be placed where the assembly CarbonAware.WebApi.dll is located under data-sources/json directory. For instance, if the application is installed under /app, copy the file to /app/data-sources/json. This can be done before the application starts by setting JsonDataSourceConfiguration__DataFileLocation environment variable.
cp <mydir>/newdataset.json /app/data-sources/json
export CarbonAwareVars__CarbonIntensityDataSource=JSON
export JsonDataSourceConfiguration__DataFileLocation=newdataset.json
dotnet /app/CarbonAware.WebApi.dllAs soon a first request is performed, a log entry shows:
info: CarbonAware.DataSources.Json.JsonDataSource[0]
Reading Json data from /app/data-sources/json/newdataset.json
To improve performance communicating with the WattTime API service, the client caches the data mapping location coordinates to balancing authorities. By default, this data is stored in an in-memory cache for 86400 seconds, but expiration can be configured using the setting BalancingAuthorityCacheTTL (Set to "0" to not use cache). The regional boundaries of a balancing authority tend to be stable, but as they can change, the WattTime documentation recommends not caching for longer than 1 month.
WattTimeClient__BalancingAuthorityCacheTTL="90"By setting LocationDataSourcesConfiguration property with one or more location data sources, it is possible to load different Location data sets in order to have more than one location. For instance by setting two location regions, the property would be set as follow using environment variables:
"CarbonAwareVars__CarbonIntensityDataSource": "WattTime",
"WattTimeClient__Username": "wattTimeUsername",
"WattTimeClient__Password": "wattTimePassword",
"LocationDataSourcesConfiguration__LocationSourceFiles__0__DataFileLocation": "azure-regions.json",
"LocationDataSourcesConfiguration__LocationSourceFiles__0__Prefix": "az",
"LocationDataSourcesConfiguration__LocationSourceFiles__0__Delimiter": "-",
"LocationDataSourcesConfiguration__LocationSourceFiles__1__DataFileLocation": "custom-regions.json",
"LocationDataSourcesConfiguration__LocationSourceFiles__1__Prefix": "custom",
"LocationDataSourcesConfiguration__LocationSourceFiles__1__Delimiter": "_",This way when the application starts, it open the files specified by DataFileLocation property that should located under location-sources/json directory. The format of these files is the same as the Location Model class. In order to differentiate between regions, a Prefix and Delimiter properties are used to allow the user to select the region when a request is performed. By settings the properties, the region should be made of region=Prefix+Delimiter+RegionName, so when the query is performed, it would be found. The following example shows how to perform an http request:
PREFIX=az
DELIMITER='-'
REGION=${PREFIX}${DELIMITER}eastus
curl "http://${IP_HOST}:${PORT}/emissions/bylocations/best?location=${REGION}&time=2022-05-25&toTime=2022-05-26&durationMinutes=0"At build time, all the JSON files under <user's repo>/src/data/location-sources are copied over the destination directory <user's repo>/src/CarbonAware.WebApi/src/bin/[Debug|Publish]/net6.0/location-sources/json that is part of the CarbonAware.WebApi assembly. Also the file can be placed where the assembly CarbonAware.WebApi.dll is located under location-sources/json directory. For instance, if the application is installed under /app, copy the file to /app/location-sources/json.
CarbonAwareVars__CarbonIntensityDataSource="WattTime"
CarbonAwareVars__WebApiRoutePrefix="/microsoft/cse/fsi"
CarbonAwareVars__Proxy__UseProxy=true
CarbonAwareVars__Proxy__Url="http://10.10.10.1"
CarbonAwareVars__Proxy__Username="proxyUsername"
CarbonAwareVars__Proxy__Password="proxyPassword"
WattTimeClient__Username="wattTimeUsername"
WattTimeClient__Password="wattTimePassword"{
"carbonAwareVars": {
"carbonIntensityDataSource": "WattTime",
"webApiRoutePrefix": "/microsoft/cse/fsi",
"proxy": {
"useProxy": true,
"url": "http://10.10.10.1",
"username": "proxyUsername",
"password": "proxyPassword"
}
},
"wattTimeClient":{
"username": "wattTimeUsername",
"password": "wattTimePassword",
}
}{
"carbonAwareVars": {
"carbonIntensityDataSource": "WattTime",
},
"wattTimeClient": {
"username": "wattTimeUsername",
"password": "wattTimePassword"
},
"locationDataSourcesConfiguration": {
"locationSourceFiles": [
{
"prefix": "az",
"delimiter": "-",
"dataFileLocation": "azure-regions.json"
},
{
"prefix": "custom",
"delimiter": "_",
"dataFileLocation": "custom-regions.json"
}
]
}
}You can publish Web API for Carbon Aware SDK with container. This instruction shows how to build / run container image with Podman.
Following commands build the container which named to carbon-aware-sdk-webapi from sources.
$ cd src
$ podman build -t carbon-aware-sdk-webapi -f CarbonAware.WebApi/src/Dockerfile .Carbon Aware SDK Web API publishes the service on Port 80, so you need to map it to local port. Following commands maps it to Port 8080.
You also need to configure the SDK with environment variables. They are minimum set when you use WattTime as a data source.
$ podman run -it --rm -p 8080:80 \
-e CarbonAwareVars__CarbonIntensityDataSource="WattTime" \
-e WattTimeClient__Username="wattTimeUsername" \
-e WattTimeClient__Password="wattTimePassword" \
carbon-aware-sdk-webapiWhen you success to run the container, you can access it via HTTP client.
$ curl -s http://localhost:8080/emissions/forecasts/current?location=westus2 | jq
[
{
"generatedAt": "2022-08-10T14:10:00+00:00",
"optimalDataPoint": {
"location": "GCPD",
"timestamp": "2022-08-10T20:40:00+00:00",
"duration": 5,
"value": 440.4361702590741
},
: