Cdr.Banking

Banking APIs

The purpose of this sample is to demonstrate the usage of Beef in a real-world scenario. The scenario that has been chosen is a partial implementation of the Open Banking APIs as defined by the Consumer Data Standards in Australia.

Consumer Data Standards

As described on the web https://consumerdatastandards.org.au/:

The Australian government has introduced a Consumer Data Right giving consumers greater control over their data. Part of this right requires the creation of common technical standards making it easier and safer for consumers to access data held about them by businesses, and – if they choose to – share this data via application programming interfaces (APIs) with trusted, accredited third parties."

The Banking APIs and corresponding schema are documented here.

Scope

The full scope of the APIs has not been implemented; only the following endpoints have been created to demonstrate:

• Get Accounts - GET /banking/accounts
• Get Account Balance - GET /banking/accounts/{accountId}/balance
• Get Account Detail - GET /banking/accounts/{accountId}
• Get Transactions for Account GET /banking/accounts/{accountId}/transactions

Assumptions

Assumptions are as follows:

• Authentication - no formal authentication (such as OAuth) has been implemented; a faux authentication is implemented by passing a HTTP header (cdr-user) containing the user name (in plain-text) compared to a hard-coded list of values. This is obviously not how you would implement proper, but for the sake of brevity enables the basic capability.
• Authorization - again for brevity, a list of accounts are allocated to a user to limit (filter) the accounts a user should be able to see and interact with. Where a user attempts to interact with an account they do not have permission for an HTTP status of Forbidden (403) will be returned.
• Any additional request HTTP headers specified by the x- prefix, and the corresponding response enveloping data, links and meta would be managed by an API Gateway, for example Azure API Management.

Cosmos DB usage

For the purposes of this sample, Azure Cosmos DB has been chosen as the data store.

As the operations are read-only, the data store would be (should be) optimised for these required read activitiies. In this instance the assumption is that the Cosmos DB store would be a near real-time replica from the system of record. There would be an on-going process to synchronise the Cosmos DB data with the latest infomation, using the likes of event steaming for example.

The following Containers will be used:

• RefData - All Reference Data types and their corresponding items. No partitioning will be used, as there is limited benefit in partitioning per reference data type given the limited volume within, and the limited access given largely cached in memory.
• Account - All Accounts, for all users. No partioning will be used. Accounts will need to be explicitly filtered per user request to ensure correct access.
• Transaction - All Transactions, for all accounts. Partitioning will be used, a partition per Account; i.e. all transactions for a given Account will reside exclusively within its Account partition. Access to a given partition will need to be explicitly allowed per user request to ensure correct access.

The requirements specify when filtering Transactions the following:

As the date and time for a transaction can alter depending on status and transaction type two separate date/times are included in the payload. There are still some scenarios where neither of these time stamps is available. For the purpose of filtering and ordering it is expected that the data holder will use the “effective” date/time which will be defined as: - Posted date/time if available, then - Execution date/time if available, then - A reasonable date/time nominated by the data holder using internal data structures.

To simplify any querying for date and time will occur against a new property defined in the underlying data model for Transaction. This property TransactionDateTime should be updated with the correct datetime when loaded into the Cosmos DB Container; this will avoid the need for an overly complex query against multiple data and time properties.

Solution skeleton

This solution was originally created using the solution template capability, following the getting started guide.

The following command was issued to create the solution structure.

dotnet new beef --company Cdr --appname Banking --datasource Cosmos

└── Cdr.Banking
  └── Cdr.Banking.Api         # API end-point and operations
  └── Cdr.Banking.Business    # Core business logic components
  └── Cdr.Banking.CodeGen     # Entity and Reference Data code generation console
  └── Cdr.Banking.Common      # Common / shared components
  └── Cdr.Banking.Test        # Unit and intra-integration tests
  └── Cdr.Banking.sln         # Solution file that references all above projects

Note: Code generation was not performed before updating the corresponding YAML files described in the next section. Otherwise, extraneous files would have been generated that would then need to be manually removed.

Also, any files that started with Person (being the demonstration entity) were removed (deleted) from their respective projects. This then represented the base-line to build up the solution from.

Code generation

The following code-generation files require configuration:

• entity.beef.yaml - this describes the entities, their properties and operations, to fulfil the aforementioned CDR Banking API endpoint and schema requirements.
• refdata.beef.yaml - this describes the reference data entities, their properties, and corresponding get all (read) operation. These are defined seperately as different code-generation templates are used.
• datamodel.beef.yaml - this describes the Cosmos DB data models and their properties. These are logically seperated as only a basic model class is generated.

Each of the files have comments added within to aid the reader as to purpose of the configuration. Otherwise, see the related entity-driven code-generation documentation for more information.

The following command was issued to perform the code-generation.

dotnet new all

Authentication/Authorisation

To demonstrate the authentication and authorisation the following is required:

• HTTP Header (cdr-user) containing the username.
• Customised ExecutionContext with a list of Accounts the user has permission to view.
• Data access filtering to ensure only the correct Accounts and Transactions are accessed for a given user.

Execution Context

A custom ExecutionContext that inherits from the CoreEx ExecutionContext is required. An Accounts property is added to contain the list of permissable Accounts.

public class ExecutionContext : CoreEx.ExecutionContext
{
    /// <summary>
    /// Gets the current <see cref="ExecutionContext"/> instance.
    /// </summary>
    public static new ExecutionContext Current => (ExecutionContext)CoreEx.ExecutionContext.Current;

    /// <summary>
    /// Gets the list of account (identifiers) that the user has access/permission to.
    /// </summary>
    public List<string> Accounts { get; } = new();
}

Set up Execution Context

The ExecutionContext is required to be configured for each and every request, this is performed within the API Startup.cs.

Firstly, the custom ExecutionContext instantiation must be registered within the ConfigureServices method.

// Add the core services (including the customized ExecutionContext).
services.AddExecutionContext(_ => new Business.ExecutionContext())

Within the Configure method the User to Accounts mapping is performed. As stated previously in the earlier assumptions, the likes of an OAuth token would have previously been validated, then would either contain the list of Accounts, or these would be loaded from an appropriate data store into the ExecutionContext.

// Add execution context set up to the pipeline.
app.UseExecutionContext((hc, ec) =>
{
    // TODO: This would be replaced with appropriate OAuth integration, etc... - this is purely for illustrative purposes only.
    if (!hc.Request.Headers.TryGetValue("cdr-user", out var username) || username.Count != 1)
        throw new AuthenticationException();

    var bec = (Business.ExecutionContext)ec;
    bec.Timestamp = SystemTime.Get().UtcNow;

    switch (username[0])
    {
        case "jessica":
            bec.Accounts.AddRange(new string[] { "12345678", "34567890", "45678901" });
            break;

        case "jenny":
            bec.Accounts.Add("23456789");
            break;

        case "jason":
            break;

        default:
            throw new AuthenticationException();
    }

    return Task.CompletedTask;
});

Account-wide filtering

To ensure consistent filtering for all CRUD (Query, Get, Create, Update and Delete) operations an authorization filter can be applied within the CosmosDb.cs. This class inherits from the CoreEx.Cosmos.CosmosDb. The UseAuthorizeFilter can be used to define a filter to be applied to all operations that occur against the specified Model and Container.

By defining holistically, it allows the capability to be added or maintained independent of its individual usage. Whereby minimising on-going change and potential errors where not implemented consistently through-out the code base.

In this case, the filter will ensure that only Accounts that have been defined for the User can be accessed.

public class CosmosDb : CoreEx.Cosmos.CosmosDb, ICosmos
{
    /// <summary>
    /// Initializes a new instance of the <see cref="CosmosDb"/> class.
    /// </summary>
    public CosmosDb(Mac.Database database, IMapper mapper, CosmosDbInvoker? invoker = null) : base(database, mapper, invoker)
    {
        // Apply an authorization filter to all operations to ensure only the valid data is available based on the users context; i.e. only allow access to Accounts within list defined on ExecutionContext.
        UseAuthorizeFilter<Model.Account>("Account", (q) => ((IQueryable<Model.Account>)q).Where(x => ExecutionContext.Current.Accounts.Contains(x.Id!)));
        UseAuthorizeFilter<Model.Account>("Transaction", (q) => ((IQueryable<Model.Transaction>)q).Where(x => ExecutionContext.Current.Accounts.Contains(x.AccountId!)));
    }
}

Transaction-wide filtering

For the Transaction entity, we have chosen the strategy to leverage the PartitionKey as a means to divide (and isolate) the Transactions to an owning Account.

In this instance, the onus is on the developer to set the PartitionKey appropriately before performing the underlying Cosmos DB operation.

In this case, we are setting this using the code-generation for the operation. The DataCosmosPartitionKey attribute for the Operation element enables. The accountId parameter value will be used for partitioning.

# Operation to get all Transactions for a specified Account.
# Operation and Route requires accountId; e.g. api/v1/banking/accounts/{accountId}/transactions
# Supports filtering using defined properies from TransactionArgs (the args will be validated TransactionArgsValidator) to ensure valid values are passed).
# Supports paging.
# Data access will be auto-implemented for Cosmos as defined for the entity.
# Cosmos PartitionKey will be set to the accountId parameter value for data access.
# 
{ name: GetTransactions, text: Get transaction for account, type: GetColl, webApiRoute: '{accountId}/ransactions', paging: true, cosmosPartitionKey: accountId,
  parameters: [
    # Note usage of ValidatorCode which will inject the code as-is into the validation logic; being a common validator 'Validators.Account' that will perform the authorization check.
    { name: AccountId, type: string, validatorCode: Common(Validators.AccountId), webApiFrom: romRoute, isMandatory: true },
    { name: Args, type: TransactionArgs, validator: TransactionArgsValidator }
  ]
}

As stated in the above YAML comments, a common validator will be used to perform the authorization logic. The static Validators.AccountId is used to perform the validation.

    public static CommonValidator<string?> AccountId => CommonValidator.Create<string?>(v => v.Custom(ctx 
        => Result.Go().When(() => ctx.Value == null || !ExecutionContext.Current.Accounts.Contains(ctx.Value), () => Result.AuthorizationError())));

Data access

Where possible the Beef "out-of-the-box" data access is leveraged via the code-generation configuration. However, to meet the CDR requirements around data filtering this logic must be customized. This is achieved using the plug-in opportunities offered by the code generation.

Get Accounts

The Account filtering as required by /banking/accounts uses the AccountArgs entity to provide the possible selection criteria. The filtering then uses the standard LINQ filtering capabilities offered by the Cosmos DB SDK. The GetAccountsOnQuery method within the non-generated AccountData.cs partial class contains the filtering logic.

private IQueryable<Model.Account> GetAccountsOnQuery(IQueryable<Model.Account> query, AccountArgs? args)
{
    var q = query.OrderBy(x => x.Id).AsQueryable();
    if (args == null || args.IsInitial)
        return q;

    // Where an argument value has been specified then add as a filter - the WhereWhen and WhereWith are enabled by CoreEx.
    q = q.WhereWhen(!(args.OpenStatus == null) && args.OpenStatus != OpenStatus.All, x => x.OpenStatus == args.OpenStatus!.Code);
    q = q.WhereWith(args?.ProductCategory, x => x.ProductCategory == args!.ProductCategory!.Code);

    // With checking IsOwned a simple false check cannot be performed with Cosmos; assume "not IsDefined" is equivalent to false also. 
    if (args!.IsOwned == null)
        return q;

    if (args.IsOwned == true)
        return q.Where(x => x.IsOwned == true);
    else
        return q.Where(x => !x.IsOwned.IsDefined() || !x.IsOwned);
}

Get Account Balance

The Account balance as required is implemented in a fully customised manner. The code-generated output will invoke the GetBalanceOnImplementationAsync method within the non-generated AccountData.cs partial class.

The underlying logic queries the Account container for the specified accountId and returns the corresponding Balance.

private Task<Result<Balance?>> GetBalanceOnImplementationAsync(string? accountId)
{
    // Use the 'Account' model and select for the specified id to access the balance property.
    return Result.GoAsync(_cosmos.Accounts.ModelQuery(q => q.Where(x => x.Id == accountId)).SelectFirstOrDefaultWithResultAsync())
        .WhenAs(a => a is not null, a =>
        {
            var bal = _cosmos.Mapper.Map<Model.Balance, Balance>(a!.Balance);
            return bal.Adjust(b => b.Id = a.Id);
        });
}

Get Transactions for Account

The Transaction filtering as required by /banking/accounts/\{accountId\}/transactions uses the TransactionArgs entity to provide the possible selection criteria. The filtering then uses the standard LINQ filtering capabilities offered by the Cosmos DB SDK. The GetTransactionsOnQuery method within the non-generated TransactionData.cs partial class contains the filtering logic.

private IQueryable<Model.Transaction> GetTransactionsOnQuery(IQueryable<Model.Transaction> query, string? _, TransactionArgs? args)
{
    if (args == null || args.IsInitial)
        return query.OrderByDescending(x => x.TransactionDateTime);

    var q = query.WhereWith(args.FromDate, x => x.TransactionDateTime >= args.FromDate);
    q = q.WhereWith(args.ToDate, x => x.TransactionDateTime <= args.ToDate);
    q = q.WhereWith(args.MinAmount, x => x.Amount >= args.MinAmount);
    q = q.WhereWith(args.MaxAmount, x => x.Amount <= args.MaxAmount);

    // The text filtering will perform a case-insensitive (based on uppercase) comparison on Description and Reference properties. 
    q = q.WhereWith(args.Text, x => x.Description!.ToUpper().Contains(args.Text!.ToUpper()) || x.Reference!.ToUpper().Contains(args.Text!.ToUpper()));

    // Order by TransactionDateTime in descending order.
    return q.OrderByDescending(x => x.TransactionDateTime);
}

Validation

To minimise the bad request data, and meet the CDR functional requirements for the operation arguments, validation has been included.

Get Accounts

The validation for the AccountArgs is relatively straightforward in that the OpenStatus and ProductCategory will be checked to ensure the passed reference data values are considered valid. The AccountArgsValidator.cs provides the implementation.

/// <summary>
/// Represents a <see cref="AccountArgs"/> validator.
/// </summary>
public class AccountArgsValidator : Validator<AccountArgs>
{
    /// <summary>
    /// Initializes a new instance of the <see cref="AccountArgsValidator"/>.
    /// </summary>
    public AccountArgsValidator()
    {
        Property(x => x.OpenStatus).IsValid();
        Property(x => x.ProductCategory).IsValid();
    }
}

Get Transactions for Account

The validation for the TransactionArgs is a little more nuanced. The following needs to be performed as per the CDR requirements:

• Default FromDate where not provided, as 90 days less than ToDate; where no ToDate then assume today (now).
• Make sure FromDate is not greater than ToDate.
• Make sure MinAmount is not greater than MaxAmount.
• Additionally, make sure Text does not include the '*' wildcard character (do not want to give appearance is support).

The TransactionArgsValidator.cs provides the implementation.

/// <summary>
/// Represents a <see cref="TransactionArgs"/> validator.
/// </summary>
public class TransactionArgsValidator : Validator<TransactionArgs>
{
    /// <summary>
    /// Initializes a new instance of the <see cref="TransactionArgsValidator"/>.
    /// </summary>
    public TransactionArgsValidator()
    {
        // Default FromDate where not provided, as 90 days less than ToDate; where no ToDate then assume today (now). Make sure FromDate is not greater than ToDate.
        Property(x => x.FromDate)
            .Default(a => (a.ToDate!.HasValue ? a.ToDate.Value : Business.ExecutionContext.Current.Timestamp).AddDays(-90))
            .CompareProperty(CompareOperator.LessThanEqual, y => y.ToDate).DependsOn(y => y.ToDate);

        // Make sure MinAmount is not greater than MaxAmount.
        Property(x => x.MinAmount).CompareProperty(CompareOperator.LessThanEqual, y => y.MaxAmount).DependsOn(y => y.MaxAmount);

        // Make sure the Text does not include the '*' wildcard character.
        Property(x => x.Text).Wildcard(CoreEx.Wildcards.Wildcard.None);
    }
}

Paging

Beef enables, and supports, paging out-of-the-box, with built-in support to minimize the requirement for a developer to implement beyond declaring intent through the code-generation configuration. This is how the paging capability has been implemented within the solution.

However, the CDR specification specifies that the paging is to be supported using the following query string parameters:

• page - representing the page number
• page-size - represents the page size (defaults to 25).

These are not supported out-of-the-box and support must be added. This is performed within the API Startup.cs. The following is added to the Startup method.

// Add "page" and "page-size" to the supported paging query string parameters as defined by the CDR specification.
HttpConsts.PagingArgsPageQueryStringNames.Add("page");
HttpConsts.PagingArgsTakeQueryStringNames.Add("page-size");

Testing

A reasonably thorough set of intra-domain integration tests have been added to demonstrate usage, as well as validate that the selected CDR Banking operations function as described. For the most part the tests should be self-explanatory.

• AccountTest.cs
• TransactionTest.cs

Of note, within the FixtureSetup.cs the authorization header and paging configuration is set up.

// TODO: Passing the username as an http header for all requests; this would be replaced with OAuth integration, etc.
TestSetUp.Default.OnBeforeHttpRequestMessageSendAsync = (req, userName, _) =>
{
    req.Headers.Add("cdr-user", userName);
    return Task.CompletedTask;
};

// Set "page" and "page-size" as the supported paging query string parameters as defined by the CDR specification.
CoreEx.Http.HttpConsts.PagingArgsPageQueryStringName = "page";
CoreEx.Http.HttpConsts.PagingArgsSizeQueryStringName = "page-size";