Noting Paper 323 - NFR Workshops

[CDR-API-Stream]

This item is provided to provide detail for the upcoming workshops on non-functional requirements (NFRs).

The DSB doesn't normally create a noting paper thread for workshops but we are doing so in this case as a vehicle for requesting suggestions for topics to discuss relating to NFRs. Substantial time in the planned workshops is being reserved to discuss specific topics related to NFRs.

The noting paper is attached below:
Noting Paper - NFR Workshops.pdf

Note that this paper is a PDF of a slide deck so is a different format than usual. We will presenting this slides at the implementation call on 3rd August.

This thread will remain open until after the final workshop in September is complete and we will also post any summaries of the workshop feedback that is produced.

[CDR-API-Stream]

The following issue was raised via standards maintenance.

Should a Primary Data Holder throttle calls that will be served by a Secondary Data Holder or should the Secondary Data Holder be solely accountable for doing this throttling?

This is the subject of the following CRs:
ConsumerDataStandardsAustralia/standards-maintenance#602
ConsumerDataStandardsAustralia/standards-maintenance#603
ConsumerDataStandardsAustralia/standards-maintenance#604

[CDR-API-Stream]

The following issue was raised via regular meetings hosted by AEMO and the AEC.

For C&I customers in energy a small number of customers have thousands of accounts. Does this create an issue with the current NFR thresholds? Should an alternate mechanism, such as an asynchronous call, be used to accommodate these scenarios?

[CDR-API-Stream]

The following issue was raised via general discussions in DSB forums.

Are there any scenarios in banking where the NFRs for low velocity data sets should apply?

[CDR-API-Stream]

The following issue was raised via feedback on decision 288.

Are the current consent based thresholds for site wide traffic set correctly or should they be amended in light of real world experience?

[CDR-API-Stream]

As the first workshop is tomorrow and we have committed to all three workshops having a consistent agenda we will be setting the final workshop agenda today after the CDR Implementation Call.

If anyone would like to raise another issue to be considered in the workshops please add it to this thread before that time.

[johnAEMO]

Where should NFR times be measured?
Recent analysis of transactions between secondary data holder and primary data holder across a full day of transactions noted that the transmission lag between the two (ie the difference between the secondary call duration as measured at each) was 101 ms on average with a range varying between 25 and 643 ms per transaction. (Not surprisingly there was a correlation between the response payload size and the size of the lag with a transaction of 1000 records taking an additional 643ms to get to the primary data holder. This was for all successful transactions on the day, across all APIs, transmitted by Internet and not AEMO's MarketNet.)
If that lag was to be replicated between the primary data holder and the ADR, the ADR could see an increased duration of 0.2 secs on average but up to 1.3 secs for large payloads.
If the NFR times were measured at the ADR, the transmission lag for large payloads would make some of the NFR times unachievable. The strong preference is that for NFR times to be measured at the source data holder or to make allowances for transmission lag times if measured at the ADR.

[CDR-API-Stream]

At the first workshop last week the following additional topics were added to this list. These will be put on the agenda at the second workshop, in Melbourne next Tuesday.

• Due to the importance of the consent flow to customer adoption, it may be helpful to have additional non-functional requirements for the consent flow. Alternatively, the non-functional requirements for availability could be modified to separate out the consent flow as a distinct and specific requirement
• How can the obligation of the non-functional requirements best be cascaded down to outsource service providers by data holders?
• Should we revisit the non-functional requirements for dynamic client registration?
• How can we, collectively and collaboratively, manage and plan demand so that traffic can be shaped to avoid the need for traffic throttling?

[CDR-API-Stream]

This note is an update on the work arising from these workshops.

Takeaways

The key takeaways from the workshops for the DSB are outlined below...

Sharing of performance data

• There was consistent feedback provided that NFRs should be set based on data provided by participants on existing performance.
• The sharing of data in a fully open and transparent consultation is a barrier for many participants for a variety of reasons.
• The type of data that would need to be shared would likely evolve over time and, in some cases, may be tailored to a specific issue being addressed.
• The internal legal teams of some DHs were explicitly concerned about sharing detailed performance data with the regulator due to the risk that the data could lead to compliance action being taken.
• Counter feedback was also provided that the involvement of the regulator would be of benefit since performance issues are frequently raised as incidents between participants.

Future planning and forecasting

• DHs indicated that a multi-year forecast for NFRs would be beneficial due to the long lead time of infrastructure investment planning in most organisations.
• It was acknowledged that the ability to forecast traffic and usage is very difficult and unlikely to be accurate but that this would still be useful for participants.
• It was also acknowledged that a forecast of future usage can only be done by the participants. The DSB would not be able to do this kind of planning without active involvement of both ADRs and DHs.

The process of consultation

• Participants appreciate the transparency and openness of the current consultation process and would like to ensure that this is not lost if the process is changed.
• There was feedback during the workshops, and in response to Decision Proposal 288, that a working group of participants to collaborate on proposals to change the NFRs that are then publicly consulted on would be a possible solution.
• Participants indicated that a working group would need members that have technical expertise in performance and infrastructure management and that it would be best if there was continuity of membership over time.

Next Steps

It has been decided to trial working group for the development of Non-functional Requirements to be established by the Data Standards Chair as an advisory group.

Inter-agency consultation has been undertaken to determine the next steps. Based on the feedback received the next steps will be:

1. Develop a concrete proposal for an initial trial for a NFR Working Group and table this with the Data Standards Advisory Committee for comment.
2. After incorporating feedback, seek the approval of the Data Standards Chair to proceed.
3. Execute this trial and perform a retrospective to determine whether the approach was effective and how it could be improved.
4. In light of the trial, develop an ongoing approach to consultation on NFRs to be approved by the Data Standards Chair.

The trial proposal will be tabled at the November DSAC meeting. Assuming DSAC support and the approval of the Chair we will be seeking nominations for members of the trial working group. Technically oriented people from a variety of active participants in the CDR ecosystems will be preferred. The members will be personally selected by the Chair.

Please consider if you, or someone you know would be a good candidate.

[biza-io]

Biza is encouraged by the establishment of an NFR Working Group and would like to nominate for participation.

[CDR-API-Stream]

@biza-io, the nomination for Advisory and Consultative groups established by the Chair require the nomination of an individual. If you wish to nominate an individual to be considered by the Chair please send their details to contact@consumerdatastandards.gov.au