Special Interest Group - AI Readiness

[ColinEberhardt]

Business Problem

The recent advances in AI (and more specifically Generative AI) are poised to have a disruptive impact across both our business and home lives. The rapid advances (that are showing no sign of slowing) are leading to a wealth or potential that touches upon the products we offer to clients, our personal productivity as well as the operational aspects of our business. Very few technologies in the past have had such a broad 'reach'.

If you need any further persuading, McKinsey predict an impact of circa $200 billion on the Banking Industry.

There are certainly challenges and issues with the current technology (hallucinations, prompt injections), but thing are moving at pace, and the technology challenges we see today may not exist tomorrow.

There is a very strong desire to on-board, trial, experiment and release products using this technology across the FINOS membership and broader FS community. However, as an industry we have some unique challenges that the vendors, academics, and startups are not going to solve for us.

All banks have mature processes for on-boarding technology, however, (Generative) AI presents some new challenges that the existing processes may not be well-suited to addressing. Without going into the details, there is much work needed to allow the 'safe' use of AI (where safety considers both the customer and the bank) and ultimately allow FS organisations to rapidly adopt new technologies as they emerge.

Proposed Solution

The goal of this SIG is to provide a space to discuss challenges that are common to all members. This has parallels with Open Source Readiness and the SIG may choose to follow similar initiatives, creating educational materials, training and raising awareness. However, there is the opportunity to go further and mutually develop a governance framework that managed the on-boarding, development of, and running AI-based solutions within FS organisation. Ideally this would allow us to get ahead of future regulations.

Current State

There have been numerous conversations with organisations who have expressed an interest and nominated individuals who would participate in the SIG. Currently we have commitment from:

• Lloyds Banking Group
• Citi
• Microsoft
• Scott Logic

We also have a growing list of organisation who have expressed and interest, and we expect commitment to follow shortly.

Existing Materials

None relevant

SIG Maintainers

Colin Eberhardt (@ColinEberhardt), Scott Logic CTO, FINOS TOC chair has volunteered to co-chair this group. We have an expression of interest from another co-chair, with details following shortly.

SIG Contribution and Onboarding process (v. 1.0, last updated on October 27, 2020)

Below is the list of tasks that FINOS Team and the contribution author goes through in order to complete the FINOS SIG onboarding process. Please do not edit these contents at contribution time!

Voting (SIG Lead and FINOS ED)

• Contribution author triggers community socialization to gauge interest - recommended emailing issue link to community@finos.org
• If there's positive response from the Community, prepare short SIG proposal deck for the FINOS Governing Board. You can start from this template or see a previous SIG example)
• Assign issue to Executive Director (@mindthegab) to trigger vote
• Executive Director brings SIG to the Board for additional socialization and approval
• FINOS board accepts the SIG (and the onboarding process can move forward)

Identify project meta (Lead: FINOS SIG POC, Support: FINOS Marketing)

• Identify and Assign FINOS SIG POC
• Link to SIG approval resolution - // public link to board resolution
• SIG Name - AI Readiness
  • Assess current trademark status
  • Define new SIG name (if applicable)
  • Design new SIG logo (if applicable)
  • Trademark new SIG name and logo (if applicable)
• Category and sub-category (for FINOS Landscape) - Enablement / AI
• Existing content/assets or new Github repository - new
• Team composition: lead maintainer and other maintainers - Colin Eberhardt, Co-chair, Ian Micallef, Co-chair
• Meetings? (existing/yes/no) - yes, fortnightly
• Meeting minutes, agenda, attendance tracking (existing/yes/no) - yes, through github
• Documentation website (existing/yes/no) - no, we'll use markdown files in Github for now
• Define SIG slug : Dependant on Define new SIG name - ai-readiness

Project Communication Channel(s)

• Ask maintainers which communications channels they'd like to use and if more than one are needed. These may include
  • Creation of one or more project mailing lists (e.g. general@, or dev@ and users@)
  • Creation of one or more Slack or Symphony chats for the project
• Create the identified communication channels during infra set up
• Link communication channels linked front and center in the project README.md.

Repository transfer (Lead: FINOS Infra)

• Check GitHub repository transfer requirements:
  • finos-admin has Admin to all repositories to transfer
  • finos-admin is allowed to transfer repositories out of the org
  • if the repository is owned by a user (and not an org), the user must be able to transfer the repository to finos-admin
• Transfer all assets as GitHub repositories under github.com/finos
• Invite GitHub usernames to GitHub FINOS Org
• Create <sig-name>-maintainers GitHub team and invite users
• Configure finos-admins and finos-staff team permissions

Infra setup (Lead: FINOS Infra)

• Update SIG badge
• Update SIG README
• Enable meeting attendance tracking (optional)

Metadata update (Lead: FINOS Infra)

• Add SIG to metadata
• Add identities, orgs and affiliations to metadata
• Add logo to FINOS landscape
• Add SIG Leads emails to finos-project-maintainers@finos.org list
• Add SIG Leads GitHub usernames to the project-maintainers Team

Update FINOS SIG Roster

• FINOS Governance

Announcement (Lead: FINOS POC)

• Work with FINOS marketing to send out announcement to announce@finos.org , checkout announcement template at the Contribution page.
• Notify FINOS POC and FINOS marketing manager once the announcement has been sent out (FINOS infra)

Press Release (OPTIONAL) (Lead: FINOS POC)

• Identify quotes for press release
• Draft press release
• Send embargoed press release to reporters
• Coordinate social media promotion of the press release with FINOS marketing
  • Post on FINOS social media
  • Post on LF social media

Onboarding and training

• FINOS SIG Governance
• FINOS Project Lifecycle
• ODP Training

Contributors Metadata (Lead: FINOS SIG POC, Support: FINOS infra)

• For each SIG material contributor identified in the previous step, collect: the following info:
  • Fullname
  • GitHub username
  • Corporate email address
• Identify other existing contributors (assuming there's a contribution history (eg Git history)

[MikeWilsonZngly]

@ColinEberhardt. This is a great idea and we would be delighted to take part.We've deployed across ZNGLY conversational AI using Google Vertex and PaLM. This is across all media types - PDFs, videos, sharpoint sources too. It's being used in production today with a range of fintech and insurtech clients, and the banks they are working with. Delighted to share our experience with the group - it was a significant undertaking - and great to share what we learnt.

[Neetuj]

I love the Idea. I learned a lot from OSR and helped there a bit too. Will love to learn and contribute to this SIG too

[BrittanyIstenes]

@ColinEberhardt - I would like to sit down and discuss the FNMA participation within this SIG. Currently I sit on the co-chair for OSR and InnerSource. I would need to more than likely drop one to participate.

[tenatzen]

👍🏼 You've got the backing of all of us at the Zenith SIG on this one

[caradelia]

@ColinEberhardt this is a great idea for a SIG. With seeing full support from the Zenith SIG, I see no issues.

[awaiken]

definitely good idea and I'm sure I can get someone from my firm to be active here

[opatrascoiu]

this is a good idea 👍

[cm-howard]

Very supportive of this and I'd be very keen to get EPAM engaged and support bringing along some new individuals to contribute too.

[pcheong-lbg]

Great idea

[cronky]

Very keen to get involved. As some may know I've written quite a bit on the Enterprise Readiness topic (i.e. https://blog.scottlogic.com/2023/05/04/generative-ai-solution-architecture.html and this has also come up on Architect Tomorrow a few times - i.e. https://www.youtube.com/watch?v=l-zHfUkVYzc) and have experience with architecture for deploying and scaling advanced ML architectures for Financial services: https://www.youtube.com/watch?v=o3OF5aRSQco&

[pholleran]

I and GitHub are interested in participating 😄

[ajgnacht]

I and Microsoft are supportive and interested in participating.

[ColinEberhardt]

Thanks for all the support so for. Interestingly the UK government have just published a framework for GenAI:

https://www.gov.uk/government/publications/generative-ai-framework-for-hmg/generative-ai-framework-for-hmg-html

But this document is an ‘advice’ framework, not a ‘governance’ framework, which those trying to use this technology very much need.

[mcleo-d]

👋🏻 - It would be great to have a round table to discuss the purpose & governance surrounding the SIG as this could be a prime opportunity to gather requirements from banks without contributing materials back to an open standard / project.

Also, it would be great to have a FINOS member bank(s) as part of the leadership to support the nomination, help set the direction / agenda and keep the SIG delivering openly for the benefit of the industry.

This is extremely positive, but I do think we need to understand the community rules of engagement before fully committing firms and people.

[brunodom]

@ColinEberhardt I am supporting this initiative on behalf of Intel and have active participation in the SIG

[vicenteherrera]

I'm already participating on the Common Cloud Controls, and would like also to participate on this one.

[jstclair2019]

@ColinEberhardt would welcome participation here. I am working with a state non-profit coding academy and we have a grant for statewide AI readiness and adoption.

[ferguskeenan]

Keen to be involved as my company has a lot of experience managing more traditional software adoption within FS and it would be interesting to contribute that knowledge to this new problem space.

[TheFoxAtWork]

👋🏻 I would love to be a participant or support a leadership role depending on expected time commitment and availability. I'm currently keeping track of several AI and AI security related efforts across two foundations (among many other tasks and activities)

[eddie-knight]

Myself and others from Sonatype are interested in supporting this effort.

[miglaros]

From an onboarding perspective one of the things that stands out to me is the relative distance from the GenAI incubation work that tends to take place in most banks innovation centres and the operational risk assessments needing to be conducted by Information Security. All too often Innovation is understandably early without the benefit of fully formed learnings and engineering feedback whilst Info Sec is harried with yet another emerging tech to ramp up on.

I think there exists a happy middle with a light weight self-assessment process inspired by Microsoft's RAI standard (https://blogs.microsoft.com/wp-content/uploads/prod/sites/5/2022/06/[Microsoft-Responsible-AI-Standard-v2-General-Requirements-3.pdf](https://blogs.microsoft.com/wp-content/uploads/prod/sites/5/2022/06/Microsoft-Responsible-AI-Standard-v2-General-Requirements-3.pdf)) and OWASP’s Top
Ten for LLM applications https://owasp.org/www-project-top-10-for-large-language-model-applications/.

Perhaps in the GitHub we could contribute some potential assessment checklist ideas/approaches to be completed as part of onboarding best practice?

[rocketstack-matt]

There is crossover interest from us in the DevOps Automation SIG, @ashukla13 has raised the topic of use specifically within the context of AI Code Assistants (finos/devops-automation#179) and was planning a session dedicated to this topic.

One thing I think would be useful to come out of such a framework would be industry agreed metrics for measuring the effectiveness of GenAI in development. This report from GitClear highlights potential issues from increased code churn (code committed to the repo and subsequently reverted or changed within two weeks).

Whilst we thankfully moved away from counting lines of code as a measure of productivity long ago, we need to ensure we don't inadvertently bring it back as some measure of success for AI code assistants.

[ColinEberhardt]

From an onboarding perspective one of the things that stands out to me is the relative distance from the GenAI incubation work that tends to take place in most banks innovation centres and the operational risk assessments needing to be conducted by Information Security. All too often Innovation is understandably early without the benefit of fully formed learnings and engineering feedback whilst Info Sec is harried with yet another emerging tech to ramp up on.

Hi @miglaros - completely agree, and this is the issue we want to tackle through this group.

[ashukla13]

The topic that the DevOps SIG was interested in discussing at the next meeting is finos/devops-automation#181
This is a narrow subset of the broader set of issues around AI Readiness

[vicenteherrera]

I'm very interested in collaborating with this, as we at Control Plane are right now doing research on AI security.

[miglaros]

Perhaps we can draw inspiration from this pattern and establish some sample code/demo application/reference implementation: https://mlops.community/how-to-build-a-knowledge-assistant-at-scale/

[mindthegab]

Thank you @ColinEberhardt and everyone involved in this proposal. I am happy to report that the group was successfully unanimously approved by the FINOS Governing Board on Monday - so we're off to the races! Thank you for the overwhelming interest and support.

The group was approved to start as FINOS Member only for its initial formation phase (3/6 months) and then go fully open, so if you are a FINOS member please do reach out to @ColinEberhardt or to membersuccess@finos.org to be added to the meetings and activities. If you are not a member and you'd like to join as FINOS member you can do so here or reach out to membership@finos.org.

We will be announcing this publicly, as part of the broader AI Strategic Initiative, in a couple of weeks, so appreciate coordinating with marketing@finos.org for further public promotion of this activity.

Thanks again for your support and looking forward to another amazingly impactful collaboration here!

[satyampsoni]

How to get involved ?

[ColinEberhardt]

Hi @satyampsoni you can find further information here: https://github.com/finos/ai-readiness

[salmenus]

Hi everyone. I worked on trading tech at Goldman, BofA, and other banks, and contributed to AG Grid.
I recently started an OS project to simplify the integration of Gen AI into web apps (fin use-case here).

I've been thinking about some of the topics raised here — issues like output filtering, supporting explainability, cross-applications Gen AI integration, etc; I'll be happy to contribute to the conversation.

Thanks @mcleo-d for pointing me to this conversation and this SIG.