The main repository for the 2023 hackathon with all the information for the hackathon and pointers to all repositories. Hackathon informaton pertaining to 2023 specifically will remain in here. The individual repositories are made to span multiple years. This years hackathon will focus on ensuring the proper foundation for growth is establed for each use case. This means the repositories created this year will be use again in subsequent years provided these use cases are repeated again. We need to ensure these solutions will lead to real world outcomes.
Bring developers from the Financial Services industry together to demonstrate how innovative Microsoft technologies can transform autism services by tackling real world use cases provided by families and creating lasting open-source projects for the community.
The first Financial Services Autism Hackathon was held in 2018 founded by Leo Junquera, working at Microsoft at the time, and Peter Smulovics from Morgan Stanley, as a grass roots effort to combine the unique value proposition of Microsoft (technology innovation) with the technology talent which exists in Financial Services, and direct it towards an industry in dire need of technology innovation. When Leo joined EY, he brought the event to EY and in 2022 the on-premise portion of the hackathon was held in the NY Wavespace and included professional service expertise from EY.
Leo's youngest son is autistic and this is what drove him to start the hackathon. Seeing how amazing technology and talented people could transform the financial services landscape, and wanting to bring that combination to the problems he saw at home and how services were deliverd to his son.
The date of the event is April 19th and 20th. The event will be hybrid. The in-person venue will be in NYC at the EY Headquarter in 1 Manhattan West during World Autism Awareness Month (April).
More information can be found on the site including registration.
There are two events:
- Autism Hackathon Day 1 - 4/19 9:00am to 4:00pm
- 9:00 - 9:45 Welcome, Introduction and breakfast
- 9:45 - 12:00 Hacking
- 12:00 - 12:45 Lunch
- 12:45 - 4:00 Hacking
- Autism Hackathon Day 2 - 4/20 9:00am to 4:00pm
- 9:00 - 9:30 Welcome back and breakfast
- 9:30 - 12:00 Hacking
- 12:00 - 12:45 Lunch
- 12:45 - 4:00 Hacking
- 4:00-4:10 - Introduction
- 4:10-4:30 - Use Case 1
- 4:30-4:50 - Use Case 2
- 4:50-5:10 - Use Case 3
- 5:10-5:30 - Use Case 4
- 5:30-5:50 - Use Case 5
- 5:50-6:00 - Closing
The following are the use cases for 2023. Each use case will have it's own repository with more specifics including the architecture.
Dynamic programs require adaptations to data collection which are not supported by these systems. In the end teams frequently resort back to pen, paper, and mechanical devices such as hand tally counters. Can we take a different approach to this problem using an IoT pattern? Can we find a way to simplify data collection using simple mechanisms and use the cloud to store the data and classify it after the fact using modern tools and technologies? Can there be a repository of patterns to match the individual needs of learners and care providers?
https://github.com/fsi-hack4autism/scenario-1
The team in 2022 made amazing progress in realizing this use case and it is close to a prodcution prototype. The goal for this year will be to build an alpha version which is ready for testing.
| Name | Criteria |
|---|---|
| Mobile App | |
| IoT Device | |
| Server | |
| Documentation |
| Name | Role | Company |
|---|---|---|
| GinSiu Cheng | Tech Lead | Microsoft |
| Sridher Manivel | Tech Lead | Microsoft |
| Rob Reese | Tech Lead | Microsoft |
| Thanda Oo | Tech Lead | EY |
Data collection is essential to successful outcomes for learners, but analysts are missing one of the most critical pieces of data when they are reviewing progress and creating programs, video of the learner. Social Media knows how important and compelling this information is but it is not being used to help people with Autism more effectively. This use case looks to develop a platform to capture this information, integrate it with data from other sources, and enrich the analysis with AI. This is an ambitious use case to transform the industry using a vital form of information which is currently missing.
https://github.com/fsi-hack4autism/video-learner-insight-transformation
Leadership Team:
| Name | Role | Company |
|---|---|---|
| Devanshi Thakar | Tech Lead | Microsoft |
| Pierre Mathieu | Tech Lead | Microsoft |
There is a long-held belief that each individual case of autism is so unique that comparisons cannot be drawn to other cases, but marketing companies are able to use data to target the right message to us at the right time to sell their product. The therapies used to help learners with autism are data intensive. Can we use this data to transform outcomes for learners using modern technology and methods?
Leadership Team:
| Name | Role | Company |
|---|---|---|
| Halaa Menasy | Tech Lead | Microsoft |
| Peter Shand | Tech Lead | Microsoft |
https://github.com/fsi-hack4autism/ML-To-Transform-Learner-Outcomes
This use case will focus on the use of HaloLens for helping to teach job skills to people with autism, particularly those aging out of the supports provided in the school environment. Unemployment for those with autism is significant and the supports in the work environment are limited. The use case will use HaloLens augmented reality to help with basic job skills by presenting a visual cues on how to complete task such as stocking shelves or preparing food. This scenario should also address the issue of moving away from 1:1 support to a 1:n as an a student becomes an adult.
Leadership Team:
| Name | Role | Company |
|---|---|---|
| Rishi Bhatnagar | Tech Lead | Morgan Stanley |
| Pardeep Singla | Tech Lead | Microsoft |
| Ted Dinsmore | Tech Lead (Spheregen) |
https://github.com/fsi-hack4autism/HoloLens-Skills-Training
Developing social relationships with peers can be one of the biggest challenges for people with autism, despite a strong desire to form them. Social programs can be hard to practice due to limited opportunities, which can lead to disappointing outcomes in the few interactions, and stress can and frustration. The Metaverse offers an opportunity to practice social interactions in a fully immersive environment and the ability to formulate successful programs which can repleted at scale. This use case will lay the groundwork for a social interaction in the Metaverse between a subject and an AI generated peer.
Leadership Team:
| Name | Role | Company |
|---|---|---|
| Domhnaill Hernon | Project Lead | EY |
| Tommy Sharkey | Tech Lead | EY |
| Alim Williams | Tech Lead | EY |
| Caroline Matthews | Tech Lead | Microsoft |
Read more here: https://github.com/fsi-hack4autism/Metaverse-Social-Practice
- Unity Development
- 3D Mesh Modeling and Texturing
- Lighting
- Rigging
- Mocap Acting
- Hand Animation
- Interaction and Experience Design
- Storyboard artists and video editors
- User Testing and Debugging
- Documentation
- OpenAI (ChatGPT)
- Speech-To-Text
- Text-To-Speech
- Visemes
- App Service / Virtual Machine
- Nvidia Omniverse (maybe - fingers crossed)
We often get asked "how can I participate" and have therefore outlined a list of roles and their responsiblities.
Curate resources and organize sessions in order to equip participants with the knowledge needed to prepare for the hackathon.
- Build technical training guide with learning geared towards each use case
- Put together contextual training guide focused on autism - current state, challenges, and gaps
- Organize and distribute training schedule, including “ask me anything” and technical/contextual skilling sessions
Serve as the administrator of a use case. Ensure that both the tech leads and participants have what they need before and during the hackathon.
- Support planning efforts and help in aligning 2-3 tech leads to use case
- Organize prep sessions leading up to the hackathon and ensure participants are connected with tech leads
- Oversee activity and logistics during the two-day hackathon
- Work with Tech Lead to capture summary/results
Serve as technical point of contact for a use case, reviewing output from previous years and helping to unblock technical barriers against use case objectives.
- Provide recommendations and help to finalize use case
- Lead prep sessions for participants and be available for technical questions
- Manage sponsored subscriptions and set up for participants
- Lead and guide team(s) during the two-day hackathon
- Work with Use Case Lead to capture summary/results
Organize marketing material for the Hackathon and generate interest internally and externally.
- Build a folder of references including:
- Word and PowerPoint branded templates
- Brochure to be distributed to interested external organizations
- One-pager highlighting the event
Ensure that everything is running smoothly across technical planning details of the Hackathon.
- Partner with individuals building registration platform to oversee and manage registration, maintaining organized list
- Point of contact to navigate any external access issues
- Create Team that includes organize training content, use case details, and channels and assist with adding/managing members
- Responsible for day-of operations, overseeing schedule and making sure everyone knows where they are supposed to be
Serve as the point person for a given customer organization, ensuring they have what they need to prepare their developers for the hackathon.
- Identify customer “champion” lead who will be your primary liaison for the hackathon
- Work with Marketing Lead and Training Leads to get content and resources to share with your customer
- Keep your customer informed on a regular basis on registration timelines, training sessions, and other event logistics