Analysis task - genereate safety themes from safety issuess

[1jamesthompson1]

Currently the model has generated general themes based off of the "ThemeSummary" generated for each report.

These theme summaries do not conform to the TAIC investigation model. Therefore it would be good to get some overall safety themes after reading all of the safety issues.

This will be completed as a oneoff analysis task. The answer g iven and it could be intergrated into the engine.

[1jamesthompson1]

After a dicussion aroudn the robustness and unsuccessfulness of using the llm a more transpraent method will be investigiated.

This will involve using clustering of the embeded safety issues.

From there we could see what the clusters are.

[1jamesthompson1]

I have developed some potential models that do a reasonable job of clustering the safety issues.

There are individual models for each mode, merged models and topic models fit to all of the safety issues.

Becuase I am working with such a low amount of documents I expect it to be much more sophisticated and powerful when more safety issues are added to the dataset from other organisations.

[1jamesthompson1]

Safety themes models seemed to be good enough. I need to try and create some simple 2d graph that can show all of the safety issues and hteh associated safety themes.

[1jamesthompson1]

I am preforming an exhatstive hyper parameter search to see what the best topic model will be.

Then I will find the one that has the lowest number of outliers as well as the most resaonable looking safety themes.

[1jamesthompson1]

Meeting with TAIC has come up with these points:

• A transport investigator can be acquired to do some human validation of the topic models.
• Try out adding context to the safety issues and see what happens with the safety issues.
• Use the metric of how 'confident' i.e how close all issues are in a cluster.

[1jamesthompson1]

The feedback on these demo models have some actionable items.
Theme_generation_model_evalulation_for_.zip

• Work on having more themes. Having more themes will result in each theme being more specific
• Work on representation and see if a description could be provided for the themes

[1jamesthompson1]

After meeting with TAIC this goal has been put on hold so that focus can be put on the searching problem #146.