Garage week Project – Integration of Firefall for improvement suggestions

[dzehnder]

This PR implements the Garageweek Project to Fix Recommendations based on Code-Diffs using Firefall (https://adobe.brightidea.com/D50152)

Demo Script:

1. Trigger LHS report for website without any deggradations:
   Generate a baseline Lighthouse report for the website in its optimal state to establish performance benchmarks.
2. Make a controlled degradation
   Introduce a deliberate degradation in the website (e.g., adding a blocking while loop for a couple of seconds) to simulate a performance issue.

  // timeout for 2 seconds
  const end = Date.now() + 2 * 1000;
  while (Date.now() < end) {
    console.log("test")
  }

3. Trigger LHS report for degraded website
   Generate a new Lighthouse report for the website post-degradation.
4. Observe Slack channel that should show recommendations for the degradation
   Monitor the designated Slack channel for automated recommendations. The system should identify the degradation and suggest actionable recommendations to mitigate the identified issues.

Current PR status:

• Roundtrip possible:
  It is possible to trigger a LHS report using the API, the audit-worker conducts the actual test and sends the result via SQS to the post-processor. The post processor crafts a request to the Firefall API, which should return insights and recommendations based on the code diff, markdown diff and the calculated scores.
• Sending message to slack:
  After Firefall returns the insights and recommendations, a Slack message is crafted that visualizes the LHS differences and presents the actionable results from Firefall.

Next steps:

• Productize:
  During the Garage Week, the whole solution was implemented as a prototype. Before the merge it first needs to be transformed into a production-ready solution. This includes creating a Firefall Client that handles the API request and knows how to obtain the prompt, a module that creates the Slack message and a handler that stitches everything together.

• Reduce Requests to Firefall API
  We can reduce the amount of requests to the Firefall API a lot by identifying cases, where no recommendations are needed. For example when the score improved above a certain threshold (like > 90).

• Testing:
  For the Garage Week, test coverage was decreased to 0. This needs to be increased to 100 again. Every newly addd module needs rigorous testing to ensure reliability.

[github-actions]

This PR will trigger a minor release when merged.