Feat/prediction with rules and report

[jhehemann]

This pull request adds three tools to the mech agent. The prediction tool "prediction_with_rules_and_report" and the two supporting tools "infer_market_rules" and "research" that the prediction tool depends on.

RESEARCH:
This tool generates summaries of the most relevant content from the most relevant websites for answering the market question. The workflow is as follows:

1. Search queries are generated and the most relevant are selected
2. Selected URLs are scraped and the text is separated into chunks
3. These chunks are embedded and sorted
4. The overall top chunks are selected and grouped by the websites they belong to
5. The grouped chunks are summarized separately with respect to the market question
6. The resulting summaries are combined and "summarized" as a whole again to remove redundancies and irrelevant paragraphs while still preserving the reference to the summaries original website.
7. The summary of summaries is outputted in a structured format where each summary is attached with its publication date and publisher.

INFER_MARKET_RULES:
It was observed that when gpt-3.5 is used to make predictions it struggles to conclude the correct outcome of a market question even if the event's date is mentioned in the search output. This tool infers status information and definitions that the question's phrasing implies and defines outcome rules based on the timing that is asked for in the question ("on", "by"). Few shot prompts are used here as it turned out that they manage to convey the subtle differences in phrasing the event, timeline, etc. best.

PREDICTION_WITH_RULES_AND_REPORT:
This tool uses the research tool and the infer_market_rules tool to get additional information. With this information a report is generated that analyses based on the research output as well as the status information and definitions from the infer_market_rules tool if and when the described event could happen. For this, the date is removed from the question to avoid any bias. A final prediction is made based on this report and the outcome rules from the infer_market_rules tool.

[jhehemann]

@jhehemann can you please merge main, and revert any third_party package changes to packages.json. Steps are:

1. Merge main to this branch.

2. Get third_party package hashes from main.

3. Run `autonomy packages sync --update-packages`

4. Run `autonomy packages lock`.

5. Commit changes.

@0xArdi Thanks for the instructions. I did all of this. It seems that the tool checks fail because the environment variables (API-keys) are not loaded in the workflow properly. Locally the tool checks succeed. I'm not sure how to fix this.

[jhehemann]

@jhehemann can you please merge main, and revert any third_party package changes to packages.json. Steps are:

1. Merge main to this branch.

2. Get third_party package hashes from main.

3. Run `autonomy packages sync --update-packages`

4. Run `autonomy packages lock`.

5. Commit changes.

@0xArdi Thanks for the instructions. I did all of this. It seems that the tool checks fail because the environment variables (API-keys) are not loaded in the workflow properly. Locally the tool checks succeed. I'm not sure how to fix this.

@0xArdi Have you had this issue before? Do you know how to fix this? Does this maybe have something to do with the python-dotenv package I removed from the pyproject.toml?

[dvilelaf]

@jhehemann are you still experiencing this issue? Can you merge main and push?