Execute Lit Agent from run file with passable config based on pydantic settings

[jbrry]

Summary 📝

This PR enables running an agent from a python file and passing dynamic configs at run time.

Details

Introduces scripts and config folders to store run scripts and associated configs.
Uses pydantic settings to create settings/config objects which can have various levels of abstractions, e.g. project settings, or lit agent settings.

The idea is top-level project settings will be passed to /extracted from ProjectSettings and agent-specific settings in LitAgentSettings, for example.

ProjectSettings takes in values from a .env file using model_config.
LitAgentSettings is initialised via a toml file, which can be changed between runs.

[NISH1001]

I think instead of creating the new settings class for each tool in the config, maybe we could import the config and create those objects like:

class SomeSettings(BaseSettings):
    model_settings: ...
    search_settings: SearxNgToolConfig(...)
    scraper_settings: SearxNgToolConfig(...)

etc. This way, we will have config for each tool and agents tightly coupled with their module scope. What do you think.

I liked the toml way of loading as well. It's neat. Maybe we can find some common grounds for this? But yours look better with global scope.

[jbrry]

Yes, I was actually thinking of whether to use the config object in the settings class directly, vs. going from purely settings parameters which are then passed into the config object at a later point.

I think this simplifies it a bit and has the benefit that all of the agent/tool settings will be instantiated in one place. I can refactor to incorporate this change.

Glad you think the toml approach might work -- yes we can discuss some approaches to handle passing different parameters when running the agent, and try to find some common ground between approaches.

[jbrry]

I updated the lit_config.py to use the settings directly from the appropriate tools.

I'm not 100% sure on the file names etc. but I think the general idea of the PR is that the CONFIG object can still be imported where necessary throughout the codebase (reference).

There is now a Settings object for the agent, e.g. LitAgentSettings, which incorporates all the agent-specific settings, like scraper, search tool etc. This way, if we add another agent, we can compose a settings object of the necessary configs. Happy to let this evolve as the use-case/code changes though.

[NISH1001]

Thanks. I will have a look at this once I merge the factreasoner PR.

[NISH1001]

We might have to pin down pyproject pydantic deps to:

    "pydantic>=2.10.3",
    "pydantic-settings==2.9.1",