feat: Implement dynamic MCP server integration with authentication

[eranco74]

Description

• Add ModelContextProtocolServer configuration model
• Add MCP server registration during app startup lifecycle
• Update query handler to use configured MCP tools with authentication
  • Integrate Agent with dynamic MCP tool registration from config
  • Implement access token forwarding to MCP servers via headers
• Refactor auth dependency to extract and return access tokens
• Migrate from startup events to lifespan context manager

This enables dynamic registration and authentication of multiple MCP servers through configuration without hardcoded endpoints.

Type of change

• Refactor
• New feature
• Bug fix
• CVE fix
• Optimization
• Documentation Update
• Configuration Update
• Bump-up service version
• Bump-up dependent library
• Bump-up library or tool used for development (does not change the final image)
• CI configuration change
• Konflux configuration change
• Unit tests improvement
• Integration tests improvement
• End to end tests improvement

Related Tickets & Documents

• Related Issue #
• Closes #

Checklist before requesting a review

• I have performed a self-review of my code.
• PR has passed all pre-merge test jobs.
• If it is a core feature, I have added thorough tests.

Testing

• Please provide detailed steps to perform tests related to this code change.
• How were the fix/results from this change verified? Please provide relevant screenshots or results.

[manstis]

I welcome this PR as we're dynamically registering MCP Severs with a llama-stack server.

In order to move to lightspeed-stack we will also need this facility.

I wonder however whether the authentication should/could be moved to a separate PR.

The approach proposed in this PR does not fulfil our requirements (of providing different headers for different MCP servers).

[eranco74]

we're dynamically registering MCP Severs with a llama-stack server.

In order to move to lightspeed-stack we will also need this facility.

I wonder however whether the authentication should/could be moved to a sep

Thanks for the review.
You are right that the authentication pice cover one case and won't work for mcp servers that require anything other than access token, yet I added it because without it we can register MCP servers all we want but as long they require any auth it won't work, so I figured it's better start with something basic that works e2e.

[tisnik]

@eranco74 if you like, I will add you as maintainer for the core, so all the CI tests will run automatically. Agreed?

[manstis]

Sweet LGTM 👍

However, since I'm not a maintainer of lightspeed-stack defer approval to @tisnik

[tisnik]

it looks ok!
Please:

1. change from draft to ready for review state
2. accept invitation, so your changes will trigger CI (TBH dunno if it works for existing PRs)
3. try to fix linter errors, seems like pretty straightforward

LGTM instead

[tisnik]

Awesome!