[Feature] Adding Lambda LLMs

[SibaRajendran]

Overview

This PR introduces a new LambdaLLM implementation that allows routing LLM requests through AWS Lambda functions. It also enhances the AWS utilities with async support and improves error handling with a robust retry mechanism.

Key Changes

• Added LambdaLLM implementation for routing LLM requests through AWS Lambda functions
• Enhanced BotoFactory with async session support via aioboto3
• Implemented a backoff-based retry mechanism with configurable exception handling
• Added proper typing to LLM classes to improve type safety
• Added new dependencies: glom and aioboto3
• Updated dspy to version 2.6.11

Implementation Details

• LambdaLLM uses both synchronous and asynchronous AWS clients for Lambda invocation
• Lambda requests use OpenAI-compatible message format for consistency
• Added retry utilities with exponential backoff strategy and configurable exception handling
• Enhanced type safety with proper generics in LLM implementations
• Extended configuration types to support Lambda function ARNs and retry logic

Testing

The PR includes updates to the local test script to demonstrate both Bedrock and Lambda inference options.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
Executing Chunk...:   0%|          | 0/10 [00:00<?, ?it/s]
Processing chunk 4:   0%|          | 0/10 [00:00<?, ?it/s]
Processing chunk 7:   0%|          | 0/10 [00:00<?, ?it/s]
Processing chunk 3:   0%|          | 0/10 [00:00<?, ?it/s]
Processing chunk 9:   0%|          | 0/10 [00:00<?, ?it/s]
Processing chunk 6:   0%|          | 0/10 [00:00<?, ?it/s]
Processing chunk 2:   0%|          | 0/10 [00:00<?, ?it/s]
Processing chunk 8:   0%|          | 0/10 [00:00<?, ?it/s]
Processing chunk 1:   0%|          | 0/10 [00:00<?, ?it/s]
Processing chunk 0:   0%|          | 0/10 [00:00<?, ?it/s]
Processing chunk 5:   0%|          | 0/10 [00:00<?, ?it/s]
Processing chunk 5:  10%|█         | 1/10 [00:03<00:30,  3.38s/it]
Processing chunk 9:  10%|█         | 1/10 [00:03<00:30,  3.42s/it]
Processing chunk 4:  10%|█         | 1/10 [00:06<00:56,  6.25s/it]
Processing chunk 7:  10%|█         | 1/10 [00:04<00:41,  4.63s/it]
Processing chunk 1:  10%|█         | 1/10 [00:06<00:55,  6.20s/it]
Processing chunk 0:  10%|█         | 1/10 [00:06<00:57,  6.41s/it]
Processing chunk 3:  10%|█         | 1/10 [00:06<00:59,  6.65s/it]
Processing chunk 9:  20%|██        | 2/10 [00:06<00:27,  3.47s/it]
Processing chunk 2:  10%|█         | 1/10 [00:07<01:04,  7.12s/it]
Processing chunk 6:  10%|█         | 1/10 [00:07<01:04,  7.22s/it]
Processing chunk 1:  20%|██        | 2/10 [00:07<00:26,  3.35s/it]
Processing chunk 8:  10%|█         | 1/10 [00:08<01:17,  8.60s/it]
Processing chunk 3:  20%|██        | 2/10 [00:08<00:32,  4.00s/it]
Processing chunk 4:  20%|██        | 2/10 [00:11<00:44,  5.50s/it]
Processing chunk 5:  20%|██        | 2/10 [00:09<00:39,  4.93s/it]
Processing chunk 7:  20%|██        | 2/10 [00:10<00:43,  5.40s/it]
Processing chunk 6:  20%|██        | 2/10 [00:11<00:45,  5.71s/it]
Processing chunk 0:  20%|██        | 2/10 [00:12<00:48,  6.12s/it]
Processing chunk 1:  30%|███       | 3/10 [00:12<00:28,  4.10s/it]
Processing chunk 8:  20%|██        | 2/10 [00:13<00:49,  6.23s/it]
Processing chunk 5:  30%|███       | 3/10 [00:13<00:31,  4.43s/it]
Processing chunk 9:  30%|███       | 3/10 [00:13<00:34,  4.86s/it]
Processing chunk 3:  30%|███       | 3/10 [00:13<00:31,  4.54s/it]
Processing chunk 4:  30%|███       | 3/10 [00:16<00:38,  5.54s/it]
Processing chunk 1:  40%|████      | 4/10 [00:14<00:20,  3.38s/it]
Processing chunk 7:  30%|███       | 3/10 [00:15<00:36,  5.20s/it]
Processing chunk 0:  30%|███       | 3/10 [00:15<00:31,  4.56s/it]
Processing chunk 2:  20%|██        | 2/10 [00:15<01:01,  7.67s/it]
Processing chunk 6:  30%|███       | 3/10 [00:17<00:39,  5.67s/it]
Processing chunk 9:  40%|████      | 4/10 [00:17<00:27,  4.63s/it]
Processing chunk 3:  40%|████      | 4/10 [00:18<00:26,  4.47s/it]
Processing chunk 5:  40%|████      | 4/10 [00:18<00:29,  4.84s/it]
Processing chunk 2:  30%|███       | 3/10 [00:18<00:40,  5.82s/it]
Processing chunk 8:  30%|███       | 3/10 [00:19<00:43,  6.25s/it]
Processing chunk 1:  50%|█████     | 5/10 [00:19<00:19,  3.94s/it]
Processing chunk 7:  40%|████      | 4/10 [00:20<00:30,  5.12s/it]
Processing chunk 0:  40%|████      | 4/10 [00:20<00:29,  4.88s/it]
Processing chunk 1:  60%|██████    | 6/10 [00:21<00:12,  3.16s/it]
Processing chunk 7:  50%|█████     | 5/10 [00:22<00:19,  3.90s/it]
Processing chunk 4:  40%|████      | 4/10 [00:24<00:37,  6.25s/it]
Processing chunk 3:  50%|█████     | 5/10 [00:22<00:20,  4.19s/it]
Processing chunk 6:  40%|████      | 4/10 [00:23<00:34,  5.75s/it]
Processing chunk 2:  40%|████      | 4/10 [00:23<00:32,  5.35s/it]
Processing chunk 5:  50%|█████     | 5/10 [00:23<00:24,  4.86s/it]
Processing chunk 0:  50%|█████     | 5/10 [00:24<00:22,  4.47s/it]
Processing chunk 8:  40%|████      | 4/10 [00:24<00:34,  5.83s/it]
Processing chunk 9:  50%|█████     | 5/10 [00:25<00:28,  5.72s/it]
Processing chunk 4:  50%|█████     | 5/10 [00:27<00:26,  5.37s/it]
Processing chunk 6:  50%|█████     | 5/10 [00:26<00:23,  4.76s/it]
Processing chunk 7:  60%|██████    | 6/10 [00:27<00:17,  4.34s/it]
Processing chunk 3:  60%|██████    | 6/10 [00:27<00:18,  4.68s/it]
Processing chunk 4:  60%|██████    | 6/10 [00:30<00:17,  4.25s/it]
Processing chunk 1:  70%|███████   | 7/10 [00:27<00:12,  4.23s/it]
Processing chunk 2:  50%|█████     | 5/10 [00:28<00:25,  5.12s/it]
Processing chunk 6:  60%|██████    | 6/10 [00:28<00:15,  3.90s/it]
Processing chunk 8:  50%|█████     | 5/10 [00:29<00:27,  5.41s/it]
Processing chunk 2:  60%|██████    | 6/10 [00:29<00:15,  3.94s/it]
Processing chunk 5:  60%|██████    | 6/10 [00:29<00:21,  5.32s/it]
Processing chunk 0:  60%|██████    | 6/10 [00:30<00:20,  5.08s/it]
Processing chunk 3:  70%|███████   | 7/10 [00:30<00:12,  4.21s/it]
Processing chunk 9:  60%|██████    | 6/10 [00:31<00:22,  5.70s/it]
Processing chunk 0:  70%|███████   | 7/10 [00:31<00:11,  3.93s/it]
Processing chunk 1:  80%|████████  | 8/10 [00:33<00:09,  4.56s/it]
Processing chunk 9:  70%|███████   | 7/10 [00:33<00:13,  4.64s/it]
Processing chunk 7:  70%|███████   | 7/10 [00:34<00:15,  5.11s/it]
Processing chunk 6:  70%|███████   | 7/10 [00:33<00:12,  4.30s/it]
Processing chunk 8:  60%|██████    | 6/10 [00:35<00:23,  5.76s/it]
Processing chunk 4:  70%|███████   | 7/10 [00:38<00:16,  5.58s/it]
Processing chunk 5:  70%|███████   | 7/10 [00:36<00:17,  5.69s/it]
Processing chunk 2:  70%|███████   | 7/10 [00:36<00:14,  4.83s/it]
Processing chunk 9:  80%|████████  | 8/10 [00:37<00:08,  4.31s/it]
Processing chunk 3:  80%|████████  | 8/10 [00:37<00:09,  4.85s/it]
Processing chunk 8:  70%|███████   | 7/10 [00:37<00:13,  4.50s/it]
Processing chunk 6:  80%|████████  | 8/10 [00:38<00:09,  4.52s/it]
Processing chunk 1:  90%|█████████ | 9/10 [00:38<00:04,  4.92s/it]
Processing chunk 0:  80%|████████  | 8/10 [00:38<00:09,  4.89s/it]
Processing chunk 6:  90%|█████████ | 9/10 [00:40<00:03,  3.71s/it]
Processing chunk 1: 100%|██████████| 10/10 [00:41<00:00,  4.14s/it]
Processing chunk 5:  80%|████████  | 8/10 [00:41<00:11,  5.60s/it]
Processing chunk 8:  80%|████████  | 8/10 [00:41<00:08,  4.38s/it]
Processing chunk 4:  80%|████████  | 8/10 [00:44<00:11,  5.66s/it]
Processing chunk 3:  90%|█████████ | 9/10 [00:42<00:04,  4.91s/it]
Processing chunk 9:  90%|█████████ | 9/10 [00:42<00:04,  4.66s/it]
Processing chunk 2:  80%|████████  | 8/10 [00:42<00:10,  5.38s/it]
Processing chunk 7:  80%|████████  | 8/10 [00:44<00:13,  6.76s/it]
Processing chunk 5:  90%|█████████ | 9/10 [00:45<00:05,  5.01s/it]
Processing chunk 8:  90%|█████████ | 9/10 [00:45<00:04,  4.17s/it]
Processing chunk 0:  90%|█████████ | 9/10 [00:45<00:05,  5.42s/it]
Processing chunk 3: 100%|██████████| 10/10 [00:46<00:00,  4.66s/it]
Processing chunk 6: 100%|██████████| 10/10 [00:46<00:00,  4.69s/it]
Processing chunk 4:  90%|█████████ | 9/10 [00:49<00:05,  5.52s/it]
Processing chunk 7:  90%|█████████ | 9/10 [00:47<00:05,  5.66s/it]
Processing chunk 2:  90%|█████████ | 9/10 [00:48<00:05,  5.34s/it]
Processing chunk 9: 100%|██████████| 10/10 [00:48<00:00,  4.88s/it]
Processing chunk 0: 100%|██████████| 10/10 [00:51<00:00,  5.11s/it]
Executing Chunk...:  10%|█         | 1/10 [00:56<08:24, 56.03s/it]
Processing chunk 5: 100%|██████████| 10/10 [00:51<00:00,  5.16s/it]
Processing chunk 7: 100%|██████████| 10/10 [00:53<00:00,  5.30s/it]
Processing chunk 4: 100%|██████████| 10/10 [00:55<00:00,  5.57s/it]
Processing chunk 8: 100%|██████████| 10/10 [00:53<00:00,  5.35s/it]
Processing chunk 2: 100%|██████████| 10/10 [00:53<00:00,  5.38s/it]
Executing Chunk...: 100%|██████████| 10/10 [00:58<00:00,  5.87s/it]
Process finished with exit code 0