v1.2.0

📦 Overview

The NeMo Agent toolkit, formerly known as Agent Intelligence (AIQ) toolkit, has been renamed to align with the NVIDIA NeMo family of products. This release brings significant new capabilities and improvements across authentication, resource management, observability, and developer experience. The toolkit continues to offer backwards compatibility, making the transition seamless for existing users.

While NeMo Agent Toolkit is designed to be compatible with the previous version, users are encouraged to update their code to follow the latest conventions and best practices. Migration instructions are provided in the migration guide.

🚨 Breaking Changes

• Remove outdated/unsupported devcontainer by @dagardner-nv in #626
• Rename aiq namespace to nat by @dagardner-nv in #618
• Update AIQ to NAT in documentation and comments by @yczhang-nv in #614
• Remove AIQ prefix from class and function names by @dagardner-nv in #606
• Rename aiqtoolkit packages to nvidia-nat by @dagardner-nv in #598
• Observability redesign to reduce dependencies and improve flexibility by @mpenn in #379

🚀 Notable Features and Improvements

• Authentication for Tool Calling: Implement robust authentication mechanisms that enable secure and configurable access management for tool invocation within agent workflows.
• Test Time Compute: Dynamically reallocate compute resources after model training, allowing agents to optimize reasoning, factual accuracy, and system robustness without retraining the base model.
• Sizing Calculator: Estimate GPU cluster requirements to support your target number of users and desired response times, simplifying deployment planning and scaling.
• Object Store Integration: Connect and manage data through supported object stores, improving agent extensibility and enabling advanced data workflows.
• Enhanced Cursor Rules: Build new workflows or extend existing ones by leveraging cursor rules, making agent development faster and more flexible.
• Interactive Notebooks: Access a suite of onboarding and example notebooks to accelerate agent workflow development, testing, and experimentation.
• Observability Refactor: Onboard new observability and monitoring platforms more easily, and take advantage of improved plug-in architecture for workflow inspection and analysis.
• Examples Reorganization: Organize examples by functionality, making it easier to find and use the examples.

📜 Full Change Log

• Use consistent casing for ReAct agent by @dagardner-nv in #293
• Update alert triage agent's prompt by @hsin-c in #297
• Move Wikipedia search to separate file by @jkornblum-nv in #237
• Release documentation fixes by @dagardner-nv in #300
• Add a pyproject.toml to simple_rag example allowing for declared dependencies by @dagardner-nv in #284
• Update version in develop, in prep for the next release by @AnuradhaKaruppiah in #294
• Add field validation for the evaluate API by @dagardner-nv in #311
• Intermediate steps: evaluation fix by @titericz in #312
• Fix or silence warnings emitted by tests by @dagardner-nv in #305
• Add documentation for load_workflow() by @yczhang-nv in #306
• Adding pytest-pretty for nice test outputs by @benomahony in #194
• feat(telemetry): add langfuse and langsmith telemetry exporters #233 by @briancaffey in #235
• Check links in markdown files by @dagardner-nv in #323
• Eval doc updates by @AnuradhaKaruppiah in #322
• Add unit tests for the alert triage agent example by @hsin-c in #252
• Add support for AWS Bedrock LLM Provider by @yczhang-nv in #238
• Add missing import in load_workflow documentation by @yczhang-nv in #329
• propose another solution to problem[copy] by @LunaticMaestro in #298
• Support additional_instructions by @gfreeman-nvidia in #302
• Update installing.md by @manny-pi in #316
• Add an async version of the /generate endpoint by @dagardner-nv in #315
• Update trajectory eval documentation by @hsin-c in #338
• Rename test mode to offline mode by @hsin-c in #343
• Simplify offline mode with aiq eval by @hsin-c in #344
• fix mcp client schema creation in flat lists by @slopp in #346
• Refactor for better prompt and tool description organization by @hsin-c in #350
• Extend IntermediateStep to support tool schemas in tool calling LLM requests by @mpenn in #357
• Fix AttributeError bug for otel_telemetry_exporter by @ZhongxuanWang in #335
• Update OpenAIModelConfig to support stream_usage option by @mdemoret-nv in #328
• Rename to NeMo Agent toolkit by @dagardner-nv in #359
• fix(phoenix): set project name when using phoenix telemetry exporter (#337) by @briancaffey in #347
• Account for the "required fields" list in the mcp_input_schema by @AnuradhaKaruppiah in #360
• Provide a config to pass the complete dataset entry as an EvalInputItem field to evaluators by @AnuradhaKaruppiah in #355
• Simplify custom evaluator definition by @AnuradhaKaruppiah in #358
• Add Patronus OTEL Exporter by @hersheybar in #341
• Expand Alert Triage Agent Offline Dataset by @hsin-c in #369
• Add Custom Classification Accuracy Evaluator for the Alert Triage Agent by @hsin-c in #373
• Add Cursor rules to improve Cursor support for development by @yczhang-nv in #319
• Added LLM retry logic to handle rate limiting LLM without frequent Exception by @liamy-nv in #368
• Fixes Function and LambdaFunction classes to push active function instance names by @mpenn in #374
• TunableRagEvaluator: Re-enable inheriting from the base abc by @AnuradhaKaruppiah in #375
• Add Job ID Appending to Output Directories and Maximum Folders Threshold by @ZhongxuanWang in #331
• Add support for custom functions in bottleneck analysis by @dnandakumar-nv in #380
• Persist chat conversation ID for workflow tool usage by @ericevans-nv in #326
• Add support for Weave evaluation by @ayulockin in #264
• Update the information displayed in the Weave Eval dashboard by @AnuradhaKaruppiah in #390
• Allow non-json string outputs for workflows that use unstructured datasets by @AnuradhaKaruppiah in #396
• Add aws region config for s3 eval uploads by @munjalp6 in #397
• add support for union types in mcp client by @cheese-head in #372
• Add percentile computation (p90, p95, p99) to profiling by @dnandakumar-nv in #398
• Ragas custom evaluation field in evaluator by @dnandakumar-nv in #400
• Reorganize the examples into categories and improve re-use of example components by @mpenn in #411
• Improve descriptions in top level examples README.md by @mpenn in #415
• Add ragaai catalyst exporters by @vishalk-06 in #395
• Update MCP version by @dagardner-nv in #417
• feature request: Add galileo tracing workflow by @franz101 in #404
• Update index.md by @sugsharma in #420
• Windows compatibility for temp file handling by @ericevans-nv in #423
• Sizing calculator to estimate the number of GPU for a target number of users by @AnuradhaKaruppiah in #399
• Update and move W&B Weave Redact PII example by @yczhang-nv in #424
• Refactor IntermediateStep parent_id for clarification by @mdemoret-nv in #330
• Add Cursor rules for latinisms by @yczhang-nv in #426
• Resolve examples organization drift by @mpenn in #429
• NeMo Agent rename by @lvojtku in #422
• Removes redundant config variable from Retry Agent Function. by @ericevans-nv in #447
• Add otelcollector doc and example by @slopp in #451
• Improve error logging during workflow initialization failure by @yczhang-nv in #464
• Added an AIQToolkit function that can be invoked to perform a simple completions task, given a natural language prompt. by @sayalinvidia in #460
• Improve MCP error logging with connection failures by @mpenn in #470
• Enable testing tools in isolation by @yczhang-nv in #391
• Refactor examples to improve discoverability and improve uniformity by @mdemoret-nv in #476
• Add Inference Time Scaling Module by @dnandakumar-nv in #381
• Refactor Agno Personal Finance Function and Update Configuration by @mpenn in #477
• Add object store by @balvisio in #299
• Observability redesign to reduce dependencies and improve flexibility by @mpenn in #379
• Adding OpenAI Chat Completions API compatibility by @dfagnou in #421
• Enhance code execution sandbox with improved error handling and debugging by @vikalluru in #409
• Fixing inheritance on the OTel collector exporter and adding project name by @mdemoret-nv in #481
• Refactor retry mechanism and update retry mixin field config by @dnandakumar-nv in #480
• Integrate latest RetryMixin fixes with aiqtoolkit_agno subpackage by @mpenn in #483
• Fix incorrect file paths in simple calculator example by @mpenn in #482
• Documentation edits for sizing calculator by @lvojtku in #436
• feat(redis): add redis memory backend and redis memory example #376 by @briancaffey in #377
• Improve error handling and recovery mechanisms in agents by @yczhang-nv in #418
• Update git clone under /doc folder to point to main branch by @yczhang-nv in #484
• Pin datasets version in toplevel pyproject.toml by @willkill07 in #487
• Fix otelcollector to ensure project name is added to OtelSpan resource + added weave cleanup logic by @mpenn in #488
• Fix shared field reference bug in TypedBaseModel inheritance by @mpenn in #489
• Streamlining API Authentication by @ericevans-nv in #251
• Add experimental decorator to auth and ITS strategy methods by @dnandakumar-nv in #493
• Unify examples README structure by @willkill07 in #485
• Cleanup authorization settings to remove unnecessary options by @mdemoret-nv in #495
• Move WeaveMixin._weave_calls to IsolatedAttribute to avoid cleanup race conditions by @mpenn in #499
• Set SCM versioning for text_file_ingest allowing it to be built in CI by @dagardner-nv in #501
• Update PII example to improve user experience and WeaveExporter robustness by @mpenn in #502
• Fix pyproject.toml for text_file_ingest example by @dagardner-nv in #505
• Update ci/checks.sh to run all of the same checks performed by CI by @dagardner-nv in #506
• Suppress stack trace in error message in ReActOutputParserException by @yczhang-nv in #507
• Clarify intermediate output formatting in agent tool_calling example by @willkill07 in #504
• Fix fastapi endpoint for plot_charts by @willkill07 in #508
• Fix UI docs to launch the simple calculator by @dagardner-nv in #511
• Update prerequisite and system prompt of redis memory example by @yczhang-nv in #510
• Fix HITL por_to_jiratickets example by @dagardner-nv in #515
• Relax overly restrictive constraints on AIQChatRequest model by @ericevans-nv in #512
• Fix file paths for simple_calculator_eval by @willkill07 in #517
• Fix: getting_started docker containers build with added compiler dependency by @willkill07 in #518
• Documentation: Specify minimum uv version by @willkill07 in #520
• Fix Simple Calculator HITL Example. by @ericevans-nv in #519
• Fix outdated path in pyproject.toml in text_file_ingest by @yczhang-nv in #524
• Fix issue where aiq fails for certain log levels by @dagardner-nv in #523
• Update catalyst readme document by @vishalk-06 in #492
• Fix outdated file references under /examples by @yczhang-nv in #526
• Misc Documentation cleanups by @dagardner-nv in #527
• Misc cleanups/fixes for installing.md document by @dagardner-nv in #528
• Fix: file path references in examples and docs by @willkill07 in #536
• Resolve batch flushing failure during SpanExporter cleanup by @mpenn in #532
• Fix typos in the observability info commands by @AnuradhaKaruppiah in #529
• Publish the linear fit data in the CalcRunner Output by @AnuradhaKaruppiah in #498
• Restructure example README to fully align with reorganization by @willkill07 in #539
• Add example for Vulnerability Analysis for Container Security Blueprint by @ashsong-nv in #530
• Misc cleanups for docs/source/quick-start/launching-ui.md by @dagardner-nv in #537
• Fix grammar error in uninstall help string by @dagardner-nv in #540
• Fix: custom routing example typos and output clarification by @willkill07 in #541
• Update the UI submodule to adopt fixes by @dagardner-nv in #543
• Fix: Examples README output clarifications; installation command by @willkill07 in #542
• Ensure type system and functional behavior are consistent for to_type specifications by @willkill07 in #513
• Documentation: update memory section to include redis; fix code references by @willkill07 in #544
• Update the dataset in the swe-bench README by @AnuradhaKaruppiah in #546
• Fix alert triage agent documentation on system output by @hsin-c in #545
• Fix several dependency and documentation issues under /examples by @yczhang-nv in #547
• Fixes to the add-tools-to-a-workflow.md tutorial by @dagardner-nv in #548
• Example: update swe_bench README to reflect output changes by @willkill07 in #550
• Update Cursor rules and documentations to remove unnecessary installation checks by @yczhang-nv in #549
• Update object_store example to use NVIDIA key instead of missing OPENAI key by @willkill07 in #552
• Remove deprecated code usage in the por_to_jiratickets example by @dagardner-nv in #557
• Fix simple_auth link to UI repository by @willkill07 in #553
• Update the LangSmith environment variable names in simple_calculator_observability example by @dagardner-nv in #558
• Improvements to extending telemetry exporters docs by @mpenn in #554
• Misc cleanups for create-a-new-workflow.md document by @dagardner-nv in #556
• Reduce the number of warnings logged while running the getting_started examples by @dagardner-nv in #563
• Update observability system documentation to reflect modern architecture and remove snippets by @mpenn in #562
• Add docker container for oauth server to fix configuration issues by @ericevans-nv in #561
• Ensure imports are lazily loaded in plugins improving startup time by @dagardner-nv in #564
• General improvements to observe-workflow-with-catalyst.md to improve experience by @mpenn in #555
• Update Simple Auth Example Config File Path by @ericevans-nv in #566
• Convert cursor_rules_demo GIF files to Git LFS by @yczhang-nv in #567
• UI Submodule Update by @ericevans-nv in #568
• Restructure agents documentation by @dagardner-nv in #569
• Increase package/distro resolution in DiscoveryMetadata to improve utility of aiq info components by @mpenn in #574
• Minor cleanups for the run-workflows.md document by @dagardner-nv in #572
• Add CI check for path validation within repository by @willkill07 in #573
• Improvements to observability plugin documentation by @mpenn in #578
• Fixes for adding-an-authentication-provider.md by @dagardner-nv in #579
• Minor cleanups for sizing-calc.md by @dagardner-nv in #577
• minor doc update to pass lint by @gfreeman-nvidia in #582
• Fixing missing space preventing proper render of snippet in markdown by @mpenn in #581
• Fixing wrong override usage to make it compatible with py 3.11 by @vikalluru in #585
• Updating UI submodule by @ericevans-nv in #588
• Fixes for api-authentication.md by @dagardner-nv in #583
• Object Store code, documentation, and example improvements by @willkill07 in #587
• Fix module discovery errors when publishing with registry handlers by @mpenn in #592
• Update logging levels in ProcessingExporterand BatchingProcessor to reduce shutdown noise by @mpenn in #589
• Update template to use logger instead of print by @hsin-c in #590
• Fix fence indenting, remove ignore pattern by @dagardner-nv in #594
• Remove Unused Authentication Components from Refactor by @ericevans-nv in #596
• Minor cleanup to using-local-llms.md by @dagardner-nv in #595
• Merge Post VDR changes by @dagardner-nv in #597
• Rename aiqtoolkit packages to nvidia-nat by @dagardner-nv in #598
• Rename Inference Time Scaling to Test Time Compute by @dnandakumar-nv in #600
• CI: upload script updated to set the artifactory path's top level dir by @AnuradhaKaruppiah in #602
• Rename ITS tool functions to TTC tool functions by @dnandakumar-nv in #605
• Fix the artifactory component name to aiqtoolkit for all packages by @AnuradhaKaruppiah in #603
• Fix Pylint in CI by @dagardner-nv in #609
• Remove AIQ prefix from class and function names by @dagardner-nv in #606
• Add support for synchronous LangChain tool calling by @mpenn in #612
• Send Conversation ID with WebSocket Messages by @ericevans-nv in #613
• Adds support for MCP server /health endpoint, custom routes and a client mcp ping command by @AnuradhaKaruppiah in #576
• Updating UI submodule by @ericevans-nv in #622
• Rename aiq namespace to nat by @dagardner-nv in #618
• Remove outdated/unsupported devcontainer by @dagardner-nv in #626
• Use issue types instead of title prefixes by @dagardner-nv in #619
• Fixes to weave telemetry exporter to ensure traces are properly sent to Weave by @mpenn in #627
• Apply work-around for #621 to the gitlab ci scripts by @dagardner-nv in #630
• Revert unintended change to artifactory_upload.sh by @dagardner-nv in #631
• Bugfix (Object Store): remove unnecessary S3 refs in config; fix mysql upload script by @willkill07 in #628
• Refactor embedder client structure for LangChain and Llama Index. by @dnandakumar-nv in #634
• Documentation: Update Using Local LLMs by @willkill07 in #623
• Align WebSocket Workflow Output with HTTP Output by @ericevans-nv in #635
• Update AIQ to NAT in documentation and comments by @yczhang-nv in #614
• Documentation(Providers): Surface LLM; add Embedders and Retrievers by @willkill07 in #625
• CI: improve path-check utility; fix broken links; add more path check rules by @willkill07 in #601
• Fix broken additional_instructions options for ReWOO agent by @yczhang-nv in #640
• Updating ui submodule by @ericevans-nv in #641
• Fix symlink structure to be consistent across all examples by @willkill07 in #642
• Fix: add missing uv.source for simple_calculator_hitl by @willkill07 in #638
• Enable datasets with custom formats by @AnuradhaKaruppiah in #615
• Update uv.lock by @dagardner-nv in #644
• Run CI for commits to the release branch by @dagardner-nv in #645
• Add a note that the dataset needs to be uploaded to the S3 bucket by @AnuradhaKaruppiah in #646
• Add UI documentation links and installation instructions by @ericevans-nv in #647
• Consolidate CI pipelines by @dagardner-nv in #632
• Install git-lfs in docs CI stage by @dagardner-nv in #648
• Fix aiq compatibility by @dagardner-nv in #651
• Bugfix: Align Python Version Ranges by @willkill07 in #655
• Docs: Add Migration Guide by @willkill07 in #653
• Update third-party-license files for v1.2 by @dagardner-nv in #657
• Add notebooks to show users how to get started with the toolkit and build agents by @cdgamarose-nv in #656
• Remove redundant prefix from directory names by @dagardner-nv in #665
• Docs: Add Upgrade Fix to Troubleshooting by @willkill07 in #659
• Enhance README with badges, installation, instructions, and a roadmap by @mdemoret-nv in #654
• Adding .nspect-allowlist.toml to remediate false positives found by scanner by @dagardner-nv in #668
• Fix: Remove pickle from MySQL-based Object Store by @willkill07 in #669
• Enable BUILD_NAT_COMPAT by @dagardner-nv in #670
• Fix paths for compatibility packages by @dagardner-nv in #672
• Add missing compatibility package for aiqtoolkit-weave by @dagardner-nv in #674
• Add chat_history to the context of ReAct and ReWOO agent by @yczhang-nv in #673

🙌 New Contributors

• @titericz made their first contribution in #312
• @benomahony made their first contribution in #194
• @briancaffey made their first contribution in #235
• @LunaticMaestro made their first contribution in #298
• @gfreeman-nvidia made their first contribution in #302
• @manny-pi made their first contribution in #316
• @slopp made their first contribution in #346
• @ZhongxuanWang made their first contribution in #335
• @hersheybar made their first contribution in #341
• @munjalp6 made their first contribution in #397
• @cheese-head made their first contribution in #372
• @vishalk-06 made their first contribution in #395
• @franz101 made their first contribution in #404
• @sugsharma made their first contribution in #420
• @lvojtku made their first contribution in #422
• @sayalinvidia made their first contribution in #460
• @dfagnou made their first contribution in #421
• @vikalluru made their first contribution in #409
• @ashsong-nv made their first contribution in #530