fix(llma): LangChain 1.0+ compatibility for CallbackHandler

[andrewm4894]

Problem

The PostHog LangChain CallbackHandler is incompatible with LangChain 1.0+ due to deprecated import paths. LangChain 1.0 (released September 2024) moved modules from langchain.callbacks.base to langchain_core.callbacks.base.

Fixes #362

Solution

Added backward-compatible imports using try/except:

• First attempts to import from langchain_core (LangChain 1.0+ and modern 0.x)
• Falls back to legacy langchain imports for older versions

This approach maintains compatibility with both LangChain 0.x and 1.0+.

Testing

• All existing tests pass (45 → 46 passed, 8 skipped due to missing API keys)
• Added regression test for AgentAction/AgentFinish imports
• Verified with LangChain 1.0.3 in production environment
• PostHog LLM analytics events successfully captured with LangChain 1.0+

Test Coverage Improvement

Added test_agent_action_and_finish_imports() to address a gap in test coverage:

• Previous tests imported from langchain_core but never tested AgentAction/AgentFinish
• The only test using these types (test_langgraph_agent) was skipped without API keys
• New test explicitly verifies the imports work and callbacks function correctly
• Prevents future regressions of this compatibility issue

Changes

# Before (lines 23-24)
from langchain.callbacks.base import BaseCallbackHandler
from langchain.schema.agent import AgentAction, AgentFinish

# After (lines 23-30)
try:
    from langchain_core.callbacks.base import BaseCallbackHandler
    from langchain_core.agents import AgentAction, AgentFinish
except (ImportError, ModuleNotFoundError):
    from langchain.callbacks.base import BaseCallbackHandler
    from langchain.schema.agent import AgentAction, AgentFinish

[greptile-apps]

Additional Comments (1)

1. posthog/ai/langchain/callbacks.py, line 31-41 (link)

   logic: inconsistent import strategy - these langchain_core imports will fail on older LangChain versions where langchain_core isn't available, while lines 23-30 have fallbacks

   either these imports already worked in the previous version (suggesting langchain_core was always available), or they also need try/except fallbacks for true backward compatibility

_{1 file reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[andrewm4894]

Follow-up: CI Improvement Suggestion

While implementing this fix, we noticed that CI currently only tests with LangChain 1.0+ (the versions in pyproject.toml test dependencies are langchain-core>=0.3.65, etc.). This means backward compatibility with LangChain 0.x is not tested in CI.

Proposed Enhancement

Add a matrix dimension to test both LangChain versions:

tests:
    name: Python ${{ matrix.python-version }} / LangChain ${{ matrix.langchain-version }} tests
    runs-on: ubuntu-latest
    strategy:
        matrix:
            python-version: ['3.9', '3.10', '3.11', '3.12', '3.13']
            langchain-version: ['0.2', '1.0']  # NEW

    steps:
        # ... existing steps ...
        
        - name: Install test dependencies
          shell: bash
          run: |
              UV_PROJECT_ENVIRONMENT=$pythonLocation uv sync --extra test

        - name: Install LangChain ${{ matrix.langchain-version }}
          shell: bash
          run: |
              if [ "${{ matrix.langchain-version }}" = "0.2" ]; then
                # LangChain 0.2.x (pre-1.0)
                uv pip install --system 'langchain>=0.2.0,<1.0' 'langchain-community>=0.2.0,<0.3.0' 'langchain-openai>=0.1.0,<0.2.0' 'langchain-anthropic>=0.1.0,<0.2.0'
              else
                # LangChain 1.0+ already installed from test dependencies
                echo "Using LangChain 1.0+ from test dependencies"
              fi

        - name: Run posthog tests
          run: |
              pytest --verbose --timeout=30

Benefits

• Catches compatibility issues like this bug before they ship
• Tests the backward-compatible import pattern we just added
• Ensures both langchain and langchain_core import paths work
• Provides confidence when bumping version requirements

Trade-offs

• Doubles the test matrix (5 Python versions × 2 LangChain versions = 10 jobs instead of 5)
• Increases CI time slightly

Happy to implement this as a follow-up PR if the team thinks it's valuable!

[Radu-Raicea]

@andrewm4894
After our conversation, what's your decision on whether to continue supporting LC <1.0 or not?

[andrewm4894]

After our conversation, what's your decision on whether to continue supporting LC <1.0 or not?

I think for now it makes total sense to be backwards compatible and support both. As little friction for users as possible, its not that complex yet (if complexity grows can revisit). A lot of stuff last 12 months built on LC <1.0 so we should i think on balance be as flexible and easy for next 6 months or so and then can probably bump req to LC 1.0+ once most of the community on it.

[Radu-Raicea]

I'm approving this, but you need to bump the version first, before you merge.