Skip to content

Add LlmIntentClassifier and chat-to-proposal integration tests #577

@Chris0Jeky

Description

@Chris0Jeky

Parent

Part of #570 (Chat-to-proposal NLP gap)

Problem

The LlmIntentClassifier has zero dedicated test coverage. The chat-to-proposal flow through ChatService → classifier → ParseInstructionAsync is not tested end-to-end with natural language inputs.

Proposed Tests

LlmIntentClassifierTests (new file)

Unit tests documenting current behavior and gaps:

// Current behavior (should pass)
[Theory]
[InlineData("create card 'test'", true, "card.create")]
[InlineData("move card to done", true, "card.move")]
[InlineData("what's the weather?", false, null)]
public void Classify_CurrentPatterns(string input, bool expected, string? intent)

// Known gaps (documenting limitations — expected to fail until #571 is fixed)
[Theory]
[InlineData("can you create new onboarding tasks for non-technical people?")]
[InlineData("I need three new cards for the sprint")]
[InlineData("set up a project board for Q2 planning")]
public void Classify_NaturalLanguage_CurrentlyMisses(string input)

ChatService proposal flow tests

Integration tests verifying the end-to-end path with mock provider:

// Natural language → classifier misses → no proposal attempt (current behavior)
[Fact]
public async Task SendMessage_NaturalLanguage_NoProposalWhenClassifierMisses()

// Explicit RequestProposal → parser fails on natural language → graceful error
[Fact]
public async Task SendMessage_RequestProposal_GracefulErrorOnNaturalLanguage()

// Structured syntax → classifier hits → parser succeeds → proposal created
[Fact]
public async Task SendMessage_StructuredSyntax_ProposalCreated()

Affected Files

  • backend/tests/Taskdeck.Application.Tests/Services/LlmIntentClassifierTests.cs (new)
  • backend/tests/Taskdeck.Application.Tests/Services/ChatServiceTests.cs (additions)
  • backend/tests/Taskdeck.Application.Tests/Services/AutomationPlannerServiceTests.cs (additions)

Acceptance Criteria

  • LlmIntentClassifierTests covers all current patterns + documents known gaps
  • ChatService tests verify the proposal flow end-to-end
  • All tests pass with current codebase (gap tests document limitations, not assert failures)
  • Tests serve as regression guardrails for future NLP improvements

Metadata

Metadata

Assignees

No one assigned

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions