Add tool support for document queries #17

andrewstillv15 · 2025-04-23T18:44:43Z

Resolves #

Problem

The MCP server wasn't able to handle questions about retrieving documents.

Solution

This MR provides three tools to empower users to query for their documents within opslevel. They may query for their documents in bulk unconstrained, within the context of a service (both with filters by doc name) & may query for a document individually to get the contents.

There seems to be a generic problem with response length + claude that has now appeared due to the number of documents that you can fetch. I'll address this in a separate MR.

Checklist

I have run this code, and it appears to resolve the stated issue.
This PR has no user interface changes or has already received approval from product management to change the interface.
Make a changie entry that explains the customer facing outcome of this change.

src/cmd/root.go

rocktavious

LGTM but i'd like @eapache-opslevel opinion too.

wesleyjellis · 2025-04-24T13:06:03Z

src/cmd/root.go


+		// Register all documents, filtered by search term
+		s.AddTool(
+			mcp.NewTool("listDocuments",


we don't have verbs in the other commands, so for consistency this should be documents

wesleyjellis · 2025-04-24T13:07:54Z

src/cmd/root.go

+		// Register all documents, filtered by search term
+		s.AddTool(
+			mcp.NewTool("listDocuments",
+				mcp.WithDescription("Get all the documents for the opslevel account. Documents are filterable by search term"),


Can we describe what kind of things might be in documents?

Something like

"Get all the documents for the documents for the opslevel account. Documents could be things like run books, getting started guides and other forms of documentation."

If you do this copy-enriching here, similar verbage should happen on the other descriptions too.

rocktavious · 2025-04-24T15:31:55Z

src/cmd/root.go

+		// Register all documents, filtered by service id and search term
+		s.AddTool(
+			mcp.NewTool("documentsOnService",
+				mcp.WithDescription("Get all documents on a specified service for the opslevel account, specified by service id and filtered by search term. Documents could be things like runbooks, integration documentation, api documentatin, readmes, or other forms of documentation."),


Suggested change

mcp.WithDescription("Get all documents on a specified service for the opslevel account, specified by service id and filtered by search term. Documents could be things like runbooks, integration documentation, api documentatin, readmes, or other forms of documentation."),

mcp.WithDescription("Get all documents on a specified service for the opslevel account, specified by service id and filtered by search term. Documents could be things like runbooks, integration documentation, api documentation, readme's, or other forms of documentation."),

rocktavious · 2025-04-24T15:32:32Z

src/cmd/root.go

+		// Register document by id
+		s.AddTool(
+			mcp.NewTool("document",
+				mcp.WithDescription("Get document contents for the opslevel account, specified by id. Documents could be things like runbooks, integration documentation, api documentatin, readmes, or other forms of documentation."),


Suggested change

mcp.WithDescription("Get document contents for the opslevel account, specified by id. Documents could be things like runbooks, integration documentation, api documentatin, readmes, or other forms of documentation."),

mcp.WithDescription("Get document contents for the opslevel account, specified by id. Documents could be things like runbooks, integration documentation, api documentation, readme's, or other forms of documentation."),

rocktavious · 2025-04-24T15:32:44Z

src/cmd/root.go

+		// Register all documents, filtered by search term
+		s.AddTool(
+			mcp.NewTool("documents",
+				mcp.WithDescription("Get all the documents for the opslevel account. Documents are filterable by search term. Documents could be things like runbooks, integration documentation, api documentatin, readmes, or other forms of documentation."),


Suggested change

mcp.WithDescription("Get all the documents for the opslevel account. Documents are filterable by search term. Documents could be things like runbooks, integration documentation, api documentatin, readmes, or other forms of documentation."),

mcp.WithDescription("Get all the documents for the opslevel account. Documents are filterable by search term. Documents could be things like runbooks, integration documentation, api documentation, readme's, or other forms of documentation."),

eapache-opslevel · 2025-04-24T15:37:49Z

src/cmd/root.go

+			func(ctx context.Context, req mcp.CallToolRequest) (*mcp.CallToolResult, error) {
+				searchTerm := ""
+				if req.Params.Arguments["searchTerm"] != nil {
+					searchTerm = req.Params.Arguments["searchTerm"].(string)


🤔 I'm curious what type this is coming in as, that it needs casting to string?

Probably any and since searchTerm := "" is a string its needed

That's correct, it's any.

eapache-opslevel · 2025-04-24T15:39:24Z

src/cmd/root.go

+	variables := opslevel.PayloadVariables{
+		"searchTerm": searchTerm,
+		"after":      "",
+		"first":      100,


Do we need to specify first and after here? opslevel-go should fill those in and auto-paginate for us, right?

It appears the pattern is only if you pass nil so since this is initialized it won't hit that logic. That would be a good improvement in opslevel-go though.

https://github.com/OpsLevel/opslevel-go/blob/main/document.go#L38

I'll put up an MR in opslevel-go to change this in documents (and the other spots where this pattern exists too)

This is being held up by some ci issues in opslevel go. IMO we should move forwards for now without this change.

eapache-opslevel · 2025-04-24T15:39:55Z

src/cmd/root.go

 }
+
+func getListDocumentPayloadVariables(searchTerm string) opslevel.PayloadVariables {
+	variables := opslevel.PayloadVariables{


I believe you can just return the literal, you don't need to assign it to a variable and return the variable

andrewstillv15 added 4 commits April 23, 2025 14:44

get document working, initial run at list docs

2fa814b

handle no search term input

b25ddc9

upgrade opslevel-go to latest on main

3704e12

getDocumentsOnService working

d56c56f

andrewstillv15 marked this pull request as ready for review April 23, 2025 21:35

andrewstillv15 changed the title ~~get document working, initial run at list docs~~ Add tool support for document queries Apr 23, 2025

andrewstillv15 requested review from eapache-opslevel and rocktavious April 23, 2025 21:41

rename st to docNameFilter

2277122

rocktavious reviewed Apr 24, 2025

View reviewed changes

src/cmd/root.go Outdated Show resolved Hide resolved

search term is not just on name

cf37b1e

andrewstillv15 requested a review from rocktavious April 24, 2025 02:07

rocktavious approved these changes Apr 24, 2025

View reviewed changes

wesleyjellis reviewed Apr 24, 2025

View reviewed changes

update tool descriptions, remove verb from tool names

45d0e3c

andrewstillv15 requested review from BigBlue79 and wesleyjellis April 24, 2025 14:23

rocktavious reviewed Apr 24, 2025

View reviewed changes

eapache-opslevel reviewed Apr 24, 2025

View reviewed changes

andrewstillv15 added 2 commits April 24, 2025 12:32

correct def spelling mistakes

bc2e18c

return literal

18aeb0f

andrewstillv15 requested a review from eapache-opslevel April 24, 2025 16:35

andrewstillv15 merged commit e471c9e into main Apr 24, 2025
1 check passed

andrewstillv15 deleted the andrew/add-docs-tools branch April 24, 2025 18:34

	mcp.WithDescription("Get all documents on a specified service for the opslevel account, specified by service id and filtered by search term. Documents could be things like runbooks, integration documentation, api documentatin, readmes, or other forms of documentation."),
	mcp.WithDescription("Get all documents on a specified service for the opslevel account, specified by service id and filtered by search term. Documents could be things like runbooks, integration documentation, api documentation, readme's, or other forms of documentation."),

	mcp.WithDescription("Get document contents for the opslevel account, specified by id. Documents could be things like runbooks, integration documentation, api documentatin, readmes, or other forms of documentation."),
	mcp.WithDescription("Get document contents for the opslevel account, specified by id. Documents could be things like runbooks, integration documentation, api documentation, readme's, or other forms of documentation."),

	mcp.WithDescription("Get all the documents for the opslevel account. Documents are filterable by search term. Documents could be things like runbooks, integration documentation, api documentatin, readmes, or other forms of documentation."),
	mcp.WithDescription("Get all the documents for the opslevel account. Documents are filterable by search term. Documents could be things like runbooks, integration documentation, api documentation, readme's, or other forms of documentation."),

Add tool support for document queries #17

Add tool support for document queries #17

Uh oh!

Conversation

andrewstillv15 commented Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

Checklist

Uh oh!

Uh oh!

rocktavious left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

andrewstillv15 commented Apr 23, 2025 •

edited

Loading