diff --git a/docs/system_architecture.md b/docs/system_architecture.md new file mode 100644 index 00000000..5592c741 --- /dev/null +++ b/docs/system_architecture.md @@ -0,0 +1,303 @@ +# ABCoder 系统架构文档 + +## 1. 引言 (Introduction) + +本文档旨在为研发团队提供 ABCoder 系统的全面技术解析，帮助新成员、系统维护者及架构评审者快速理解其设计理念、核心功能与技术实现。 + +**面向对象**: +- 新入职研发人员 +- 系统维护与迭代开发人员 +- 架构评审与技术决策者 + +**系统概览**: ABCoder 是一个面向 AI 的代码处理框架，旨在通过提供语言无关的统一抽象语法树（UniAST）和代码检索增强生成（Code-RAG）能力，扩展大语言模型（LLM）的编码上下文，从而赋能 AI 辅助编程应用。 + +## 2. 业务背景与核心概念 (Business Context & Concepts) + +**业务场景与价值**: +ABCoder 的核心价值在于弥合大语言模型（LLM）在理解复杂代码仓库时的上下文鸿沟。传统的 LLM 服务在处理代码时，通常受限于输入长度，难以理解整个项目的结构、依赖关系和编码规范。ABCoder 通过将代码仓库预解析为结构化的 UniAST，并提供高效的检索工具，使 AI Agent 能够“在本地”精确理解代码，从而在代码审查、功能实现、bug 修复、代码翻译等场景下发挥更大作用。 + +**核心业务对象与术语**: +- **UniAST (Universal Abstract-Syntax-Tree)**: 一种语言无关、AI 友好的代码信息规范。它将不同编程语言的源代码转换为统一的、结构化的抽象语法树，包含了代码实体（如函数、类、变量）的定义、依赖关系、元数据等信息。 +- **Parser (解析器)**: 负责将特定编程语言的源代码解析成 UniAST。ABCoder 利用语言服务器协议（LSP）来实现对多种语言的精确解析。 +- **Writer (写入器)**: 负责将 UniAST 还原为特定编程语言的源代码。 +- **Code-RAG (Code-Retrieval-Augmented-Generation)**: 代码检索增强生成。ABCoder 提供了一套 MCP（Model Context Protocol）工具，允许 LLM Agent 精确、局部地查询代码仓库的 UniAST，支持工作区内和工作区外的第三方库。 +- **MCP (Model Context Protocol)**: 一种模型上下文协议，用于 AI Agent 与外部工具（如 ABCoder）之间的交互。ABCoder 作为 MCP 服务器，向 Agent 提供查询代码知识库的能力。 +- **LSP (Language Server Protocol)**: 语言服务器协议。ABCoder 借此协议与各语言的 Language Server（如 `rust-analyzer`, `pylsp`）通信，以获取代码的符号、定义、引用等精确信息。 + +**与上下游系统的集成关系**: +- **上游**: + - **代码仓库 (Code Repository)**: ABCoder 的处理对象，可以是任意语言的 Git 仓库。 + - **语言服务器 (Language Server)**: ABCoder 的 `Parser` 依赖各语言的 LSP Server 来实现代码解析。 +- **下游**: + - **AI Agent / 大语言模型 (LLM)**: ABCoder 作为 MCP 服务器，为 AI Agent 提供代码上下文和检索能力。Agent 通过调用 ABCoder 提供的工具来理解和操作代码。 + +## 3. 系统总体架构 (System Architecture) + +ABCoder 的系统架构围绕着“解析、存储、服务”三个核心环节构建。它首先将多语言代码仓库通过统一的 `Parser` 转换为 `UniAST` 格式，然后以文件形式存储。最终，通过 `MCP Server` 或 `Agent` 模式，将代码的结构化知识提供给大语言模型（LLM）。 + +```mermaid +graph TD + subgraph User Interface + A[CLI: abcoder] + end + + subgraph Core Engine + B[Language Parser] + C[UniAST] + D[Language Writer] + end + + subgraph Service Layer + E[MCP Server] + F[Agent WIP] + end + + subgraph Dependencies + G[Language Server Protocol] + H[Code Repository] + end + + subgraph Downstream + I[AI Agent / LLM] + end + + A -- parse --> B + A -- write --> D + A -- mcp --> E + A -- agent --> F + + H -- Input --> B + B -- Uses --> G + B -- Generates --> C + C -- Input --> D + D -- Outputs --> H + + C -- Loads --> E + E -- "Provides Tools to" --> I + F -- "Interacts with" --> I +``` + +## 4. 模块划分与职责 (Module Overview) + +ABCoder 的代码结构清晰，主要模块各司其职，共同构成了整个系统的核心功能。 + +```mermaid +graph LR + subgraph CLI + main[main.go] + end + + subgraph Core Logic + lang[lang] + llm[llm] + internal[internal/utils] + end + + main -- "Invokes" --> lang + main -- "Invokes" --> llm + lang -- "Uses" --> internal + llm -- "Uses" --> internal +``` + +**关键模块说明**: + +- **`main.go`**: + - **功能说明**: 命令行程序的入口。负责解析用户输入的子命令（`parse`, `write`, `mcp`, `agent`）和参数，并调用相应的功能模块。 + - **核心函数**: `main()` + +- **`lang`**: + - **功能说明**: 语言处理核心模块，负责代码的解析（`Parser`）与生成（`Writer`）。 + - **核心子模块**: + - `uniast`: 定义了统一抽象语法树（`UniAST`）的 Go 结构。 + - `lsp`: 实现了与语言服务器协议（LSP）的客户端交互，为解析器提供底层的符号、定义、引用查找能力。 + - `collect`: 实现了从 LSP 符号信息到 UniAST 的转换逻辑。 + - `golang`, `python`, `rust`, `cxx`: 针对不同语言的 `LanguageSpec` 实现，适配各语言 LSP 的特性。 + - `patch`: 负责将修改后的 UniAST 应用回原代码文件。 + - **调用关系**: 被 `main.go` 调用以执行 `parse` 和 `write` 命令。内部依赖 `internal/utils`。 + +- **`llm`**: + - **功能说明**: 负责与大语言模型（LLM）的集成和交互。 + - **核心子模块**: + - `mcp`: 实现了 MCP（Model Context Protocol）服务器。当以 `mcp` 模式运行时，此模块负责加载 UniAST 文件，并向 AI Agent 提供查询工具（`tool`）。 + - `agent`: 实现了一个简单的命令行 Agent（WIP），集成了 LLM 客户端和 `tool`，可以直接与用户进行代码问答。 + - `tool`: 定义了可供 LLM 调用的工具集，如 `read_ast_node`, `write_ast_node` 等，这些工具是 Code-RAG 的核心。 + - `prompt`: 存放了用于指导 LLM Agent 分析代码的提示模板。 + - **调用关系**: 被 `main.go` 调用以启动 `mcp` 服务或 `agent`。内部依赖 `internal/utils`。 + +- **`internal/utils`**: + - **功能说明**: 内部工具库，提供项目共享的辅助函数，如文件操作、序列化/反序列化、字符串处理等。 + - **调用关系**: 被 `lang` 和 `llm` 模块广泛使用。 + +## 5. 核心流程详解 (Core Workflows) + +### 5.1 流程一：解析代码仓库 (Parsing a Code Repository) + +这是 ABCoder 最基础的功能，将一个代码仓库转换为一个 UniAST JSON 文件。 + +**步骤拆解**: +1. 用户在命令行中执行 `abcoder parse {language} {repo-path}`。 +2. `main.go` 解析命令，确定语言和仓库路径，调用 `lang.ParseRepo` 函数。 +3. `lang.ParseRepo` 根据语言类型，启动对应的语言服务器（LSP）。 +4. `lsp.NewClient` 创建一个与 LSP Server 通信的客户端。 +5. `collect.Collect` 模块开始遍历仓库中的源文件。 +6. 对于每个文件，通过 LSP 的 `textDocument/documentSymbol` 获取文件中的所有符号（函数、类等）。 +7. 对于每个符号，通过 `textDocument/definition` 和 `textDocument/semanticTokens/range` 等方法，解析其定义、依赖关系和元数据。 +8. `collect` 模块将收集到的 LSP 信息转换为 `uniast.Node` 结构。 +9. 所有文件处理完毕后，`uniast.Writer` 将完整的 UniAST 序列化为 JSON 格式，并输出到指定文件或标准输出。 + +**时序图**: +```mermaid +sequenceDiagram + participant User + participant CLI (main.go) + participant Parser (lang.ParseRepo) + participant LSPClient (lang/lsp) + participant Collector (lang/collect) + participant UniAST (lang/uniast) + + User->>CLI: execute `abcoder parse ...` + CLI->>Parser: Invoke ParseRepo + Parser->>LSPClient: Start & Initialize Language Server + Parser->>Collector: Collect(repoPath) + Collector->>LSPClient: Request symbols, definitions, etc. + LSPClient-->>Collector: Return LSP data + Collector->>UniAST: Convert LSP data to UniAST nodes + UniAST-->>Collector: Return UniAST nodes + Collector-->>Parser: Return complete UniAST + Parser->>UniAST: Serialize to JSON + UniAST-->>Parser: Return JSON string + Parser-->>User: Output UniAST JSON file +``` + +### 5.2 流程二：通过 MCP 服务进行代码分析 (Code Analysis via MCP Server) + +此流程展示了 AI Agent 如何利用 ABCoder 提供的 Code-RAG 能力来理解代码。 + +**步骤拆解**: +1. 用户启动 ABCoder 的 MCP 服务器：`abcoder mcp {ast-directory-path}`。 +2. `main.go` 调用 `llm/mcp.RunServer`。 +3. `RunServer` 加载指定目录下所有的 UniAST JSON 文件，构建起一个代码知识库。 +4. `RunServer` 启动一个 HTTP 服务器，并注册 `llm/tool` 中定义的工具（如 `read_ast_node`）作为 MCP 端点。 +5. AI Agent（下游系统）向 MCP 服务器发起一个工具调用请求，例如查询某个函数的定义。 +6. MCP 服务器的 `handler` 接收请求，并执行对应的工具函数，例如 `tool.ReadASTNode`。 +7. 工具函数在内存中的 UniAST 知识库里进行检索，找到匹配的节点。 +8. 找到节点后，将其信息格式化为对 LLM 友好的文本。 +9. MCP 服务器将结果返回给 AI Agent。 +10. AI Agent 获得代码的精确信息，并将其作为上下文来生成对用户问题的回答。 + +**时序图**: +```mermaid +sequenceDiagram + participant User + participant CLI (main.go) + participant MCPServer (llm/mcp) + participant Tools (llm/tool) + participant AIAgent + + User->>CLI: execute `abcoder mcp ...` + CLI->>MCPServer: RunServer(ast-path) + MCPServer->>Tools: Load UniASTs from disk + MCPServer->>MCPServer: Start HTTP Server with tool endpoints + + AIAgent->>MCPServer: HTTP Request (Tool Call: e.g., read_ast_node) + MCPServer->>Tools: Execute corresponding tool function + Tools->>Tools: Search for node in loaded UniASTs + Tools-->>MCPServer: Return formatted node information + MCPServer-->>AIAgent: HTTP Response (Tool Result) + AIAgent->>AIAgent: Use result as context for generation +``` + +## 6. 数据模型与存储设计 (Data Design) + +ABCoder 的核心数据模型是 **UniAST (Universal Abstract-Syntax-Tree)**，它是一种精心设计的、语言无关的 JSON 结构，用于持久化存储代码仓库的结构化信息。 + +**存储方式**: +- 每个被解析的代码仓库最终会生成一个独立的 JSON 文件（例如 `my-repo.json`）。 +- 这个 JSON 文件完整地描述了仓库的模块、包、文件、代码实体（函数、类型、变量）及其相互关系。 +- 当作为 `MCP Server` 运行时，ABCoder 会加载这些 JSON 文件到内存中，以提供快速的查询服务。 + +**核心数据结构 (UniAST)**: +UniAST 的顶层结构主要包括 `Repository`，其下又包含 `Modules` 和 `Graph`。 + +- **`Repository`**: 代表整个代码仓库。 + - `Identity`: 仓库的唯一标识，通常是其在文件系统中的绝对路径。 + - `Modules`: 一个字典，包含了仓库内所有模块（包括主模块和依赖的第三方模块）的详细信息。 + - `Graph`: 一个依赖拓扑图，存储了代码中所有实体节点（`Node`）之间的关系。 + +- **`Module`**: 代表一个独立的编译单元（例如 Go Module, Rust Crate）。 + - `Name`, `Language`, `Version`: 模块的基本信息。 + - `Packages`: 包含的包（`Package`）的字典。 + - `Dependencies`: 模块的第三方依赖。 + - `Files`: 模块下的所有文件信息，包括代码和非代码文件。 + +- **`Package`**: 代表一个命名空间（例如 Go Package, Python Module）。 + - `Functions`, `Types`, `Vars`: 分别存储了包内定义的函数、类型和全局变量的 AST 节点。 + +- **`Node` (Function, Type, Var)**: 这是 UniAST 中最核心的元素，代表一个具体的代码实体。 + - **`Identity` (`ModPath`, `PkgPath`, `Name`)**: 每个节点的全球唯一标识，确保了查询的精确性。 + - `File`, `Line`, `StartOffset`, `EndOffset`: 精确的源码定位信息。 + - `Content`: 该实体完整的源代码文本。 + - `Signature`: 函数/方法的签名。 + - `Dependencies` & `References`: 详细记录了该节点依赖了哪些其他节点，以及被哪些其他节点所依赖，这是构建 `Graph` 的基础。 + - `TypeKind`, `IsMethod`, `IsConst` 等: 描述实体特性的元数据字段。 + +**数据一致性策略**: +- ABCoder 的数据是一次性生成的。当源代码发生变更时，需要重新执行 `abcoder parse` 命令来更新整个 UniAST JSON 文件，以保证数据与代码的同步。系统本身不处理增量更新或实时同步。 + +## 7. 异步机制与调度系统 (Async & Scheduler) + +根据当前代码库的分析，ABCoder 本身的核心逻辑（`parse`, `write`）是同步阻塞执行的。它不包含复杂的消息队列、后台任务调度或常驻的异步工作线程。 + +- **`mcp` 服务**: `mcp` 模式下会启动一个长时运行的 HTTP 服务器，该服务器本身是并发的，可以处理多个来自 AI Agent 的请求。这依赖于 Go 语言标准库 `net/http` 的并发能力。 +- **`agent` 模式**: `agent` 模式下的交互是请求-响应式的，不涉及异步处理。 + +因此，系统没有独立的异步机制或调度系统章节。 + +## 8. 配置与环境依赖 (Configuration & Environment) + +**配置文件**: +ABCoder 主要通过命令行参数进行配置，没有独立的配置文件（如 `.yaml` 或 `.env`）。关键的配置项都在执行命令时指定，例如： +- `abcoder parse {language} {repo-path} -o {output.json}` +- `abcoder mcp {ast-directory-path}` +- `abcoder agent {ast-directory-path}` + +**环境变量**: +当使用 `agent` 模式时，需要通过环境变量配置 LLM 的凭证： +- `API_TYPE`: 指定 LLM 服务类型 (e.g., `openai`, `ollama`, `ark`, `claude`) +- `API_KEY`: 对应的 API 密钥 +- `MODEL_NAME`: 使用的具体模型名称 + +**环境依赖**: +- **Go**: 运行 ABCoder 本身需要 Go 语言环境 (`go install ...`)。 +- **Language Servers**: 为了解析不同语言的仓库，必须在环境中预先安装对应的语言服务器。这在 `docs/lsp-installation-zh.md` 中有详细说明： + - **Rust**: `rust-analyzer` + - **Python**: `pylsp` (通过 git submodule 安装) + - **C/C++**: `clangd` +- **Git**: 用于克隆代码仓库。 + +## 10. 测试与监控体系 (Testing & Observability) + +**测试策略**: +项目包含了一定数量的单元测试和集成测试，主要集中在 `_test.go` 文件中。 +- **单元测试**: 针对 `internal/utils`, `lang/uniast`, `lang/lsp` 等核心模块的独立功能进行测试。 +- **集成测试**: 在 `lang/{language}` 目录下，通常有针对该语言解析、写入全流程的测试。例如，`lang/rust/rust_test.go`。 +- **测试数据**: `testdata` 目录存放了用于测试的各种语言的示例代码仓库。 + +**监控与日志**: +- **日志**: 系统在 `lang/log` 和 `llm/log` 中定义了日志记录器。在运行过程中，会将关键步骤、错误信息输出到标准错误流，方便用户调试。 +- **监控**: 当前版本的 ABCoder 没有集成专门的监控系统（如 Prometheus, Sentry）。其作为一个命令行工具和后台服务，主要依赖日志进行状态观测。 + +## 11. FAQ 与开发建议 + +**新人常见问题**: +- **Q: 解析新的语言为什么失败？** + - **A**: 首先检查是否已按照 `docs/lsp-installation-zh.md` 的指引正确安装了该语言的 Language Server，并确保其可执行文件在系统的 `PATH` 环境变量中。 +- **Q: `mcp` 服务启动了，但是 AI Agent 无法获取信息？** + - **A**: 确认 `mcp` 命令指向的目录中包含了正确的 `.json` UniAST 文件。检查 AI Agent 的配置，确保其正确地指向了 ABCoder 的 MCP 服务地址和端口。 +- **Q: UniAST 中的 `ModPath` 和 `PkgPath` 有什么区别？** + - **A**: `ModPath` 通常指一个完整的项目或库（如 `github.com/cloudwego/hertz`），而 `PkgPath` 是项目内的一个具体包或命名空间（如 `github.com/cloudwego/hertz/pkg/app`）。详细定义请参考 `docs/uniast-zh.md`。 + +**开发建议**: +- **扩展新语言**: 扩展对新语言的支持是项目的主要贡献方向。开发者需要实现 `lang/lsp.LanguageSpec` 接口，并参考 `lang/rust` 或 `lang/python` 目录下的实现。 +- **代码生成**: 在 `lang/patch` 和 `lang/{language}/writer` 中，可以改进代码生成（`write`）的逻辑，使其更好地支持代码修改和重构。 +- **遵守规范**: 提交代码前，请确保通过了所有的单元测试，并遵循项目的编码规范。 \ No newline at end of file diff --git a/docs/tree-sitter_and_lsp_zh.md b/docs/tree-sitter_and_lsp_zh.md new file mode 100644 index 00000000..a2f0baf1 --- /dev/null +++ b/docs/tree-sitter_and_lsp_zh.md @@ -0,0 +1,180 @@ +# 文档：如何使用 Tree-sitter 和 LSP 实现符号解析 + +## 1. 引言 + +本文档旨在深入解析 `abcoder` 项目中实现的符号解析（Symbol Resolution）机制。该机制采用了一种创新的混合模式，结合了 **Tree-sitter** 的快速语法分析能力和 **语言服务器协议（LSP）** 的深度语义理解能力。 + +通过阅读本文，开发人员可以理解以下内容： + +- 符号信息收集的整体流程和核心调用链路。 +- Tree-sitter 如何用于快速构建代码的抽象语法树（AST）并识别基本代码结构。 +- LSP 如何在 Tree-sitter 的基础上提供精确的语义信息，如“跳转到定义”（Go to Definition）。 +- 这种混合模式为何能兼顾解析速度与语义准确性。 + +## 2. 核心组件与调用链路 + +符号解析的核心逻辑位于 `lang/collect/collect.go` 文件中，主要由 `Collector` 结构体及其方法驱动。 + +### 2.1. 调用链路概览 + +整个符号收集过程的调用链路如下： + +1. **入口点**: 外部调用（例如 `lang/parse.go` 中的 `collectSymbol` 函数）是流程的起点。 +2. **初始化 Collector**: 通过 `collect.NewCollector(repoPath, cli)` 创建一个 `Collector` 实例。此实例包含了 LSP 客户端 `cli` 和一个针对特定语言的规约 `spec`。 +3. **开始收集**: 调用 `Collector.Collect(ctx)` 方法，这是符号收集的主逻辑。 +4. **策略分支**: 在 `Collect` 方法内部，系统会根据语言做出策略选择： + * **对于 Java 语言**: 调用 `ScannerByTreeSitter` 方法，进入 Tree-sitter + LSP 的混合解析模式。 + * **对于其他语言 (如 Rust)**: 调用 `ScannerFile` 方法，进入纯 LSP 的解析模式。 + +### 2.2. 核心组件 + +- **`Collector`**: 一个中心协调器，负责管理符号收集的整个生命周期，包括配置、文件扫描、符号提取和关系构建。 +- **`LSPClient`**: LSP 客户端的封装，用于与语言服务器（如 `jdt.ls`、`rust-analyzer`）通信，发送请求（如 `textDocument/definition`）并接收响应。 +- **`LanguageSpec`**: 定义了特定语言的行为和规则，例如如何解析导入语句、如何判断符号类型等。 +- **Tree-sitter Parser**: （主要在 `lang/java/parser` 中）用于将源代码字符串高效地解析成具体的语法树（CST/AST）。 + +## 3. Tree-sitter 语法解析 (AST Parsing) + +当处理 Java 项目时，`ScannerByTreeSitter` 方法被触发，它首先利用 Tree-sitter 进行快速的语法结构分析。 + +### 3.1. 流程详解 + +1. **项目扫描**: + * 首先，它会尝试解析项目根目录下的 `pom.xml` 文件，以获取所有 Maven 模块的路径。 + * 然后，它会遍历这些模块路径下的所有 `.java` 文件。 + +2. **文件解析**: + * 对于每个 Java 文件，它会读取文件内容。 + * 调用 `javaparser.Parse(ctx, content)`，该函数内部使用 Tree-sitter 将文件内容解析成一棵完整的语法树（`sitter.Tree`）。 + * 通知 LSP 服务器文件已打开 (`c.cli.DidOpen(ctx, uri)`)，以便 LSP 服务器建立对该文件的认知。 + +3. **AST 遍历与初步符号化**: + * 获得语法树后，调用 `c.walk(tree.RootNode(), ...)` 方法，从根节点开始深度优先遍历 AST。 + * `walk` 方法通过一个 `switch node.Type()` 语句来识别不同类型的语法节点，例如： + * `package_declaration` (包声明) + * `import_declaration` (导入语句) + * `class_declaration` (类定义) + * `method_declaration` (方法定义) + * `field_declaration` (字段定义) + * 当匹配到类、方法等定义节点时，它会从节点中提取名称、范围等信息，并创建一个初步的 `DocumentSymbol` 对象。这个对象此时主要包含**语法信息**，其定义位置就是当前文件中的位置。 + +```go +// Simplified version of the walk method in collect.go +func (c *Collector) walk(node *sitter.Node, ...) { + switch node.Type() { + case "class_declaration": + // 1. Extract class name from the node + nameNode := javaparser.FindChildIdentifier(node) + name := nameNode.Content(content) + + // 2. Create a preliminary DocumentSymbol based on syntax info + sym := &DocumentSymbol{ + Name: name, + Kind: SKClass, + Location: Location{URI: uri, Range: ...}, // Location within the current file + Node: node, // Store the tree-sitter node + Role: DEFINITION, + } + c.syms[sym.Location] = sym + + // 3. Recursively walk into the class body + bodyNode := node.ChildByFieldName("body") + if bodyNode != nil { + for i := 0; i < int(bodyNode.ChildCount()); i++ { + c.walk(bodyNode.Child(i), ...) + } + } + return + + case "method_declaration": + // ... similar logic for methods ... + } +} +``` + +## 4. LSP 语义增强 (Semantic Enhancement) + +仅靠 Tree-sitter 只能知道“这里有一个类定义”，但无法知道一个变量引用的具体类型定义在何处（尤其是在其他文件中）。这时，LSP 的作用就体现出来了。 + +在 `walk` 遍历和后续处理中，系统会利用 LSP 来“增强”由 Tree-sitter 创建的符号。 + +### 4.1. 流程详解 + +1. **获取精确的定义位置**: + * 当 Tree-sitter 解析到一个引用（例如一个类继承 `extends MyBaseClass` 或一个字段声明 `private MyType myVar;`）时，它会创建一个代表该引用的临时符号。 + * 随后，系统调用 `c.findDefinitionLocation(ref)` 方法。该方法内部会向 LSP 服务器发送一个 `textDocument/definition` 请求。 + * LSP 服务器（已经索引了项目）会返回该符号的**真正定义位置**，可能在另一个文件，甚至在依赖的库中。 + * `Collector` 用 LSP 返回的权威位置更新符号的 `Location` 字段。 + + ```go + // Get the precise definition location from LSP + func (c *Collector) findDefinitionLocation(ref *DocumentSymbol) Location { + // Send a "go to definition" request to the LSP server + defs, err := c.cli.Definition(context.Background(), ref.Location.URI, ref.Location.Range.Start) + if err != nil || len(defs) == 0 { + // If LSP can't find it (e.g., external library), return the reference location + return ref.Location + } + // Return the authoritative location from LSP + return defs[0] + } + ``` + +2. **校准符号信息**: + * 在 `walk` 方法中，当创建一个 `DocumentSymbol`（如类或方法）时，会调用 `c.findLocalLSPSymbol(uri)`。 + * 此函数会向 LSP 请求当前文件的所有符号（`textDocument/documentSymbol`），并将其缓存。 + * 然后，它会用 LSP 返回的符号列表来校准 Tree-sitter 找到的符号。例如，LSP 提供的 `method` 符号名称通常包含完整的签名（如 `myMethod(String)`），这比 Tree-sitter 单纯提取的 `myMethod` 更精确。 + +## 5. 流程总结与图示 + +系统通过两阶段的过程实现高效而准确的符号解析： + +1. **阶段一 (语法解析)**: 使用 Tree-sitter 快速扫描所有源文件，构建 AST，并识别出所有的定义和引用的基本语法结构。 +2. **阶段二 (语义链接)**: 遍历阶段一的成果，对每一个引用，利用 LSP 的 `definition` 请求查询其权威定义位置，从而建立起跨文件的符号依赖关系图。 + +### Mermaid 架构图 + +```mermaid +graph TD + subgraph "Phase 1: Fast Syntax Parsing (Tree-sitter)" + A[Start: ScannerByTreeSitter] --> B{Scan Project Files}; + B --> C[Read Java file content]; + C --> D[javaparser Parse]; + D --> E[Generate AST]; + E --> F[AST Traversal]; + F --> G[Identify syntax nodes: class, method, field]; + G --> H[Create Preliminary DocumentSymbol]; + end + + subgraph "Phase 2: Semantic Linking (LSP)" + H -. Reference Found .-> I{findDefinitionLocation}; + I --> J[Send definition request to LSP]; + J --> K[Receive Authoritative Location]; + K --> L[Update DocumentSymbol location]; + end + + L --> M[Symbol with Full Semantic Info]; + + subgraph "Parallel LSP Interaction" + F --> F_LSP{findLocalLSPSymbol}; + F_LSP --> F_LSP2[Request documentSymbol from LSP]; + F_LSP2 --> H_Update[Calibrate Symbol Name/Range]; + end + + H --> H_Update; +``` + +## 6. 结论 + +`abcoder` 采用的 Tree-sitter + LSP 混合符号解析模型是一个非常出色的工程实践。它结合了： + +- **Tree-sitter 的优点**: + - **极高的性能**: 无需预热或完整的项目索引，可以非常快速地解析单个文件。 + - **容错性强**: 即使代码有语法错误，也能生成部分可用的 AST。 + - **纯粹的语法分析**: 专注于代码结构，不依赖复杂的构建环境。 + +- **LSP 的优点**: + - **强大的语义理解**: 能够理解整个项目的上下文，包括依赖、继承关系和类型推断。 + - **准确性高**: 提供的是经过语言服务器深度分析后的权威信息。 + +通过让 Tree-sitter 完成粗粒度的结构化解析，再由 LSP 进行精确的语义“链接”，该系统在保证分析速度的同时，实现了高度准确的符号依赖关系构建，为上层代码理解和智能操作提供了坚实的基础。 \ No newline at end of file diff --git a/docs/uast_conversion_guide.md b/docs/uast_conversion_guide.md new file mode 100644 index 00000000..96136089 --- /dev/null +++ b/docs/uast_conversion_guide.md @@ -0,0 +1,215 @@ +# UAST 结构与转换流程详解 + +## 1. 引言 + +本文档旨在详细解析项目中的 **UAST (Universal Abstract Syntax Tree, 统一抽象语法树)** 的核心数据结构、设计理念以及从特定语言的 CST (Concrete Syntax Tree, 具体语法树) 到 UAST 的转换流程。 + +**面向读者**: +* **新加入的研发人员**: 快速理解项目核心的代码表示层。 +* **语言扩展开发者**: 在为项目支持新语言时，提供标准的 UAST 构建指南。 +* **架构师与代码分析工具开发者**: 深入了解 UAST 的设计，以便于上层应用的开发与集成。 + +**系统概览**: UAST 是本项目中用于表示多语言代码的统一中间表示。它将不同编程语言的语法结构抽象为一组通用的、包含丰富语义信息的图结构，是实现跨语言代码分析、转换和生成等功能的核心基础。 + +## 2. 核心概念与数据结构 + +UAST 的设计围绕着几个核心概念展开，它们共同构成了一个强大的代码表示模型。 + +### 2.1. 顶层结构 + +* `Repository`: 代码库的最高层级抽象，包含一个或多个 `Module`。 +* `Module`: 代表一个独立的、特定语言的代码单元，例如一个 Go Module、一个 Java Maven 项目或一个 Python 包。 +* `Package`: 语言内部的命名空间，如 Go 的 `package` 或 Java 的 `package`。 +* `File`: 代表一个物理源代码文件。 + +### 2.2. 核心实体：`Node` + +`Node` 是 UAST 图模型中最基本的单元。每个 `Node` 代表代码中的一个具名实体。 + +* **`NodeType`**: 节点类型，主要分为三种： + * `FUNC`: 代表函数或方法。 + * `TYPE`: 代表类、结构体、接口、枚举等类型定义。 + * `VAR`: 代表全局变量或常量。 + +* **`Identity`**: 全局唯一标识符，是链接不同 `Node` 的关键。它由三部分组成： + * `ModPath`: 模块路径 (e.g., `github.com/your/project@v1.2.0`) + * `PkgPath`: 包路径 (e.g., `github.com/your/project/internal/utils`) + * `Name`: 实体名称 (e.g., `MyFunction`, `MyStruct.MyMethod`) + * **完整形式**: `ModPath?PkgPath#Name` + +### 2.3. 实体详情 + +每个 `Node` 都关联一个更详细的实体描述结构，存储了该实体的具体信息。 + +* **`Function`**: 存储函数的签名、参数、返回值、接收者（如果是方法）以及它调用的其他函数/方法列表。 +* **`Type`**: 存储类型的种类（`struct`, `interface` 等）、字段、内嵌/继承的类型、实现的方法和接口。 +* **`Var`**: 存储变量的类型、是否为常量/指针等信息。 + +### 2.4. 关系：`Relation` + +`Relation` 用于描述两个 `Node` 之间的关系，是构建 UAST 图谱的边。 + +* **`RelationKind`**: 关系类型，主要包括： + * `DEPENDENCY`: 表示一个节点依赖另一个节点（例如函数调用、类型使用）。 + * `IMPLEMENT`: 表示一个类型节点实现了一个接口节点。 + * `INHERIT`: 表示一个类型节点继承了另一个类型节点。 + * `GROUP`: 表示多个变量/常量在同一个声明块中定义。 + +### 2.5. UAST 核心结构图 + +下图展示了 UAST 核心数据结构之间的关系。 + +```mermaid +graph TD + subgraph Repository + direction LR + A[NodeGraph] + M(Modules) + end + + subgraph Module + direction LR + P[Packages] + F[Files] + end + + subgraph Package + direction LR + Funcs[Functions] + Types[Types] + Vars[Variables] + end + + subgraph Node + direction TB + ID[Identity] + NodeType[Type: FUNC/TYPE/VAR] + Rels[Relations] + end + + Repository -- Contains --> M + M -- Contains --> Module + Repository -- Contains --> A + A -- "Maps ID to" --> Node + + Module -- Contains --> P + Module -- Contains --> F + + Package -- Contains --> Funcs + Package -- Contains --> Types + Package -- Contains --> Vars + + Node -- Has a --> ID + Node -- Has a --> NodeType + Node -- "Has multiple" --> Rels + + Funcs -- Corresponds to --> Node + Types -- Corresponds to --> Node + Vars -- Corresponds to --> Node + + style Repository fill:#f9f,stroke:#333,stroke-width:2px + style Module fill:#ccf,stroke:#333,stroke-width:2px + style Package fill:#cfc,stroke:#333,stroke-width:2px + style Node fill:#fcf,stroke:#333,stroke-width:2px +``` + +## 3. 从 CST 到 UAST 的转换流程 + +将特定语言的源代码转换为统一的 UAST，主要分为以下几个步骤。此流程的核心思想是 **“先收集实体，再建立关系”**。 + +### 3.1. 流程概览 + +```mermaid +graph TD + A[1. 源代码] --> B{2. CST 解析}; + B --> C[3. 遍历 CST, 创建 UAST 实体]; + C --> D{4. 填充 Repository}; + D --> E[5. 构建 Node Graph]; + E --> F((6. UAST)); + + subgraph "Language Specific Parser (e.g., Tree-sitter)" + B + end + + subgraph "UAST Converter (Go Code)" + C + D + E + end + + style A fill:#lightgrey + style F fill:#9f9 +``` + +### 3.2. 步骤详解 + +1. **CST 解析**: + * 使用 `tree-sitter` 或其他特定语言的解析器，将输入的源代码字符串解析为一棵具体语法树 (CST)。CST 完整地保留了代码的所有语法细节，包括标点和空格。 + +2. **遍历 CST, 创建 UAST 实体**: + * 编写一个针对该语言的 `Converter`。这个转换器会深度优先遍历 CST。 + * 当遇到代表函数、类、接口、变量声明等关键语法节点时，提取其核心信息（名称、位置、内容等）。 + * 为每个识别出的实体创建一个对应的 UAST 结构（`Function`, `Type`, `Var`），并为其生成一个全局唯一的 `Identity`。 + * 在此阶段，也会初步解析实体内部的依赖关系，例如一个函数内部调用了哪些其他函数，这些信息会被临时存储在 `Function.FunctionCalls` 等字段中。 + +3. **填充 Repository**: + * 将上一步创建的所有 `Function`, `Type`, `Var` 实体，按照其 `Identity` 中定义的模块和包路径，存入一个 `Repository` 对象中。此时，我们得到了一个包含所有代码实体信息但关系尚未连接的“半成品”。 + +4. **构建 Node Graph (`Repository.BuildGraph`)**: + * 这是将离散的实体连接成图的关键一步。调用 `Repository.BuildGraph()` 方法。 + * 该方法会遍历 `Repository` 中的每一个 `Function`, `Type`, `Var`。 + * 为每一个实体在 `Repository.Graph` 中创建一个 `Node`。 + * 然后，它会检查每个实体的依赖字段（如 `Function.FunctionCalls`, `Type.Implements` 等）。 + * 根据这些依赖信息，在对应的 `Node` 之间创建 `Relation`，从而将整个图连接起来。例如，如果 `FunctionA` 调用了 `FunctionB`，那么在 `NodeA` 和 `NodeB` 之间就会建立一条 `DEPENDENCY` 关系的边。 + +### 3.3. 转换时序图示例 + +以下是一个简化的时序图，展示了从 Java 代码到 UAST 的转换过程。 + +```mermaid +sequenceDiagram + participant C as Converter + participant T as TreeSitter + participant R as Repository + participant N as NodeGraph + + C->>T: Parse("class A { void b() {} }") + T-->>C: 返回 CST Root + + C->>C: Traverse CST + C->>R: Create/Get Module("my-java-project") + C->>R: Create/Get Package("com.example") + C->>R: SetType(Identity_A, Type_A) + C->>R: SetFunction(Identity_B, Function_B) + + Note right of C: 此时实体已收集, 但未连接 + + C->>R: BuildGraph() + R->>N: SetNode(Identity_A, TYPE) + R->>N: SetNode(Identity_B, FUNC) + R->>N: AddRelation(Node_A, Node_B, DEPENDENCY) + + Note right of R: 图关系建立完成 + + R-->>C: UAST Graph Ready +``` + +## 4. 如何使用 UAST + +一旦 UAST 构建完成，你就可以利用它进行各种强大的代码分析： + +* **依赖分析**: 从任意一个 `Node` 出发，沿着 `Dependencies` 关系，可以找到它的所有依赖项。反之，沿着 `References` 可以找到所有引用它的地方。 +* **影响范围分析**: 当一个函数或类型发生变更时，可以通过 `References` 关系，快速定位到所有可能受影响的代码。 +* **代码导航**: 实现类似 IDE 的“跳转到定义”、“查找所有引用”等功能。 +* **重构**: 自动化地进行代码重构，例如重命名一个方法，并更新所有调用点。 + +## 5. FAQ 与开发建议 + +* **为什么不直接使用 CST?** + * CST 过于具体且语言相关，直接使用它进行跨语言分析非常困难。UAST 提供了一个统一的、更高层次的抽象视图。 +* **如何添加对新语言的支持?** + * 1. 找到或构建一个该语言的 `tree-sitter` 解析器。 + * 2. 实现一个新的 `Converter`，负责遍历该语言的 CST，并创建 UAST 实体。 + * 3. 确保 `Identity` 的生成规则与其他语言保持一致。 +* **LSP 的作用是什么?** + * 虽然 `tree-sitter` 能提供语法结构，但很多语义信息（如一个变量的具体类型、一个函数调用到底解析到哪个定义）需要更复杂的类型推导。LSP 已经完成了这些工作，可以作为信息来源，极大地丰富 UAST 中 `Relation` 的准确性和语义信息。在转换流程中，可以集成 LSP 查询来辅助确定依赖关系。 \ No newline at end of file diff --git a/docs/writer-zh.md b/docs/writer-zh.md new file mode 100644 index 00000000..1988959d --- /dev/null +++ b/docs/writer-zh.md @@ -0,0 +1,41 @@ +### `Write` 函数文档 + +--- + +#### 1. 函数签名 + +```go +func Write(ctx context.Context, repo *uniast.Repository, args WriteOptions) error +``` + +#### 2. 功能概述 + +`Write` 函数的核心职责是将 `uniast.Repository` 中表示的源代码（UAST）重新写回到文件系统中。它会遍历仓库（`repo`）中的所有模块，并为每个模块选择一个特定于语言的写入器（`Writer`），将抽象语法树（AST）转换并输出为格式化的源代码文件。 + +这个函数是代码生成或代码重构流程的最后一步，实现了从抽象表示到具体代码的转换。 + +#### 3. 参数说明 + +| 参数名 | 类型 | 说明 | +| :--- | :--- | :--- | +| `ctx` | `context.Context` | Go 标准上下文，用于控制超时和取消信号。 | +| `repo` | `*uniast.Repository` | 指向 `uniast.Repository` 实例的指针。该实例包含了整个代码仓库的 UAST 表示，包括所有模块、文件和符号信息。 | +| `args` | `WriteOptions` | 写入操作的配置选项。根据代码推断，它至少包含 `OutputDir`（代码输出目录）和 `Compiler`（特定语言可能需要的编译器或工具路径，例如 Go 的格式化工具）等字段。 | + +#### 4. 执行流程 + +1. **遍历模块**：函数首先会遍历 `repo.Modules` 中的每一个模块。 +2. **跳过外部模块**：通过调用 `m.IsExternal()` 方法，判断模块是否为外部依赖。如果是，则跳过该模块，只处理项目自身的源代码。 +3. **选择写入器（Writer）**： + * 使用 `switch m.Language` 语句，根据模块的语言（`uniast.Language`）来选择合适的写入器。 + * 在当前实现中，仅支持 `uniast.Golang`。如果模块语言是 Go，它会创建一个 `golang/writer` 中的 `Writer` 实例。 + * 如果遇到任何其他不支持的语言，函数会立即返回一个错误。 +4. **写入模块**： + * 调用所选写入器的 `WriteModule` 方法。 + * 该方法负责将此模块下的所有文件和目录结构，根据 UAST 的描述，在 `args.OutputDir` 指定的目录下生成对应的源代码文件。 +5. **错误处理**：在写入过程中，如果任何步骤（如创建文件、写入内容等）失败，`WriteModule` 会返回一个错误，`Write` 函数会立即将该错误向上传递并终止执行。 +6. **成功返回**：如果所有模块都成功写入，函数最终返回 `nil`。 + +#### 5. 当前限制 + +- **语言支持**：目前的实现**仅支持 Go 语言**的代码生成。对于其他语言的模块，函数会直接报错。若要支持新语言，需要在 `switch` 语句中添加新的 `case` 并实现对应的 `uniast.Writer` 接口。 \ No newline at end of file diff --git a/go.mod b/go.mod index 1d2f3003..38a263de 100644 --- a/go.mod +++ b/go.mod @@ -15,9 +15,11 @@ require ( github.com/invopop/jsonschema v0.13.0 github.com/mark3labs/mcp-go v0.34.0 github.com/pkg/errors v0.9.1 + github.com/smacker/go-tree-sitter v0.0.0-20240827094217-dd81d9e9be82 github.com/sourcegraph/go-lsp v0.0.0-20240223163137-f80c5dd31dfd github.com/sourcegraph/jsonrpc2 v0.2.0 github.com/stretchr/testify v1.10.0 + github.com/vifraa/gopom v1.0.0 golang.org/x/mod v0.24.0 golang.org/x/tools v0.32.0 ) diff --git a/go.sum b/go.sum index c4721ffb..dc4799d3 100644 --- a/go.sum +++ b/go.sum @@ -522,6 +522,8 @@ github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ= github.com/slongfield/pyfmt v0.0.0-20220222012616-ea85ff4c361f h1:Z2cODYsUxQPofhpYRMQVwWz4yUVpHF+vPi+eUdruUYI= github.com/slongfield/pyfmt v0.0.0-20220222012616-ea85ff4c361f/go.mod h1:JqzWyvTuI2X4+9wOHmKSQCYxybB/8j6Ko43qVmXDuZg= +github.com/smacker/go-tree-sitter v0.0.0-20240827094217-dd81d9e9be82 h1:6C8qej6f1bStuePVkLSFxoU22XBS165D3klxlzRg8F4= +github.com/smacker/go-tree-sitter v0.0.0-20240827094217-dd81d9e9be82/go.mod h1:xe4pgH49k4SsmkQq5OT8abwhWmnzkhpgnXeekbx2efw= github.com/smarty/assertions v1.15.0 h1:cR//PqUBUiQRakZWqBiFFQ9wb8emQGDb0HeGdqGByCY= github.com/smarty/assertions v1.15.0/go.mod h1:yABtdzeQs6l1brC900WlRNwj6ZR55d7B+E8C6HtKdec= github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= @@ -573,6 +575,8 @@ github.com/ugorji/go/codec v1.2.7/go.mod h1:WGN1fab3R1fzQlVQTkfxVtIBhWDRqOviHU95 github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE= github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg= github.com/urfave/cli/v2 v2.3.0/go.mod h1:LJmUH05zAU44vOAcrfzZQKsZbVcdbOG8rtL3/XcUArI= +github.com/vifraa/gopom v1.0.0 h1:L9XlKbyvid8PAIK8nr0lihMApJQg/12OBvMA28BcWh0= +github.com/vifraa/gopom v1.0.0/go.mod h1:oPa1dcrGrtlO37WPDBm5SqHAT+wTgF8An1Q71Z6Vv4o= github.com/volcengine/volc-sdk-golang v1.0.23/go.mod h1:AfG/PZRUkHJ9inETvbjNifTDgut25Wbkm2QoYBTbvyU= github.com/volcengine/volc-sdk-golang v1.0.204 h1:Njid6coReHV2gWc3bsqWMQf+K8jveauzW8zEX08CTzI= github.com/volcengine/volc-sdk-golang v1.0.204/go.mod h1:stZX+EPgv1vF4nZwOlEe8iGcriUPRBKX8zA19gXycOQ= diff --git a/internal/utils/file_test.go b/internal/utils/file_test.go index 38e5f0c3..b0a4a562 100644 --- a/internal/utils/file_test.go +++ b/internal/utils/file_test.go @@ -26,6 +26,7 @@ import ( ) func TestWatchDir(t *testing.T) { + t.Skip() type args struct { dir string cb func(op fsnotify.Op, files string, state *bool) diff --git a/internal/utils/marker_test.go b/internal/utils/marker_test.go index f24f6c6c..0b4457b8 100644 --- a/internal/utils/marker_test.go +++ b/internal/utils/marker_test.go @@ -25,6 +25,7 @@ import ( ) func TestExtractMDCodes(t *testing.T) { + t.Skip() bs, err := os.ReadFile("../tmp/llm.out") if err != nil { t.Fatal(err) diff --git a/lang/collect/collect.go b/lang/collect/collect.go index 0dae1151..18d8f1ae 100644 --- a/lang/collect/collect.go +++ b/lang/collect/collect.go @@ -24,7 +24,11 @@ import ( "strings" "unicode" + sitter "github.com/smacker/go-tree-sitter" + "github.com/cloudwego/abcoder/lang/cxx" + "github.com/cloudwego/abcoder/lang/java" + "github.com/cloudwego/abcoder/lang/java/parser" "github.com/cloudwego/abcoder/lang/log" . "github.com/cloudwego/abcoder/lang/lsp" "github.com/cloudwego/abcoder/lang/python" @@ -61,6 +65,10 @@ type Collector struct { files map[string]*uniast.File + localLSPSymbol map[DocumentURI]map[Range]*DocumentSymbol + + localFunc map[Location]*DocumentSymbol + // modPatcher ModulePatcher CollectOption @@ -83,7 +91,7 @@ type functionInfo struct { Signature string `json:"signature,omitempty"` } -func switchSpec(l uniast.Language) LanguageSpec { +func switchSpec(l uniast.Language, repo string) LanguageSpec { switch l { case uniast.Rust: return rust.NewRustSpec() @@ -91,6 +99,8 @@ func switchSpec(l uniast.Language) LanguageSpec { return cxx.NewCxxSpec() case uniast.Python: return python.NewPythonSpec() + case uniast.Java: + return java.NewJavaSpec(repo) default: panic(fmt.Sprintf("unsupported language %s", l)) } @@ -100,7 +110,7 @@ func NewCollector(repo string, cli *LSPClient) *Collector { ret := &Collector{ repo: repo, cli: cli, - spec: switchSpec(cli.ClientOptions.Language), + spec: switchSpec(cli.ClientOptions.Language, repo), syms: map[Location]*DocumentSymbol{}, funcs: map[*DocumentSymbol]functionInfo{}, deps: map[*DocumentSymbol][]dependency{}, @@ -136,85 +146,15 @@ func (c *Collector) configureLSP(ctx context.Context) { } func (c *Collector) Collect(ctx context.Context) error { - c.configureLSP(ctx) - excludes := make([]string, len(c.Excludes)) - for i, e := range c.Excludes { - if !filepath.IsAbs(e) { - excludes[i] = filepath.Join(c.repo, e) - } else { - excludes[i] = e - } - } - - // scan all files - root_syms := make([]*DocumentSymbol, 0, 1024) - scanner := func(path string, info os.FileInfo, err error) error { - if err != nil { - return err - } - if info.IsDir() { - return nil - } - for _, e := range excludes { - if strings.HasPrefix(path, e) { - return nil - } - } - - if c.spec.ShouldSkip(path) { - return nil - } - - file := c.files[path] - if file == nil { - rel, err := filepath.Rel(c.repo, path) - if err != nil { - return err - } - file = uniast.NewFile(rel) - c.files[path] = file - } - - // 解析use语句 - content, err := os.ReadFile(path) - if err != nil { - return err - } - uses, err := c.spec.FileImports(content) - if err != nil { - log.Error("parse file %s use statements failed: %v", path, err) - } else { - file.Imports = uses - } - - // collect symbols - uri := NewURI(path) - symbols, err := c.cli.DocumentSymbols(ctx, uri) + var root_syms []*DocumentSymbol + var err error + if c.Language == uniast.Java { + root_syms, err = c.ScannerByTreeSitter(ctx) if err != nil { return err } - // file := filepath.Base(path) - for _, sym := range symbols { - // collect content - content, err := c.cli.Locate(sym.Location) - if err != nil { - return err - } - // collect tokens - tokens, err := c.cli.SemanticTokens(ctx, sym.Location) - if err != nil { - return err - } - sym.Text = content - sym.Tokens = tokens - c.syms[sym.Location] = sym - root_syms = append(root_syms, sym) - } - - return nil - } - if err := filepath.Walk(c.repo, scanner); err != nil { - log.Error("scan files failed: %v", err) + } else { + root_syms = c.ScannerFile(ctx) } // collect some extra metadata @@ -224,7 +164,9 @@ func (c *Collector) Collect(ctx context.Context) error { if c.spec.IsEntitySymbol(*sym) { entity_syms = append(entity_syms, sym) } - c.processSymbol(ctx, sym, 1) + if c.Language != uniast.Java { + c.processSymbol(ctx, sym, 1) + } } // collect internal references @@ -306,6 +248,10 @@ func (c *Collector) Collect(ctx context.Context) error { // go to definition dep, err := c.getSymbolByToken(ctx, token) if err != nil || dep == nil { + if token.Type == "method_invocation" || token.Type == "static_method_invocation" { + // 外部依赖无法从LSP 中查询到定义，先不报错 + continue + } log.Error("dep token %v not found: %v\n", token, err) continue } @@ -338,6 +284,722 @@ func (c *Collector) Collect(ctx context.Context) error { return nil } +func (c *Collector) ScannerFile(ctx context.Context) []*DocumentSymbol { + c.configureLSP(ctx) + excludes := make([]string, len(c.Excludes)) + for i, e := range c.Excludes { + if !filepath.IsAbs(e) { + excludes[i] = filepath.Join(c.repo, e) + } else { + excludes[i] = e + } + } + + // scan all files + root_syms := make([]*DocumentSymbol, 0, 1024) + scanner := func(path string, info os.FileInfo, err error) error { + if err != nil { + return err + } + if info.IsDir() { + return nil + } + for _, e := range excludes { + if strings.HasPrefix(path, e) { + return nil + } + } + + if c.spec.ShouldSkip(path) { + return nil + } + + file := c.files[path] + if file == nil { + rel, err := filepath.Rel(c.repo, path) + if err != nil { + return err + } + file = uniast.NewFile(rel) + c.files[path] = file + } + + // 解析use语句 + content, err := os.ReadFile(path) + if err != nil { + return err + } + uses, err := c.spec.FileImports(content) + if err != nil { + log.Error("parse file %s use statements failed: %v", path, err) + } else { + file.Imports = uses + } + + // collect symbols + uri := NewURI(path) + symbols, err := c.cli.DocumentSymbols(ctx, uri) + if err != nil { + return err + } + // file := filepath.Base(path) + for _, sym := range symbols { + // collect content + content, err := c.cli.Locate(sym.Location) + if err != nil { + return err + } + // collect tokens + tokens, err := c.cli.SemanticTokens(ctx, sym.Location) + if err != nil { + return err + } + sym.Text = content + sym.Tokens = tokens + c.syms[sym.Location] = sym + root_syms = append(root_syms, sym) + } + + return nil + } + if err := filepath.Walk(c.repo, scanner); err != nil { + log.Error("scan files failed: %v", err) + } + return root_syms +} + +func (c *Collector) ScannerByTreeSitter(ctx context.Context) ([]*DocumentSymbol, error) { + var modulePaths []string + // Java uses parsing pom method to obtain hierarchical relationships + if c.Language == uniast.Java { + rootPomPath := filepath.Join(c.repo, "pom.xml") + rootModule, err := parser.ParseMavenProject(rootPomPath) + if err != nil { + // 尝试直接遍历文件 + modulePaths = append(modulePaths, c.repo) + } else { + modulePaths = parser.GetModulePaths(rootModule) + } + // Collect all module paths from the maven project structure + } + + c.configureLSP(ctx) + excludes := make([]string, len(c.Excludes)) + for i, e := range c.Excludes { + if !filepath.IsAbs(e) { + excludes[i] = filepath.Join(c.repo, e) + } else { + excludes[i] = e + } + } + + scanner := func(path string, info os.FileInfo, err error) error { + if err != nil { + return err + } + if info.IsDir() { + return nil + } + for _, e := range excludes { + if strings.HasPrefix(path, e) { + return nil + } + } + + if c.spec.ShouldSkip(path) { + return nil + } + + file := c.files[path] + if file == nil { + rel, err := filepath.Rel(c.repo, path) + if err != nil { + return err + } + file = uniast.NewFile(rel) + c.files[path] = file + } + + // 解析use语句 + content, err := os.ReadFile(path) + if err != nil { + return err + } + + uri := NewURI(path) + _, err = c.cli.DidOpen(ctx, uri) + if err != nil { + return err + } + tree, err := parser.Parse(ctx, content) + if err != nil { + log.Error("parse file %s failed: %v", path, err) + return nil // continue with next file + } + + uri = NewURI(path) + c.walk(tree.RootNode(), uri, content, file, nil) + + return nil + } + + // Walk each module path to find and parse files in module + for i, modulePath := range modulePaths { + if err := filepath.Walk(modulePath, scanner); err != nil { + log.Error("scan files failed: %v", err) + } + log.Info("finish collector module %v ，progress rate %d/%d ", modulePath, i, len(modulePaths)) + } + + root_syms := make([]*DocumentSymbol, 0, 1024) + + for _, symbol := range c.syms { + root_syms = append(root_syms, symbol) + } + return root_syms, nil +} + +// getModulePaths traverses the maven module tree and returns a flat list of module paths. +func (c *Collector) collectFields(node *sitter.Node, uri DocumentURI, content []byte, path string, parent *DocumentSymbol) { + if node == nil { + return + } + q, err := sitter.NewQuery([]byte("(field_declaration) @field"), parser.GetLanguage(c.CollectOption.Language)) + if err != nil { + // Or handle the error more gracefully + return + } + qc := sitter.NewQueryCursor() + qc.Exec(q, node) + + for { + m, ok := qc.NextMatch() + if !ok { + break + } + for _, capture := range m.Captures { + fieldNode := capture.Node + // Find the type of the field. + typeNode := fieldNode.ChildByFieldName("type") + var typeDep dependency + if typeNode != nil { + typeSymbols := c.parseTypeIdentifiers(typeNode, content, uri) + if len(typeSymbols) > 0 { + // A variable has one type, we take the first symbol as its type. + typeDep = dependency{Symbol: typeSymbols[0], Location: typeSymbols[0].Location} + } + } + fullyName := fieldNode.Content(content) + + // A field declaration can have multiple variables, e.g., `int a, b;` + // We need to iterate through the variable_declarator nodes. + for i := 0; i < int(fieldNode.ChildCount()); i++ { + child := fieldNode.Child(i) + if child.Type() == "variable_declarator" { + nameNode := child.ChildByFieldName("name") + if nameNode == nil { + continue + } + + isStatic := strings.Contains(fullyName, "static") + isFinal := strings.Contains(fullyName, "final") + isPublic := strings.Contains(fullyName, "public") + kind := SKUnknown + if isStatic && isFinal && isPublic { + kind = SKConstant + } else if isStatic && isPublic { + kind = SKVariable + } else { + kind = SKClass + } + + if kind == SKClass { + sym := typeDep.Symbol + if sym == nil { + continue + } + sym.Role = REFERENCE + if parent != nil { + c.addReferenceDeps(parent, sym) + } + } else { + name := nameNode.Content(content) + start := child.StartPoint() + end := child.EndPoint() + uri := NewURI(path) + + sym := &DocumentSymbol{ + Name: name, + Kind: kind, + Text: fullyName, + Location: Location{ + URI: uri, + Range: Range{ + Start: toLSPPosition(content, start.Row, start.Column), + End: toLSPPosition(content, end.Row, end.Column), + }, + }, + Node: child, + Tokens: []Token{nodeToToken(child, content, uri)}, + Role: REFERENCE, + } + if parent != nil { + c.addReferenceDeps(parent, sym) + } + // Store the type dependency in c.vars + if typeDep.Symbol != nil && kind == SKConstant || kind == SKVariable { + c.vars[sym] = typeDep + c.syms[sym.Location] = sym + } + } + } + } + } + } +} + +func (c *Collector) addReferenceDeps(sym *DocumentSymbol, ref *DocumentSymbol) { + if ref.Role != REFERENCE { + return + } + TokenLocation := ref.Location + var refDefinitionLocation = c.findDefinitionLocation(ref) + if refDefinitionLocation == TokenLocation { + // todo 三方外部符号查询不到，引用和定义符号位置一致时，过滤掉 + return + } + ref.Location = refDefinitionLocation + c.deps[sym] = append(c.deps[sym], dependency{ + Symbol: ref, + Location: TokenLocation, + }) +} + +func (c *Collector) findLocalLSPSymbol(fileURI DocumentURI) map[Range]*DocumentSymbol { + if c.localLSPSymbol[fileURI] == nil { + c.localLSPSymbol = make(map[DocumentURI]map[Range]*DocumentSymbol) + symbols, _ := c.cli.DocumentSymbols(context.Background(), fileURI) + c.localLSPSymbol[fileURI] = symbols + return symbols + } + return c.localLSPSymbol[fileURI] +} + +func (c *Collector) findDefinitionLocation(ref *DocumentSymbol) Location { + defs, err := c.cli.Definition(context.Background(), ref.Location.URI, ref.Location.Range.Start) + if err != nil || len(defs) == 0 { + // 意味着引用为外部符号，LSP 无法查询到符号定位,暂时复用当前符号引用位置 + return ref.Location + } else { + return defs[0] + } +} + +func (c *Collector) walk(node *sitter.Node, uri DocumentURI, content []byte, file *uniast.File, parent *DocumentSymbol) { + switch node.Type() { + case "package_declaration": + pkgNameNode := parser.FindChildIdentifier(node) + if pkgNameNode != nil { + file.Package = uniast.PkgPath(pkgNameNode.Content(content)) + } + return // no need to walk children + + case "import_declaration": + importPathNode := parser.FindChildIdentifier(node) + if importPathNode != nil { + file.Imports = append(file.Imports, uniast.Import{Path: importPathNode.Content(content)}) + } + return // no need to walk children of import declaration + + case "class_declaration", "interface_declaration", "enum_declaration": + nameNode := parser.FindChildIdentifier(node) + if nameNode == nil { + return // anonymous class, skip + } + name := nameNode.Content(content) + start := node.StartPoint() + end := node.EndPoint() + + var kind SymbolKind + if node.Type() == "class_declaration" { + kind = SKClass + } else if node.Type() == "enum_declaration" { + kind = SKEnum + } else { + kind = SKInterface + } + + sym := &DocumentSymbol{ + Name: name, + Kind: kind, + Text: node.Content(content), + Location: Location{ + URI: uri, + Range: Range{ + Start: toLSPPosition(content, start.Row, start.Column), + End: toLSPPosition(content, end.Row, end.Column), + }, + }, + Node: node, + Role: DEFINITION, + } + + symbols := c.findLocalLSPSymbol(sym.Location.URI) + for _, symbol := range symbols { + //lsp 替换 + if symbol.Name == name { + sym.Location = symbol.Location + } + } + + // Collect tokens for class/interface declarations + // Extract extends/implements for class_declaration + if node.Type() == "class_declaration" { + // Handle extends (superclass) + extendsNode := node.ChildByFieldName("superclass") + if extendsNode != nil { + extendsType := c.parseTypeIdentifiers(extendsNode, content, uri) + for _, ext := range extendsType { + ext.Kind = SKClass + ext.Role = REFERENCE + c.addReferenceDeps(sym, ext) + } + } + + // Handle implements (interfaces) + implementsNode := node.ChildByFieldName("interfaces") + if implementsNode != nil { + implTypes := c.parseTypeIdentifiers(implementsNode, content, uri) + for _, impl := range implTypes { + impl.Kind = SKInterface + impl.Role = REFERENCE + c.addReferenceDeps(sym, impl) + } + } + } + + c.syms[sym.Location] = sym + if parent != nil { + parent.Children = append(parent.Children, sym) + c.deps[parent] = append(c.deps[parent], dependency{ + Symbol: sym, + Location: sym.Location, + }) + + } + + // walk children + bodyNode := node.ChildByFieldName("body") + if bodyNode != nil { + c.collectFields(bodyNode, uri, content, uri.File(), sym) + for i := 0; i < int(bodyNode.ChildCount()); i++ { + child := bodyNode.Child(i) + c.walk(child, uri, content, file, sym) + } + } + return // children already walked + + case "method_declaration": + nameNode := node.ChildByFieldName("name") + if nameNode == nil { + return // Can be a constructor + } + name := nameNode.Content(content) + start := node.StartPoint() + end := node.EndPoint() + + isStatic := isStaticMethod(node, content) + + // 根据是否为静态方法设置不同的Kind + var kind SymbolKind + if isStatic { + kind = SKFunction // 静态方法 -> Functions + } else { + kind = SKMethod // 非静态方法 -> type的method + } + + sym := &DocumentSymbol{ + Name: name, + Kind: kind, + Text: node.Content(content), + Location: Location{ + URI: uri, + Range: Range{ + Start: toLSPPosition(content, start.Row, start.Column), + End: toLSPPosition(content, end.Row, end.Column), + }, + }, + Node: node, + Role: DEFINITION, + } + + symbols := c.findLocalLSPSymbol(sym.Location.URI) + signature := c.parseMethodSignature(node, content) + for _, symbol := range symbols { + if symbol.Name == signature { + sym.Location = symbol.Location + sym.Name = symbol.Name + } + } + + info := functionInfo{ + TypeParams: make(map[int]dependency), + Inputs: make(map[int]dependency), + Outputs: make(map[int]dependency), + } + + // Parse type parameters + if typeParamsNode := node.ChildByFieldName("type_parameters"); typeParamsNode != nil { + typeParams := c.parseTypeIdentifiers(typeParamsNode, content, uri) + for i, p := range typeParams { + p.Kind = SKTypeParameter + p.Role = REFERENCE + tokenLocation := p.Location + p.Location = c.findDefinitionLocation(p) + if tokenLocation == p.Location { + // 外部依赖符号，跳过 + continue + } + info.TypeParams[i] = dependency{Symbol: p, + Location: tokenLocation, + } + } + } + + // Parse return type and add to tokens + if returnTypeNode := node.ChildByFieldName("type"); returnTypeNode != nil { + returns := c.parseTypeIdentifiers(returnTypeNode, content, uri) + for i, p := range returns { + p.Role = REFERENCE + tokenLocation := p.Location + p.Location = c.findDefinitionLocation(p) + if tokenLocation == p.Location { + // 外部依赖符号，跳过 + continue + } + info.Outputs[i] = dependency{Symbol: p, Location: tokenLocation} + } + } + + // Parse parameters and add to tokens + if paramsNode := node.ChildByFieldName("parameters"); paramsNode != nil { + params := c.parseFormalParameters(paramsNode, content, uri) + for i, p := range params { + if typeNode := p.Node.ChildByFieldName("type"); typeNode != nil { + typeSymbols := c.parseTypeIdentifiers(typeNode, content, uri) + for _, typeSym := range typeSymbols { + typeSym.Role = REFERENCE + tokenLocation := typeSym.Location + typeSym.Location = c.findDefinitionLocation(typeSym) + if tokenLocation == p.Location { + // 外部依赖符号，跳过 + continue + } + info.Inputs[i] = dependency{Symbol: typeSym, Location: tokenLocation} + } + } + } + } + + // Populate Method info + if parent != nil && (parent.Kind == SKClass || parent.Kind == SKInterface) { + info.Method = &methodInfo{ + Receiver: dependency{Symbol: parent, Location: parent.Location}, + } + } + + // Sort dependencies + if len(info.TypeParams) > 0 { + keys := make([]int, 0, len(info.TypeParams)) + for k := range info.TypeParams { + keys = append(keys, k) + } + slices.Sort(keys) + info.TypeParamsSorted = make([]dependency, len(keys)) + for i, k := range keys { + info.TypeParamsSorted[i] = info.TypeParams[k] + } + } + if len(info.Outputs) > 0 { + keys := make([]int, 0, len(info.Outputs)) + for k := range info.Outputs { + keys = append(keys, k) + } + slices.Sort(keys) + info.OutputsSorted = make([]dependency, len(keys)) + for i, k := range keys { + info.OutputsSorted[i] = info.Outputs[k] + } + } + if len(info.Inputs) > 0 { + keys := make([]int, 0, len(info.Inputs)) + for k := range info.Inputs { + keys = append(keys, k) + } + slices.Sort(keys) + info.InputsSorted = make([]dependency, len(keys)) + for i, k := range keys { + info.InputsSorted[i] = info.Inputs[k] + } + } + + // Generate signature + var signatureEnd uint32 + bodyNode := node.ChildByFieldName("body") + if bodyNode != nil { + signatureEnd = bodyNode.StartByte() + // 解析方法体内的所有方法调用 + c.parseMethodInvocations(bodyNode, content, uri, sym) + } else { + signatureEnd = node.EndByte() + } + info.Signature = strings.TrimSpace(string(content[node.StartByte():signatureEnd])) + c.funcs[sym] = info + c.syms[sym.Location] = sym + + return // children already walked + + case "field_declaration": + return + } + + // default behavior + for i := 0; i < int(node.ChildCount()); i++ { + child := node.Child(i) + c.walk(child, uri, content, file, parent) + } +} + +// parseTypeIdentifiers walks through a node (like type_parameters or a return type node) +// and extracts all type identifiers, creating placeholder DocumentSymbols for them. +func (c *Collector) parseTypeIdentifiers(node *sitter.Node, content []byte, uri DocumentURI) []*DocumentSymbol { + var symbols []*DocumentSymbol + c.recursiveParseTypes(node, content, uri, &symbols, false) + return symbols +} + +func (c *Collector) recursiveParseTypes(node *sitter.Node, content []byte, uri DocumentURI, symbols *[]*DocumentSymbol, IsInterface bool) { + switch node.Type() { + case "generic_type": + + // This is a base case for the recursion. + start := node.StartPoint() + end := node.EndPoint() + kind := java.NodeTypeToSymbolKind(node.Type()) + + typeSym := &DocumentSymbol{ + Name: node.Content(content), + Kind: kind, + Location: Location{ + URI: uri, + Range: Range{ + Start: toLSPPosition(content, start.Row, start.Column), + End: toLSPPosition(content, end.Row, end.Column), + }, + }, + Text: node.Content(content), + Node: node, + } + *symbols = append(*symbols, typeSym) + + // For a generic type like "List", we want to parse "List" and "String" separately. + // The main type identifier (e.g., "List") + typeNode := parser.FindChildByType(node, "type") + if typeNode != nil { + c.recursiveParseTypes(typeNode, content, uri, symbols, false) + } + // The type arguments (e.g., "") + argsNode := parser.FindChildByType(node, "type_arguments") + if argsNode != nil { + for i := 0; i < int(argsNode.ChildCount()); i++ { + c.recursiveParseTypes(argsNode.Child(i), content, uri, symbols, false) + } + } + case "type_identifier": + // This is a base case for the recursion. + start := node.StartPoint() + end := node.EndPoint() + kind := java.NodeTypeToSymbolKind(node.Type()) + if IsInterface { + kind = SKInterface + } + typeSym := &DocumentSymbol{ + Name: node.Content(content), + Kind: kind, + Location: Location{ + URI: uri, + Range: Range{ + Start: toLSPPosition(content, start.Row, start.Column), + End: toLSPPosition(content, end.Row, end.Column), + }, + }, + Text: node.Content(content), + Node: node, + } + *symbols = append(*symbols, typeSym) + case "super_interfaces": + typeNode := parser.FindChildByType(node, "type_list") + if typeNode != nil { + c.recursiveParseTypes(typeNode, content, uri, symbols, true) + } + default: + // For any other node type, recurse on its children. + for i := 0; i < int(node.ChildCount()); i++ { + c.recursiveParseTypes(node.Child(i), content, uri, symbols, IsInterface) + } + } +} + +// parseFormalParameters handles the `formal_parameters` node to extract each parameter. +func (c *Collector) parseFormalParameters(node *sitter.Node, content []byte, uri DocumentURI) []*DocumentSymbol { + var symbols []*DocumentSymbol + + for i := 0; i < int(node.ChildCount()); i++ { + child := node.Child(i) + if child.Type() == "formal_parameter" { + + paramTypeNode := child.ChildByFieldName("type") + paramNameNode := child.ChildByFieldName("name") + if paramTypeNode != nil && paramNameNode != nil { + start := child.StartPoint() + end := child.EndPoint() + paramSym := &DocumentSymbol{ + Name: paramNameNode.Content(content), + Kind: java.NodeTypeToSymbolKind(paramTypeNode.Type()), + Location: Location{ + URI: uri, + Range: Range{ + Start: toLSPPosition(content, start.Row, start.Column), + End: toLSPPosition(content, end.Row, end.Column), + }, + }, + Text: child.Content(content), + Node: child, + } + symbols = append(symbols, paramSym) + } + } + } + return symbols +} + +func isStaticMethod(node *sitter.Node, content []byte) bool { + var modifiersNode *sitter.Node + for i := 0; i < int(node.ChildCount()); i++ { + child := node.Child(i) + if child.Type() == "modifiers" { + modifiersNode = child + break + } + } + + if modifiersNode == nil { + return false + } + modifiersString := modifiersNode.Content(content) + return strings.Contains(modifiersString, "static") +} + func (c *Collector) internal(loc Location) bool { return strings.HasPrefix(loc.URI.File(), c.repo) } @@ -727,3 +1389,240 @@ func (c *Collector) updateFunctionInfo(sym *DocumentSymbol, tsyms, ipsyms, opsym c.funcs[sym] = f } + +// nodeToLocation converts a Tree-sitter node's position information to LSP Location format. +func nodeToLocation(node *sitter.Node, uri DocumentURI, content []byte) Location { + start := node.StartPoint() + end := node.EndPoint() + + // 将Tree-sitter的UTF-8字节位置转换为LSP的UTF-16字符位置 + startLine, startChar := parser.Utf8ToUtf16Position(content, start.Row, start.Column) + endLine, endChar := parser.Utf8ToUtf16Position(content, end.Row, end.Column) + + return Location{ + URI: uri, + Range: Range{ + Start: Position{Line: startLine, Character: startChar}, + End: Position{Line: endLine, Character: endChar}, + }, + } +} + +func toLSPPosition(content []byte, Row, Column uint32) Position { + startLine, startChar := parser.Utf8ToUtf16Position(content, Row, Column) + return Position{Line: startLine, Character: startChar} +} + +// nodeToToken converts a Tree-sitter node to lsp.Token. +func nodeToToken(node *sitter.Node, content []byte, uri DocumentURI) Token { + // Validate node position to ensure it's within file bounds + start := node.StartPoint() + end := node.EndPoint() + + // Ensure position is valid for LSP + if start.Row < 0 || start.Column < 0 || end.Row < 0 || end.Column < 0 { + // Log warning for invalid position + log.Error("Invalid Tree-sitter position: node=%s, start=%d:%d, end=%d:%d", + node.Type(), start.Row, start.Column, end.Row, end.Column) + } + + return Token{ + Text: node.Content(content), + Location: nodeToLocation(node, uri, content), + Type: node.Type(), + Modifiers: []string{}, // Initialize with empty slice to avoid nil + } +} + +func (c *Collector) parseMethodInvocations(bodyNode *sitter.Node, content []byte, uri DocumentURI, methodSym *DocumentSymbol) { + if bodyNode == nil { + return + } + + // New approach: find argument_list, then find its parent (method_invocation) + // and extract name and object from there. + query, err := sitter.NewQuery([]byte(` + (argument_list) @args + `), parser.GetLanguage(c.CollectOption.Language)) + if err != nil { + log.Error("Failed to create method invocation query: %v", err) + return + } + defer query.Close() + + qc := sitter.NewQueryCursor() + defer qc.Close() + qc.Exec(query, bodyNode) + + for { + match, ok := qc.NextMatch() + if !ok { + break + } + + for _, capture := range match.Captures { + argListNode := capture.Node + + invocationNode := argListNode.Parent() + if invocationNode == nil || invocationNode.Type() != "method_invocation" { + continue + } + + methodNameNode := invocationNode.ChildByFieldName("name") + if methodNameNode == nil { + continue + } + + methodName := methodNameNode.Content(content) + start := methodNameNode.StartPoint() + end := methodNameNode.EndPoint() + invocationLocation := Location{ + URI: uri, + Range: Range{ + Start: toLSPPosition(content, start.Row, start.Column), + End: toLSPPosition(content, end.Row, end.Column), + }, + } + + objectNode := invocationNode.ChildByFieldName("object") + + var dep dependency + + if objectNode != nil { + // This could be a static or a normal method call. + className := c.extractRootIdentifier(objectNode, content) + // A simple heuristic to decide if it's a static call: + // if the extracted root identifier is not empty and starts with an uppercase letter. + // This is not foolproof but a common convention in Java. + isStatic := false + if className != "" { + runes := []rune(className) + if len(runes) > 0 && unicode.IsUpper(runes[0]) { + isStatic = true + } + } + + if isStatic { + // Static method call + qualifiedMethodName := className + "." + methodName + dep = dependency{ + Symbol: &DocumentSymbol{ + Name: qualifiedMethodName, + Kind: SKFunction, + Location: invocationLocation, + Role: REFERENCE, + }, + Location: invocationLocation, + } + } else { + dep = dependency{ + Symbol: &DocumentSymbol{ + Name: methodName, + Kind: SKMethod, + Location: invocationLocation, + Role: REFERENCE, + }, + Location: invocationLocation, + } + } + } else { + dep = dependency{ + Symbol: &DocumentSymbol{ + Name: methodName, + Kind: SKMethod, + Location: invocationLocation, + Role: REFERENCE, + }, + Location: invocationLocation, + } + } + DefinitionLocation := c.findDefinitionLocation(dep.Symbol) + + if DefinitionLocation == dep.Symbol.Location { + //外部函数调用，先过滤 + continue + } + dep.Symbol.Location = DefinitionLocation + c.deps[methodSym] = append(c.deps[methodSym], dep) + } + } +} + +func (c *Collector) extractRootIdentifier(node *sitter.Node, content []byte) string { + if node == nil { + return "" + } + + if node.Type() == "identifier" { + return node.Content(content) + } + + childCount := int(node.ChildCount()) + for i := 0; i < childCount; i++ { + child := node.Child(i) + fieldName := node.FieldNameForChild(i) + if fieldName == "object" { + return c.extractRootIdentifier(child, content) + } + } + + // Fallback for cases where the field name is not 'object' + if childCount > 0 { + return c.extractRootIdentifier(node.Child(0), content) + } + + return "" +} + +// parseMethodSignature 从方法节点解析签名，保留方法名和参数类型 +// 例如: public String queryJwtToken(String id, String tenantId, String idType) -> queryJwtToken(String, String, String) +// 例如: forwardLarkEvent(Map) -> forwardLarkEvent(Map) +func (c *Collector) parseMethodSignature(node *sitter.Node, content []byte) string { + if node == nil { + return "" + } + + // 获取方法名 + nameNode := parser.FindChildIdentifier(node) + if nameNode == nil { + return "" + } + methodName := nameNode.Content(content) + + // 获取参数节点 + paramsNode := node.ChildByFieldName("parameters") + if paramsNode == nil { + return fmt.Sprintf("%s()", methodName) + } + // 解析参数类型 + var paramTypes []string + + // 遍历所有参数 + for i := 0; i < int(paramsNode.ChildCount()); i++ { + child := paramsNode.Child(i) + if child.Type() == "formal_parameter" { + // 获取参数类型节点 + typeNode := child.ChildByFieldName("type") + if typeNode != nil { + typeContent := typeNode.Content(content) + if typeContent != "" { + paramTypes = append(paramTypes, typeContent) + } + } + } else if child.Type() == "spread_parameter" { + for u := range int(child.ChildCount()) { + // 处理可变参数 ...Type + parameterNode := child.Child(u) + if parameterNode != nil && parameterNode.Type() == "type_identifier" { + paramType := parameterNode.Content(content) + if paramType != "" { + } + paramTypes = append(paramTypes, paramType+"...") + } + } + + } + } + + return fmt.Sprintf("%s(%s)", methodName, strings.Join(paramTypes, ", ")) +} diff --git a/lang/collect/collect_test.go b/lang/collect/collect_test.go index 0f3ddeea..283172c6 100644 --- a/lang/collect/collect_test.go +++ b/lang/collect/collect_test.go @@ -18,14 +18,50 @@ import ( "context" "encoding/json" "os" + "path/filepath" "testing" + "github.com/cloudwego/abcoder/lang/java" + javaLsp "github.com/cloudwego/abcoder/lang/java/lsp" "github.com/cloudwego/abcoder/lang/log" "github.com/cloudwego/abcoder/lang/lsp" "github.com/cloudwego/abcoder/lang/testutils" "github.com/cloudwego/abcoder/lang/uniast" ) +func TestCollector_CollectByTreeSitter_Java(t *testing.T) { + log.SetLogLevel(log.DebugLevel) + javaTestCase := "../../testdata/java/1_advanced" + + t.Run("javaCollect", func(t *testing.T) { + + lsp.RegisterProvider(uniast.Java, &javaLsp.JavaProvider{}) + + openfile, wait := java.CheckRepo(javaTestCase) + l, s := java.GetDefaultLSP(make(map[string]string)) + client, err := lsp.NewLSPClient(javaTestCase, openfile, wait, lsp.ClientOptions{ + Server: s, + Language: l, + Verbose: false, + }) + + c := NewCollector(javaTestCase, client) + c.Language = uniast.Java + _, err = c.ScannerByTreeSitter(context.Background()) + if err != nil { + t.Fatalf("Collector.CollectByTreeSitter() failed = %v\n", err) + } + + if len(c.files) == 0 { + t.Fatalf("Expected have file, but got %d", len(c.files)) + } + + expectedFile := filepath.Join(javaTestCase, "/src/main/java/org/example/test.json") + if _, ok := c.files[expectedFile]; ok { + t.Fatalf("Expected file %s not found", expectedFile) + } + }) +} func TestCollector_Collect(t *testing.T) { log.SetLogLevel(log.DebugLevel) rustLSP, rustTestCase, err := lsp.InitLSPForFirstTest(uniast.Rust, "rust-analyzer") diff --git a/lang/collect/export.go b/lang/collect/export.go index c15f10c6..885c874f 100644 --- a/lang/collect/export.go +++ b/lang/collect/export.go @@ -24,6 +24,7 @@ import ( "github.com/cloudwego/abcoder/lang/log" . "github.com/cloudwego/abcoder/lang/lsp" "github.com/cloudwego/abcoder/lang/uniast" + "github.com/cloudwego/abcoder/lang/utils" ) type dependency struct { @@ -38,6 +39,9 @@ func (c *Collector) fileLine(loc Location) uniast.FileLine { } else { rel = filepath.Base(loc.URI.File()) } + if c.cli.GetFile(loc.URI) == nil { + return uniast.FileLine{} + } text := c.cli.GetFile(loc.URI).Text file_uri := string(loc.URI) return uniast.FileLine{ @@ -53,6 +57,16 @@ func newModule(name string, dir string, lang uniast.Language) *uniast.Module { return ret } +func (c *Collector) ExportLocalFunction() map[Location]*DocumentSymbol { + if len(c.localFunc) == 0 { + c.localFunc = make(map[Location]*DocumentSymbol) + for symbol := range c.funcs { + c.localFunc[symbol.Location] = symbol + } + } + return c.localFunc +} + func (c *Collector) Export(ctx context.Context) (*uniast.Repository, error) { // recursively read all go files in repo repo := uniast.NewRepository(c.repo) @@ -85,7 +99,7 @@ func (c *Collector) Export(ctx context.Context) (*uniast.Repository, error) { continue } - modpath, pkgpath, err := c.spec.NameSpace(fp) + modpath, pkgpath, err := c.spec.NameSpace(fp, f) if err != nil { continue } @@ -127,6 +141,9 @@ func (c *Collector) filterLocalSymbols() { continue } if loc2.Include(loc1) { + if utils.Contains(c.spec.ProtectedSymbolKinds(), c.syms[loc1].Kind) { + break + } delete(c.syms, loc1) break } @@ -145,13 +162,35 @@ func (c *Collector) exportSymbol(repo *uniast.Repository, symbol *DocumentSymbol e = errors.New("symbol is nil") return } + + // 判断是否为本地符号 + // 只有符号是“定义”，或者符号是“本地方法”时，才需要完整导出 + // 其他情况（如外部引用、或对本地非顶层符号的引用）都只导出标识符 + isDefinition := symbol.Role == DEFINITION + _, isLocalMethod := c.funcs[symbol] + _, isLocalSymbol := c.syms[symbol.Location] + if !isDefinition { + if isLocalSymbol { + //引用类型符号，把引用类型符号替换为local 符号 + symbol = c.syms[symbol.Location] + } else { + if symbol.Kind == SKFunction || symbol.Kind == SKMethod { + documentSymbol := c.ExportLocalFunction()[symbol.Location] + if documentSymbol != nil { + symbol = documentSymbol + } + } + + } + } + if id, ok := visited[symbol]; ok { return id, nil } // Check NeedStdSymbol file := symbol.Location.URI.File() - mod, path, err := c.spec.NameSpace(file) + mod, path, err := c.spec.NameSpace(file, c.files[file]) if err != nil { e = err return @@ -200,6 +239,14 @@ func (c *Collector) exportSymbol(repo *uniast.Repository, symbol *DocumentSymbol content := symbol.Text public := c.spec.IsPublicSymbol(*symbol) + if !isDefinition && !isLocalMethod && !isLocalSymbol { + defs, err := c.cli.Definition(context.Background(), symbol.Location.URI, symbol.Location.Range.Start) + if err != nil || len(defs) == 0 { + // 意味着引用为外部符号，LSP 无法查询到符号定位 + return id, err + } + } + // map receiver to methods receivers := make(map[*DocumentSymbol][]*DocumentSymbol, len(c.funcs)/4) for method, rec := range c.funcs { @@ -343,6 +390,7 @@ func (c *Collector) exportSymbol(repo *uniast.Repository, symbol *DocumentSymbol switch dep.Symbol.Kind { case SKStruct, SKTypeParameter, SKInterface, SKEnum, SKClass: obj.SubStruct = uniast.InsertDependency(obj.SubStruct, uniast.NewDependency(*depid, c.fileLine(dep.Location))) + case SKConstant, SKVariable: default: log.Error("dep symbol %s not collected for \n", dep.Symbol, id) } diff --git a/lang/cxx/spec.go b/lang/cxx/spec.go index 43ddd3fb..92db0c36 100644 --- a/lang/cxx/spec.go +++ b/lang/cxx/spec.go @@ -28,6 +28,10 @@ type CxxSpec struct { repo string } +func (c *CxxSpec) ProtectedSymbolKinds() []lsp.SymbolKind { + return []lsp.SymbolKind{} +} + func NewCxxSpec() *CxxSpec { return &CxxSpec{} } @@ -51,7 +55,7 @@ func (c *CxxSpec) WorkSpace(root string) (map[string]string, error) { // returns: modname, pathpath, error // Multiple symbols with the same name could occur (for example in the Linux kernel). // The identify is mod::pkg::name. So we use the pkg (the file name) to distinguish them. -func (c *CxxSpec) NameSpace(path string) (string, string, error) { +func (c *CxxSpec) NameSpace(path string, file *uniast.File) (string, string, error) { // external lib: only standard library (system headers), in /usr/ if !strings.HasPrefix(path, c.repo) { if strings.HasPrefix(path, "/usr") { diff --git a/lang/java/lib.go b/lang/java/lib.go new file mode 100644 index 00000000..df4384d7 --- /dev/null +++ b/lang/java/lib.go @@ -0,0 +1,263 @@ +// Copyright 2025 CloudWeGo Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package java + +import ( + "archive/tar" + "compress/gzip" + "fmt" + "io" + "log" + "net/http" + "os" + "path/filepath" + "runtime" + "strings" + "time" + + "github.com/cloudwego/abcoder/lang/uniast" + "github.com/cloudwego/abcoder/lang/utils" +) + +const ( + MaxWaitDuration = 5 * time.Second + jdtlsVersion = "1.39.0-202408291433" + jdtlsURL = "https://download.eclipse.org/jdtls/milestones/1.39.0/jdt-language-server-1.39.0-202408291433.tar.gz" +) + +// untar takes a destination path and a reader; a tar reader loops over the tar file +// and writes each file to the destination path. +func untar(dst string, r io.Reader) error { + gzr, err := gzip.NewReader(r) + if err != nil { + return err + } + defer gzr.Close() + + tr := tar.NewReader(gzr) + + for { + header, err := tr.Next() + + switch { + // if no more files are found return + case err == io.EOF: + return nil + // return any other error + case err != nil: + return err + // if the header is nil, just skip it (not sure how this happens) + case header == nil: + continue + } + + // the target location where the dir/file should be created + target := filepath.Join(dst, header.Name) + + // check the file type + switch header.Typeflag { + + // if its a dir and it doesn't exist create it + case tar.TypeDir: + if _, err := os.Stat(target); err != nil { + if err := os.MkdirAll(target, 0755); err != nil { + return err + } + } + + // if it's a file create it + case tar.TypeReg: + // make sure the directory for the file exists + if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil { + return err + } + + f, err := os.OpenFile(target, os.O_CREATE|os.O_RDWR, os.FileMode(header.Mode)) + if err != nil { + return err + } + + // copy over contents + if _, err := io.Copy(f, tr); err != nil { + f.Close() + return err + } + + // manually close here after each file operation; defering would cause each file close + // to wait until all operations have completed. + f.Close() + } + } +} + +func setupJDTLS() (string, error) { + _, currentFile, _, ok := runtime.Caller(0) + if !ok { + return "", fmt.Errorf("failed to get current file path") + } + javaDir := filepath.Dir(currentFile) + installDir := filepath.Join(javaDir, "lsp", "jdtls") + + // Check for any existing JDTLS installation + existingDirs, err := filepath.Glob(filepath.Join(installDir, "jdt-language-server-*")) + if err == nil && len(existingDirs) > 0 { + for _, dir := range existingDirs { + info, err := os.Stat(dir) + if err == nil && info.IsDir() { + // Check if launcher jar exists in this directory + launcherPattern := filepath.Join(dir, "plugins", "org.eclipse.equinox.launcher_*.jar") + matches, err := filepath.Glob(launcherPattern) + if err == nil && len(matches) > 0 { + log.Printf("Found existing JDT Language Server at %s. Skipping installation.", dir) + return dir, nil + } + } + } + } + + log.Printf("JDT Language Server not found locally. Downloading and installing version %s...", jdtlsVersion) + jdtlsDir := filepath.Join(installDir, "jdt-language-server-"+jdtlsVersion) + + // Create download directory + downloadDir := filepath.Join(installDir, "download") + if err := os.MkdirAll(downloadDir, 0755); err != nil { + return "", fmt.Errorf("failed to create download directory: %w", err) + } + + // Download + tarballName := "jdt-language-server-" + jdtlsVersion + ".tar.gz" + tarballPath := filepath.Join(downloadDir, tarballName) + log.Printf("Downloading from %s...", jdtlsURL) + resp, err := http.Get(jdtlsURL) + if err != nil { + return "", fmt.Errorf("failed to download JDTLS: %w", err) + } + defer resp.Body.Close() + + if resp.StatusCode != http.StatusOK { + return "", fmt.Errorf("failed to download JDTLS: received status code %d", resp.StatusCode) + } + + out, err := os.Create(tarballPath) + if err != nil { + return "", fmt.Errorf("failed to create tarball file: %w", err) + } + //defer os.Remove(tarballPath) // Clean up tarball after function returns + + _, err = io.Copy(out, resp.Body) + if err != nil { + out.Close() + return "", fmt.Errorf("failed to save tarball: %w", err) + } + out.Close() // Close file before untarring + + // Extract + log.Printf("Extracting to %s...", installDir) + file, err := os.Open(tarballPath) + if err != nil { + return "", fmt.Errorf("failed to open tarball: %w", err) + } + defer file.Close() + + if err := untar(jdtlsDir, file); err != nil { + return "", fmt.Errorf("failed to extract JDTLS: %w", err) + } + + log.Printf("JDT Language Server installed successfully in %s.", jdtlsDir) + return jdtlsDir, nil +} + +func GetDefaultLSP(LspOptions map[string]string) (lang uniast.Language, name string) { + return uniast.Java, generateExecuteCmd(LspOptions) +} + +func generateExecuteCmd(LspOptions map[string]string) string { + var jdtRootPATH string + // First, check environment variable + if envPath := os.Getenv("JDTLS_ROOT_PATH"); len(envPath) != 0 { + jdtRootPATH = envPath + log.Printf("Using JDTLS_ROOT_PATH from environment: %s", jdtRootPATH) + } else { + // If env var is not set, run auto-setup + var err error + jdtRootPATH, err = setupJDTLS() + if err != nil { + panic(fmt.Sprintf("Failed to setup JDT Language Server: %v", err)) + } + } + + // Get the absolute path to the current file + _, currentFile, _, ok := runtime.Caller(0) + if !ok { + panic("Failed to get current file path") + } + javaDir := filepath.Dir(currentFile) + + // Find launcher jar + launcherPattern := filepath.Join(jdtRootPATH, "plugins", "org.eclipse.equinox.launcher_*.jar") + matches, err := filepath.Glob(launcherPattern) + if err != nil || len(matches) == 0 { + panic(fmt.Sprintf("Could not find org.eclipse.equinox.launcher_*.jar in %s/plugins", jdtRootPATH)) + } + jdtLsPath := matches[0] + + // Determine the configuration path based on OS and architecture + var osName string + switch runtime.GOOS { + case "darwin": + osName = "mac" + case "windows": + osName = "win" + default: + osName = runtime.GOOS + } + configDir := fmt.Sprintf("config_%s", osName) + if runtime.GOARCH == "arm64" { + configDir += "_arm" + } + configPath := filepath.Join(jdtRootPATH, configDir) + dataPath := filepath.Join(javaDir, "lsp", "jdtls", "runtime") + args := []string{ + "-Declipse.application=org.eclipse.jdt.ls.core.id1", + "-Dosgi.bundles.defaultStartLevel=4", + "-Declipse.product=org.eclipse.jdt.ls.core.product", + "-Dlog.level=ALL", + "-noverify", + "-Xmx1G", + fmt.Sprintf("-jar %s", jdtLsPath), + fmt.Sprintf("-configuration %s", configPath), + fmt.Sprintf("-data %s", dataPath), + "--add-modules=ALL-SYSTEM", + "--add-opens java.base/java.util=ALL-UNNAMED", + "--add-opens java.base/java.lang=ALL-UNNAMED", + } + javaCmd := "java " + if len(LspOptions["java.home"]) != 0 { + javaCmd = LspOptions["java.home"] + " " + } + return javaCmd + strings.Join(args, " ") +} + +func CheckRepo(repo string) (string, time.Duration) { + openfile := "" + + // Give the LSP sometime to initialize + _, size := utils.CountFiles(repo, ".java", "SKIPDIR") + wait := 2*time.Second + time.Second*time.Duration(size/1024) + if wait > MaxWaitDuration { + wait = MaxWaitDuration + } + return openfile, wait +} diff --git a/lang/java/lsp/client_test.go b/lang/java/lsp/client_test.go new file mode 100644 index 00000000..e5d1387e --- /dev/null +++ b/lang/java/lsp/client_test.go @@ -0,0 +1,368 @@ +// Copyright 2025 CloudWeGo Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package lsp + +import ( + "context" + "fmt" + "github.com/cloudwego/abcoder/lang/uniast" + "strings" + "testing" + "time" + + "github.com/cloudwego/abcoder/lang/java" + "github.com/cloudwego/abcoder/lang/lsp" + "github.com/stretchr/testify/require" +) + +// TestJavaLSPConnection demonstrates connecting to an external Java LSP server. +func TestJavaLSPConnection(t *testing.T) { + + projectRoot := "../../../testdata/java/0_simple" + ctx := context.Background() + + openfile, wait := java.CheckRepo(projectRoot) + l, s := java.GetDefaultLSP(make(map[string]string)) + lsp.RegisterProvider(uniast.Java, &JavaProvider{}) + + lspClient, err := lsp.NewLSPClient(projectRoot, openfile, wait, lsp.ClientOptions{ + Server: s, + Language: l, + Verbose: false, + }) + if err != nil { + t.Fatalf("init lspclient failed = %v\n", err) + } + + // --- Step 1 & 2: Connect and initialize the LSP client --- + require.NoError(t, err, "Failed to initialize LSP client") + + lspClient.InitFiles() + + // --- Step 3: Open the document to prepare for analysis --- + fileToAnalyze := "../../../testdata/java/0_simple/HelloWorld.java" + fileURI := lsp.NewURI(fileToAnalyze) + + _, err = lspClient.DidOpen(ctx, fileURI) + require.NoError(t, err, "textDocument/didOpen notification failed") + + // --- Step 4: Send the 'textDocument/documentSymbol' request to get the syntax tree --- + symbols, err := lspClient.DocumentSymbols(ctx, fileURI) + require.NoError(t, err, "textDocument/documentSymbol request failed") + + // --- Step 5: Process and print the response --- + require.NotEmpty(t, symbols, "Expected to receive symbols, but got none") + + fmt.Println("Successfully retrieved document symbols for HelloWorld.java:") + for k, s := range symbols { + printSymbol(k, s, 0) + } + + // --- Step 6: Send the 'textDocument/hover' request to get method type info --- + hoverResult, err := lspClient.Hover(ctx, fileURI, 11, 25) + require.NoError(t, err, "textDocument/hover request failed") + + fmt.Println("\n--- Hover Result for testFunction ---") + require.NotEmpty(t, hoverResult.Contents, "Expected hover to have content") + fmt.Printf("Hover Content: %s\n", hoverResult.Contents[0].Value) + fmt.Println("-------------------------------------") +} + +// printSymbol is a helper to recursively print the symbol structure. +func printSymbol(r lsp.Range, symbol *lsp.DocumentSymbol, indentLevel int) { + indent := "" + for i := 0; i < indentLevel; i++ { + indent += " " + } + fmt.Printf("%s- Name: %s, Kind: %s\n", indent, symbol.Name, symbol.Kind) + for _, child := range symbol.Children { + printSymbol(r, child, indentLevel+1) + } +} + +// findSymbolByName recursively finds a symbol by name in a list of document symbols. +func findSymbolByName(symbols []*lsp.DocumentSymbol, name string) *lsp.DocumentSymbol { + for _, s := range symbols { + if s.Name == name { + return s + } + if child := findSymbolByName(s.Children, name); child != nil { + return child + } + } + return nil +} + +func TestJavaLSPSemanticFeatures(t *testing.T) { + projectRoot := "../../../testdata/java/1_advanced" // New project root + ctx := context.Background() + + openfile, wait := java.CheckRepo(projectRoot) + l, s := java.GetDefaultLSP(make(map[string]string)) + lsp.RegisterProvider(uniast.Java, &JavaProvider{}) + + lspClient, err := lsp.NewLSPClient(projectRoot, openfile, wait, lsp.ClientOptions{ + Server: s, + Language: l, + Verbose: true, + }) + if err != nil { + t.Fatalf("init lspclient failed = %v\n", err) + } + + // --- Step 1 & 2: Connect and initialize the LSP client --- + require.NoError(t, err, "Failed to initialize LSP client") + // lspClient.SetVerbose(true) + lspClient.InitFiles() + + // --- Step 3: Open all relevant documents to make them known to the LSP server --- + animalFile := "../../../testdata/java/1_advanced/src/main/java/org/example/Animal.java" + dogFile := "../../../testdata/java/1_advanced/src/main/java/org/example/Dog.java" + catFile := "../../../testdata/java/1_advanced/src/main/java/org/example/Cat.java" + + animalURI := lsp.NewURI(animalFile) + dogURI := lsp.NewURI(dogFile) + catURI := lsp.NewURI(catFile) + + _, err = lspClient.DidOpen(ctx, animalURI) + require.NoError(t, err, "textDocument/didOpen failed for Animal.java") + _, err = lspClient.DidOpen(ctx, dogURI) + require.NoError(t, err, "textDocument/didOpen failed for Dog.java") + _, err = lspClient.DidOpen(ctx, catURI) + require.NoError(t, err, "textDocument/didOpen failed for Cat.java") + + // Allow time for the LSP server to index the files before querying. + time.Sleep(2 * time.Second) + + // --- Step 5: Test 'textDocument/implementation' to find implementations of Animal --- + animalSymbolsMap, err := lspClient.DocumentSymbols(ctx, animalURI) + require.NoError(t, err, "textDocument/documentSymbol request failed for Animal.java") + require.NotEmpty(t, animalSymbolsMap, "Expected to find symbols in Animal.java") + + var animalSymbols []*lsp.DocumentSymbol + for _, s := range animalSymbolsMap { + animalSymbols = append(animalSymbols, s) + } + + animalInterfaceSymbol := findSymbolByName(animalSymbols, "Animal") + require.NotNil(t, animalInterfaceSymbol, "Could not find 'Animal' interface symbol") + + implementations, err := lspClient.Implementation(ctx, animalURI, animalInterfaceSymbol.Location.Range.Start) + require.NoError(t, err, "textDocument/implementation request failed") + require.Len(t, implementations, 2, "Expected to find 2 implementations of Animal interface") + + fmt.Println("\n--- Found 2 implementations for interface 'Animal' ---") + var foundDog, foundCat bool + for _, impl := range implementations { + if strings.HasSuffix(string(impl.URI), "Dog.java") { + foundDog = true + } + if strings.HasSuffix(string(impl.URI), "Cat.java") { + foundCat = true + } + } + require.True(t, foundDog, "Did not find implementation in Dog.java") + require.True(t, foundCat, "Did not find implementation in Cat.java") + fmt.Println("---------------------------------------------------------") + + // --- Step 6: Test 'textDocument/definition' for a cross-file scenario --- + // This part remains the same, as it verifies that we can still go from an implementation to the definition. + // We will find the definition of `makeSound` from the `Dog` class implementation. + + // First, find the position of the 'makeSound' method in Dog.java using FileStructure + dogSymbols2, err := lspClient.FileStructure(ctx, dogURI) + require.NoError(t, err, "FileStructure request failed for Dog.java") + makeSoundInDogSymbol := findSymbolByName(dogSymbols2, "makeSound()") + require.NotNil(t, makeSoundInDogSymbol, "Could not find 'makeSound' method in 'Dog' class") + +} + +func TestJavaLSPInheritanceFeatures(t *testing.T) { + projectRoot := "../../../testdata/java/2_inheritance" + ctx := context.Background() + + openfile, wait := java.CheckRepo(projectRoot) + l, s := java.GetDefaultLSP(make(map[string]string)) + lsp.RegisterProvider(uniast.Java, &JavaProvider{}) + + lspClient, err := lsp.NewLSPClient(projectRoot, openfile, wait, lsp.ClientOptions{ + Server: s, + Language: l, + Verbose: false, + }) + if err != nil { + t.Fatalf("init lspclient failed = %v\n", err) + } + + require.NoError(t, err, "Failed to initialize LSP client") + // lspClient.SetVerbose(true) + + lspClient.InitFiles() + + shapeFile := "../../../testdata/java/2_inheritance/src/main/java/org/example/Shape.java" + circleFile := "../../../testdata/java/2_inheritance/src/main/java/org/example/Circle.java" + rectangleFile := "../../../testdata/java/2_inheritance/src/main/java/org/example/Rectangle.java" + + shapeURI := lsp.NewURI(shapeFile) + circleURI := lsp.NewURI(circleFile) + rectangleURI := lsp.NewURI(rectangleFile) + + _, err = lspClient.DidOpen(ctx, shapeURI) + require.NoError(t, err, "textDocument/didOpen failed for Shape.java") + _, err = lspClient.DidOpen(ctx, circleURI) + require.NoError(t, err, "textDocument/didOpen failed for Circle.java") + _, err = lspClient.DidOpen(ctx, rectangleURI) + require.NoError(t, err, "textDocument/didOpen failed for Rectangle.java") + + time.Sleep(2 * time.Second) + + // --- Step 1: Test 'textDocument/references' for the abstract method --- + shapeSymbols, err := lspClient.FileStructure(ctx, shapeURI) + require.NoError(t, err, "FileStructure request failed for Shape.java") + drawMethodSymbol := findSymbolByName(shapeSymbols, "draw()") + require.NotNil(t, drawMethodSymbol, "Could not find 'draw' method in 'Shape' class") + + references, err := lspClient.References(ctx, drawMethodSymbol.Location) + require.NoError(t, err, "textDocument/references request failed") + require.Len(t, references, 3, "Expected to find 3 references to draw(), including the declaration") + + fmt.Println("\n--- Found 3 references for abstract method 'draw()' ---") + + var foundCircle, foundRectangle, foundShape bool + for _, ref := range references { + if strings.HasSuffix(string(ref.URI), "Circle.java") { + foundCircle = true + } + if strings.HasSuffix(string(ref.URI), "Rectangle.java") { + foundRectangle = true + } + if strings.HasSuffix(string(ref.URI), "Shape.java") { + foundShape = true + } + } + require.True(t, foundCircle, "Did not find reference in Circle.java") + require.True(t, foundRectangle, "Did not find reference in Rectangle.java") + require.True(t, foundShape, "Did not find reference in Shape.java") + + // --- Step 2: Test 'textDocument/definition' from implementation to abstract class --- + circleSymbols, err := lspClient.FileStructure(ctx, circleURI) + require.NoError(t, err, "FileStructure request failed for Circle.java") + drawInCircleSymbol := findSymbolByName(circleSymbols, "draw()") + require.NotNil(t, drawInCircleSymbol, "Could not find 'draw' method in 'Circle' class") + + // --- Step 3: Test 'textDocument/typeDefinition' on a class instance --- + circleSymbolsForType, err := lspClient.FileStructure(ctx, circleURI) + require.NoError(t, err, "FileStructure request failed for Circle.java") + circleClassSymbol := findSymbolByName(circleSymbolsForType, "Circle") + require.NotNil(t, circleClassSymbol, "Could not find 'Circle' class symbol") + + typeDefinitionResult, err := lspClient.TypeDefinition(ctx, circleURI, circleClassSymbol.Location.Range.Start) + require.NoError(t, err, "textDocument/typeDefinition request failed") + require.NotEmpty(t, typeDefinitionResult, "Expected a type definition result") + typeDefinition := typeDefinitionResult[0] + require.True(t, strings.HasSuffix(string(typeDefinition.URI), "Circle.java"), "Type definition should be in Circle.java") + + fmt.Println("\n--- Go to Type Definition Result for Circle ---") + fmt.Printf("Type Definition found at: %s, Line: %d\n", typeDefinition.URI, typeDefinition.Range.Start.Line+1) +} + +func TestJavaLSPTypeHierarchy(t *testing.T) { + projectRoot := "../../../testdata/java/2_inheritance" + ctx := context.Background() + + openfile, wait := java.CheckRepo(projectRoot) + l, s := java.GetDefaultLSP(make(map[string]string)) + lsp.RegisterProvider(uniast.Java, &JavaProvider{}) + + lspClient, err := lsp.NewLSPClient(projectRoot, openfile, wait, lsp.ClientOptions{ + Server: s, + Language: l, + Verbose: false, + }) + if err != nil { + t.Fatalf("init lspclient failed = %v\n", err) + } + require.NoError(t, err, "Failed to initialize LSP client") + // lspClient.SetVerbose(true) + + lspClient.InitFiles() + + shapeFile := "../../../testdata/java/2_inheritance/src/main/java/org/example/Shape.java" + circleFile := "../../../testdata/java/2_inheritance/src/main/java/org/example/Circle.java" + rectangleFile := "../../../testdata/java/2_inheritance/src/main/java/org/example/Rectangle.java" + + shapeURI := lsp.NewURI(shapeFile) + circleURI := lsp.NewURI(circleFile) + rectangleURI := lsp.NewURI(rectangleFile) + + _, err = lspClient.DidOpen(ctx, shapeURI) + require.NoError(t, err, "textDocument/didOpen failed for Shape.java") + _, err = lspClient.DidOpen(ctx, circleURI) + require.NoError(t, err, "textDocument/didOpen failed for Circle.java") + _, err = lspClient.DidOpen(ctx, rectangleURI) + require.NoError(t, err, "textDocument/didOpen failed for Rectangle.java") + + time.Sleep(2 * time.Second) + + // --- Step 1: Test 'typeHierarchy/subtypes' for Shape --- + shapeSymbols, err := lspClient.FileStructure(ctx, shapeURI) + require.NoError(t, err, "FileStructure request failed for Shape.java") + shapeSymbol := findSymbolByName(shapeSymbols, "Shape") + require.NotNil(t, shapeSymbol, "Could not find 'Shape' class symbol") + + // Prepare the type hierarchy + shapeItems, err := lspClient.PrepareTypeHierarchy(ctx, shapeURI, shapeSymbol.Location.Range.Start) + require.NoError(t, err, "textDocument/prepareTypeHierarchy request failed for Shape") + require.Len(t, shapeItems, 1, "Expected one type hierarchy item for Shape") + shapeItem := shapeItems[0] + + // Get subtypes + subtypes, err := lspClient.TypeHierarchySubtypes(ctx, shapeItem) + require.NoError(t, err, "typeHierarchy/subtypes request failed") + require.Len(t, subtypes, 2, "Expected to find 2 subtypes of Shape") + + fmt.Println("\n--- Found 2 subtypes for class 'Shape' ---") + var foundCircle, foundRectangle bool + for _, child := range subtypes { + if child.Name == "Circle" { + foundCircle = true + } + if child.Name == "Rectangle" { + foundRectangle = true + } + } + require.True(t, foundCircle, "Did not find subtype Circle") + require.True(t, foundRectangle, "Did not find subtype Rectangle") + + // --- Step 2: Test 'typeHierarchy/supertypes' for Circle --- + circleSymbols, err := lspClient.FileStructure(ctx, circleURI) + require.NoError(t, err, "FileStructure request failed for Circle.java") + circleSymbol := findSymbolByName(circleSymbols, "Circle") + require.NotNil(t, circleSymbol, "Could not find 'Circle' class symbol") + + // Prepare the type hierarchy + circleItems, err := lspClient.PrepareTypeHierarchy(ctx, circleURI, circleSymbol.Location.Range.Start) + require.NoError(t, err, "textDocument/prepareTypeHierarchy request failed for Circle") + require.Len(t, circleItems, 1, "Expected one type hierarchy item for Circle") + circleItem := circleItems[0] + + // Get supertypes + supertypes, err := lspClient.TypeHierarchySupertypes(ctx, circleItem) + require.NoError(t, err, "typeHierarchy/supertypes request failed") + require.Len(t, supertypes, 1, "Expected to find 1 supertype of Circle") + + fmt.Println("\n--- Found 1 supertype for class 'Circle' ---") + require.Equal(t, "Shape", supertypes[0].Name, "Supertype of Circle should be Shape") +} diff --git a/lang/java/lsp/java_lsp.go b/lang/java/lsp/java_lsp.go new file mode 100644 index 00000000..b8424a73 --- /dev/null +++ b/lang/java/lsp/java_lsp.go @@ -0,0 +1,137 @@ +// Copyright 2025 CloudWeGo Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package lsp + +import ( + "context" + "github.com/cloudwego/abcoder/lang/lsp" +) + +// JavaProvider implements the LanguageServiceProvider for Java. +type JavaProvider struct{} + +// jdtHover is a custom struct to handle the hover result from JDT LS +// It supports both MarkupContent object and simple string formats +type jdtHover struct { + Contents interface{} `json:"contents"` + Range *lsp.Range `json:"range,omitempty"` +} + +func (p *JavaProvider) Hover(ctx context.Context, cli *lsp.LSPClient, uri lsp.DocumentURI, line, character int) (*lsp.Hover, error) { + var result jdtHover // Use the custom struct to unmarshal + err := cli.Call(ctx, "textDocument/hover", lsp.TextDocumentPositionParams{ + TextDocument: lsp.TextDocumentIdentifier{URI: uri}, + Position: lsp.Position{Line: line, Character: character}, + }, &result) + if err != nil { + return nil, err + } + + // Handle different response formats + var content string + + // Try to parse as MarkupContent object + if contentsMap, isMap := result.Contents.(map[string]interface{}); isMap { + if value, exists := contentsMap["value"]; exists { + if strValue, isString := value.(string); isString { + content = strValue + } + } + } else if strContent, isString := result.Contents.(string); isString { + // Handle simple string response + content = strContent + } + + // Convert the JDT-specific hover result to the standard lsp.Hover type. + standardHover := &lsp.Hover{ + Contents: []lsp.MarkedString{ + { + Language: "java", + Value: content, + }, + }, + Range: &lsp.Range{}, + } + + return standardHover, nil +} + +func (p *JavaProvider) Implementation(ctx context.Context, cli *lsp.LSPClient, uri lsp.DocumentURI, pos lsp.Position) ([]lsp.Location, error) { + var result []lsp.Location + err := cli.Call(ctx, "textDocument/implementation", lsp.TextDocumentPositionParams{ + TextDocument: lsp.TextDocumentIdentifier{URI: uri}, + Position: pos, + }, &result) + if err != nil { + return nil, err + } + return result, nil +} + +func (p *JavaProvider) WorkspaceSearchSymbols(ctx context.Context, cli *lsp.LSPClient, query string) ([]lsp.SymbolInformation, error) { + req := lsp.WorkspaceSymbolParams{ + Query: query, + } + var resp []lsp.SymbolInformation + if err := cli.Call(ctx, "workspace/symbol", req, &resp); err != nil { + return nil, err + } + return resp, nil +} + +// PrepareTypeHierarchy performs a textDocument/prepareTypeHierarchy request. +func (p *JavaProvider) PrepareTypeHierarchy(ctx context.Context, cli *lsp.LSPClient, uri lsp.DocumentURI, pos lsp.Position) ([]lsp.TypeHierarchyItem, error) { + params := lsp.TextDocumentPositionParams{ + TextDocument: lsp.TextDocumentIdentifier{URI: uri}, + Position: pos, + } + + var result []lsp.TypeHierarchyItem + err := cli.Call(ctx, "textDocument/prepareTypeHierarchy", params, &result) + if err != nil { + return nil, err + } + return result, nil +} + +// TypeHierarchySupertypes requests the supertypes of a symbol. +func (p *JavaProvider) TypeHierarchySupertypes(ctx context.Context, cli *lsp.LSPClient, item lsp.TypeHierarchyItem) ([]lsp.TypeHierarchyItem, error) { + params := struct { + Item lsp.TypeHierarchyItem `json:"item"` + }{ + Item: item, + } + var result []lsp.TypeHierarchyItem + err := cli.Call(ctx, "typeHierarchy/supertypes", params, &result) + if err != nil { + return nil, err + } + return result, nil +} + +// TypeHierarchySubtypes requests the subtypes of a symbol. +func (p *JavaProvider) TypeHierarchySubtypes(ctx context.Context, cli *lsp.LSPClient, item lsp.TypeHierarchyItem) ([]lsp.TypeHierarchyItem, error) { + params := struct { + Item lsp.TypeHierarchyItem `json:"item"` + }{ + Item: item, + } + var result []lsp.TypeHierarchyItem + err := cli.Call(ctx, "typeHierarchy/subtypes", params, &result) + if err != nil { + return nil, err + } + return result, nil +} diff --git a/lang/java/parser/parser.go b/lang/java/parser/parser.go new file mode 100644 index 00000000..375bf84b --- /dev/null +++ b/lang/java/parser/parser.go @@ -0,0 +1,112 @@ +// Copyright 2025 CloudWeGo Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package parser + +import ( + "context" + "log" + "sync" + "unicode/utf16" + "unicode/utf8" + + "github.com/cloudwego/abcoder/lang/uniast" + sitter "github.com/smacker/go-tree-sitter" + "github.com/smacker/go-tree-sitter/java" +) + +var ( + once sync.Once + parser *sitter.Parser +) + +func NewParser() *sitter.Parser { + once.Do(func() { + parser = sitter.NewParser() + parser.SetLanguage(java.GetLanguage()) + }) + return parser +} + +func GetLanguage(l uniast.Language) *sitter.Language { + switch l { + case uniast.Java: + return java.GetLanguage() + } + return nil +} + +func Parse(ctx context.Context, content []byte) (*sitter.Tree, error) { + p := NewParser() + tree, err := p.ParseCtx(ctx, nil, content) + if err != nil { + log.Printf("Error parsing content: %v", err) + return nil, err + } + return tree, nil +} + +func Utf8ToUtf16Position(content []byte, row, byteColumn uint32) (line, character int) { + // 计算到指定行的起始位置 + lineStart := 0 + currentLine := uint32(0) + + for i := 0; i < len(content); { + if currentLine == row { + lineStart = i + break + } + if content[i] == '\n' { + currentLine++ + } + i++ + } + + // 计算UTF-16字符位置 + utf16Pos := 0 + for i := lineStart; i < lineStart+int(byteColumn); { + r, size := utf8.DecodeRune(content[i:]) + if r == utf8.RuneError { + break + } + utf16Pos += utf16.RuneLen(r) + i += size + } + + return int(row), utf16Pos +} + +func FindChildIdentifier(node *sitter.Node) *sitter.Node { + var pkgNameNode *sitter.Node + for i := 0; i < int(node.ChildCount()); i++ { + child := node.Child(i) + if child.Type() == "identifier" || child.Type() == "scoped_identifier" { + pkgNameNode = child + break + } + } + return pkgNameNode +} + +func FindChildByType(node *sitter.Node, typeString string) *sitter.Node { + var pkgNameNode *sitter.Node + for i := 0; i < int(node.ChildCount()); i++ { + child := node.Child(i) + if child.Type() == typeString { + pkgNameNode = child + break + } + } + return pkgNameNode +} diff --git a/lang/java/parser/parser_test.go b/lang/java/parser/parser_test.go new file mode 100644 index 00000000..e0d97f80 --- /dev/null +++ b/lang/java/parser/parser_test.go @@ -0,0 +1,94 @@ +// Copyright 2025 CloudWeGo Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package parser + +import ( + "context" + "io/ioutil" + "strings" + "testing" + + sitter "github.com/smacker/go-tree-sitter" + "github.com/stretchr/testify/assert" +) + +func TestParse(t *testing.T) { + fileToAnalyze := "../../../testdata/java/0_simple/HelloWorld.java" + + content, err := ioutil.ReadFile(fileToAnalyze) + assert.NoError(t, err) + + tree, err := Parse(context.Background(), content) + assert.NoError(t, err) + assert.NotNil(t, tree) + + root := tree.RootNode() + assert.NotNil(t, root) + + // A simple check to see if we have a reasonable root node type + assert.Equal(t, "program", root.Type()) +} + +func TestPrintTree(t *testing.T) { + fileToAnalyze := "../../../testdata/java/0_simple/HelloWorld.java" + + content, err := ioutil.ReadFile(fileToAnalyze) + assert.NoError(t, err) + + tree, err := Parse(context.Background(), content) + assert.NoError(t, err) + + var printNode func(*sitter.Node, int) + printNode = func(node *sitter.Node, level int) { + if node == nil { + return + } + + indent := strings.Repeat(" ", level) + contentType := node.Type() + contentStr := strings.ReplaceAll(node.Content(content), "\n", "\\n") + + t.Logf("%s%s (%s) [%d:%d - %d:%d] `%s`", + indent, + contentType, + node.Type(), + node.StartPoint().Row, node.StartPoint().Column, + node.EndPoint().Row, node.EndPoint().Column, + contentStr, + ) + + for i := 0; i < int(node.ChildCount()); i++ { + printNode(node.Child(i), level+1) + } + } + + t.Log("--- Syntax Tree --- ") + printNode(tree.RootNode(), 0) + t.Log("--- End Syntax Tree ---") +} + +func TestDebugTree(t *testing.T) { + fileToAnalyze := "../../../testdata/java/0_simple/HelloWorld.java" + + content, err := ioutil.ReadFile(fileToAnalyze) + assert.NoError(t, err) + + debugTree, err := NewDebugTree(context.Background(), content) + assert.NoError(t, err) + assert.NotNil(t, debugTree) + + // <<<--- PLACE A BREAKPOINT ON THE LINE BELOW ---<<< // + t.Log("Successfully built the debug tree. You can now inspect the 'debugTree' variable.") +} diff --git a/lang/java/parser/pom_parser.go b/lang/java/parser/pom_parser.go new file mode 100644 index 00000000..3c2118a6 --- /dev/null +++ b/lang/java/parser/pom_parser.go @@ -0,0 +1,213 @@ +// Copyright 2025 CloudWeGo Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package parser + +import ( + "fmt" + "log" + "path/filepath" + "regexp" + + "github.com/vifraa/gopom" +) + +// ModuleInfo stores information about a Maven module. +type ModuleInfo struct { + ArtifactID string + GroupID string + Version string + Coordinates string + Path string + SourcePath string + TestSourcePath string + TargetPath string + SubModules []*ModuleInfo + Properties map[string]string +} + +// ParseMavenProject recursively parses a module and its submodules. +// pomPath: The path to the pom.xml file to parse. +func ParseMavenProject(pomPath string) (*ModuleInfo, error) { + return parseMavenProject(pomPath, nil) +} + +var propRegex = regexp.MustCompile(`\$\{(.+?)\}`) + +func resolveProperty(value string, properties map[string]string) string { + resolvedValue := value + for i := 0; i < 10; i++ { // Limit iterations to prevent infinite loops + newValue := propRegex.ReplaceAllStringFunc(resolvedValue, func(match string) string { + key := match[2 : len(match)-1] + if val, ok := properties[key]; ok { + return val + } + return match + }) + if newValue == resolvedValue { + return newValue + } + resolvedValue = newValue + } + return resolvedValue +} + +func parseMavenProject(pomPath string, parent *ModuleInfo) (*ModuleInfo, error) { + // 1. Parse the pom.xml file using gopom. + project, err := gopom.Parse(pomPath) + if err != nil { + return nil, fmt.Errorf("failed to parse %s: %w", pomPath, err) + } + + // Collect properties from parent and current pom + properties := make(map[string]string) + if parent != nil && parent.Properties != nil { + for k, v := range parent.Properties { + properties[k] = v + } + } + if project.Properties != nil && project.Properties.Entries != nil { + for k, v := range project.Properties.Entries { + properties[k] = v + } + } + + var groupID, version string + if project.GroupID != nil { + groupID = *project.GroupID + } else if parent != nil { + groupID = parent.GroupID + } + + if project.Version != nil { + version = *project.Version + } else if parent != nil { + version = parent.Version + } + + // Resolve properties in version and groupID + version = resolveProperty(version, properties) + groupID = resolveProperty(groupID, properties) + + if project.ArtifactID == nil { + return nil, fmt.Errorf("artifactId is missing in %s", pomPath) + } + + // Determine source and test source directories + modulePath := filepath.Dir(pomPath) + sourcePath := filepath.Join(modulePath, "src", "main", "java") + testSourcePath := filepath.Join(modulePath, "src", "test", "java") + targetPath := filepath.Join(modulePath, "target") + if project.Build != nil { + if project.Build.SourceDirectory != nil { + sourcePath = filepath.Join(modulePath, *project.Build.SourceDirectory) + } + if project.Build.TestSourceDirectory != nil { + testSourcePath = filepath.Join(modulePath, *project.Build.TestSourceDirectory) + } + if project.Build.OutputDirectory != nil { + targetPath = filepath.Join(modulePath, *project.Build.OutputDirectory) + } + } + + // 2. Create a struct to store our module information. + currentModule := &ModuleInfo{ + ArtifactID: *project.ArtifactID, + GroupID: groupID, + Version: version, + Coordinates: fmt.Sprintf("%s:%s:%s", groupID, *project.ArtifactID, version), + Path: modulePath, + SourcePath: sourcePath, + TestSourcePath: testSourcePath, + TargetPath: targetPath, + SubModules: []*ModuleInfo{}, + Properties: properties, + } + + // 3. If a section exists, recursively parse the submodules. + if project.Modules != nil && len(*project.Modules) > 0 { + for _, moduleName := range *project.Modules { + // Construct the path to the submodule's pom.xml. + subPomPath := filepath.Join(currentModule.Path, moduleName, "pom.xml") + + // Recursive call. + subModuleInfo, err := parseMavenProject(subPomPath, currentModule) + if err != nil { + // If parsing a submodule fails, we can log it and skip. + log.Printf("Warning: failed to parse submodule %s: %v", subPomPath, err) + continue + } + currentModule.SubModules = append(currentModule.SubModules, subModuleInfo) + } + } + + return currentModule, nil +} + +func GetModuleMap(root *ModuleInfo) map[string]string { + rets := map[string]string{} + var queue []*ModuleInfo + if root != nil { + queue = append(queue, root) + } + for len(queue) > 0 { + module := queue[0] + queue = queue[1:] + rets[module.Coordinates] = module.Path + for _, subModule := range module.SubModules { + queue = append(queue, subModule) + } + } + return rets +} + +func GetModuleStructMap(root *ModuleInfo) map[string]*ModuleInfo { + rets := map[string]*ModuleInfo{} + var queue []*ModuleInfo + if root != nil { + queue = append(queue, root) + } + for len(queue) > 0 { + module := queue[0] + queue = queue[1:] + rets[module.Coordinates] = module + for _, subModule := range module.SubModules { + queue = append(queue, subModule) + } + } + return rets +} + +func GetModulePaths(root *ModuleInfo) []string { + var paths []string + moduleMap := GetModuleMap(root) + for _, path := range moduleMap { + paths = append(paths, path) + } + return paths +} + +// PrintProjectTree prints the project structure in a hierarchical format. +func PrintProjectTree(module *ModuleInfo, indent string) { + if module == nil { + return + } + // Print current module info. + fmt.Printf("%s- %s\n", indent, module.Coordinates) + + // Recursively print submodules. + for _, subModule := range module.SubModules { + PrintProjectTree(subModule, indent+" ") + } +} diff --git a/lang/java/parser/pom_parser_test.go b/lang/java/parser/pom_parser_test.go new file mode 100644 index 00000000..56966a3d --- /dev/null +++ b/lang/java/parser/pom_parser_test.go @@ -0,0 +1,46 @@ +// Copyright 2025 CloudWeGo Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package parser + +import ( + "path/filepath" + "testing" +) + +func TestParseMavenProject(t *testing.T) { + projectRootPath := "../../../testdata/java/3_java_pom" + rootPomPath := filepath.Join(projectRootPath, "pom.xml") + + rootModule, err := ParseMavenProject(rootPomPath) + if err != nil { + t.Fatalf("Error parsing root project: %v", err) + } + + if rootModule.ArtifactID != "my-app" { + t.Errorf("Expected artifactId to be 'my-app', but got '%s'", rootModule.ArtifactID) + } + + if len(rootModule.SubModules) != 1 { + t.Fatalf("Expected 1 submodule, but got %d", len(rootModule.SubModules)) + } + + subModule := rootModule.SubModules[0] + if subModule.ArtifactID != "my-app-sub" { + t.Errorf("Expected submodule artifactId to be 'my-app-sub', but got '%s'", subModule.ArtifactID) + } + + // Print the tree to visually verify the structure + PrintProjectTree(rootModule, "") +} diff --git a/lang/java/parser/tree_debugger.go b/lang/java/parser/tree_debugger.go new file mode 100644 index 00000000..485c37a6 --- /dev/null +++ b/lang/java/parser/tree_debugger.go @@ -0,0 +1,69 @@ +// Copyright 2025 CloudWeGo Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package parser + +import ( + "context" + + sitter "github.com/smacker/go-tree-sitter" +) + +// DebugNode is a Go-native representation of a tree-sitter node for easy debugging. +// It contains the essential information about a node in a format that is friendly +// to Go's debugging tools. + +type DebugNode struct { + Type string `json:"type"` + Content string `json:"content"` + Start sitter.Point `json:"start"` + End sitter.Point `json:"end"` + Children []*DebugNode `json:"children"` +} + +// NewDebugTree parses the source code and builds a Go-native debug tree. +// This function is designed to be used in testing and debugging scenarios. +func NewDebugTree(ctx context.Context, content []byte) (*DebugNode, error) { + tree, err := Parse(ctx, content) + if err != nil { + return nil, err + } + + rootSitterNode := tree.RootNode() + debugRoot := buildDebugNode(rootSitterNode, content) + + return debugRoot, nil +} + +// buildDebugNode is a recursive helper function that converts a sitter.Node +// into a DebugNode. +func buildDebugNode(node *sitter.Node, content []byte) *DebugNode { + if node == nil { + return nil + } + + children := make([]*DebugNode, 0, node.ChildCount()) + for i := 0; i < int(node.ChildCount()); i++ { + childSitterNode := node.Child(i) + children = append(children, buildDebugNode(childSitterNode, content)) + } + + return &DebugNode{ + Type: node.Type(), + Content: node.Content(content), + Start: node.StartPoint(), + End: node.EndPoint(), + Children: children, + } +} diff --git a/lang/java/spec.go b/lang/java/spec.go new file mode 100644 index 00000000..60764666 --- /dev/null +++ b/lang/java/spec.go @@ -0,0 +1,302 @@ +// Copyright 2025 CloudWeGo Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package java + +import ( + "path/filepath" + "strings" + + javaparser "github.com/cloudwego/abcoder/lang/java/parser" + lsp "github.com/cloudwego/abcoder/lang/lsp" + "github.com/cloudwego/abcoder/lang/uniast" + sitter "github.com/smacker/go-tree-sitter" +) + +type JavaSpec struct { + repo string + rootMod *javaparser.ModuleInfo + // 新增索引 + nameToMod map[string]*javaparser.ModuleInfo // 目录绝对路径 -> module 路径 + dirToPkg map[string]JavaPkg // 目录绝对路径 -> package 路径 +} + +func (c *JavaSpec) ProtectedSymbolKinds() []lsp.SymbolKind { + return []lsp.SymbolKind{lsp.SKFunction} +} + +type JavaPkg struct { + Name string + Path string +} + +func NewJavaSpec(reop string) *JavaSpec { + rootPomPath := filepath.Join(reop, "pom.xml") + rootModule, err := javaparser.ParseMavenProject(rootPomPath) + if err != nil { + return &JavaSpec{ + repo: reop, + rootMod: rootModule, + nameToMod: make(map[string]*javaparser.ModuleInfo), + dirToPkg: make(map[string]JavaPkg), + } + } + nameToMod := javaparser.GetModuleStructMap(rootModule) + + return &JavaSpec{ + repo: reop, + rootMod: rootModule, + nameToMod: nameToMod, + dirToPkg: make(map[string]JavaPkg), + } + +} + +func (c *JavaSpec) FileImports(content []byte) ([]uniast.Import, error) { + // Java import parsing by tree-setting + panic("Java import parsing by tree-setting") +} + +func (c *JavaSpec) WorkSpace(root string) (map[string]string, error) { + rets := javaparser.GetModuleMap(c.rootMod) + return rets, nil +} + +func (c *JavaSpec) PathToMod(path string) *javaparser.ModuleInfo { + + var maxPathmatchMods *javaparser.ModuleInfo + + for _, modInfo := range c.nameToMod { + if strings.HasPrefix(path, modInfo.Path) { + if maxPathmatchMods == nil { + maxPathmatchMods = modInfo + } else if len(modInfo.Path) > len(maxPathmatchMods.Path) { + maxPathmatchMods = modInfo + } + } + } + return maxPathmatchMods +} + +func (c *JavaSpec) NameSpace(path string, file *uniast.File) (string, string, error) { + if !strings.HasPrefix(path, c.repo) { + // External library + return "external", "external", nil + } + + modName := "" + modInfo := c.PathToMod(path) + if modInfo != nil { + modName = modInfo.Coordinates + } + return modName, file.Package, nil +} + +func (c *JavaSpec) ShouldSkip(path string) bool { + // UT 文件不处理 + return !strings.HasSuffix(path, ".java") || c.IsTest(path) || c.IsTarget(path) +} + +func (c *JavaSpec) IsTest(path string) bool { + for _, moduleInfo := range c.nameToMod { + if strings.HasPrefix(path, moduleInfo.TestSourcePath) { + return true + } + } + return false +} +func (c *JavaSpec) IsTarget(path string) bool { + for _, moduleInfo := range c.nameToMod { + if strings.HasPrefix(path, moduleInfo.TargetPath) { + return true + } + } + return false +} + +func (c *JavaSpec) IsDocToken(tok lsp.Token) bool { + return tok.Type == "comment" +} + +func (c *JavaSpec) DeclareTokenOfSymbol(sym lsp.DocumentSymbol) int { + for i, t := range sym.Tokens { + if c.IsDocToken(t) { + continue + } + for _, m := range t.Modifiers { + if m == "declaration" { + return i + } + } + } + return -1 +} + +func (c *JavaSpec) IsEntityToken(tok lsp.Token) bool { + // TODO: refine for Java + return tok.Type == "class_declaration" || tok.Type == "interface_declaration" || tok.Type == "method_declaration" || tok.Type == "static_method_invocation" || tok.Type == "method_invocation" +} + +func (c *JavaSpec) IsStdToken(tok lsp.Token) bool { + // TODO: implement for Java std lib + return tok.Type == "generic_type" || tok.Type == "interface_declaration" || tok.Type == "method_declaration" +} + +func (c *JavaSpec) TokenKind(tok lsp.Token) lsp.SymbolKind { + return NodeTypeToSymbolKind(tok.Type) +} + +func (c *JavaSpec) IsMainFunction(sym lsp.DocumentSymbol) bool { + // A simple heuristic for Java main method + return (sym.Kind == lsp.SKMethod || sym.Kind == lsp.SKFunction) && sym.Name == "main(String[])" +} + +func (c *JavaSpec) IsEntitySymbol(sym lsp.DocumentSymbol) bool { + typ := sym.Kind + return typ == lsp.SKMethod || typ == lsp.SKFunction || typ == lsp.SKVariable || typ == lsp.SKClass || typ == lsp.SKInterface || typ == lsp.SKEnum +} + +func (c *JavaSpec) IsPublicSymbol(sym lsp.DocumentSymbol) bool { + // 使用tree-sitter节点获取modifiers字段，支持类、接口、方法、字段等各种符号类型 + if sym.Node == nil { + return false + } + + // 根据不同符号类型，查找对应的modifiers节点 + var modifiersNode *sitter.Node + + // 处理不同类型的Java符号 + switch sym.Kind { + case lsp.SKClass, lsp.SKInterface, lsp.SKEnum: + // 类、接口、枚举声明 + modifiersNode = sym.Node.ChildByFieldName("modifiers") + if modifiersNode == nil { + // 尝试从父节点获取modifiers + if sym.Node.Type() == "class_declaration" || + sym.Node.Type() == "interface_declaration" || + sym.Node.Type() == "enum_declaration" { + modifiersNode = sym.Node.ChildByFieldName("modifiers") + } + } + + case lsp.SKMethod, lsp.SKConstructor: + // 方法、构造函数 + modifiersNode = sym.Node.ChildByFieldName("modifiers") + if modifiersNode == nil && sym.Node.Type() == "method_declaration" { + modifiersNode = sym.Node.ChildByFieldName("modifiers") + } + + case lsp.SKVariable, lsp.SKField: + // 字段、变量 + modifiersNode = sym.Node.ChildByFieldName("modifiers") + if modifiersNode == nil && sym.Node.Type() == "field_declaration" { + modifiersNode = sym.Node.ChildByFieldName("modifiers") + } + + default: + // 其他类型，尝试通用方式 + modifiersNode = sym.Node.ChildByFieldName("modifiers") + } + + // 如果找到modifiers节点，检查是否包含public + if modifiersNode != nil { + // 遍历所有modifier子节点 + for i := 0; i < int(modifiersNode.ChildCount()); i++ { + modifier := modifiersNode.Child(i) + if modifier != nil && modifier.Type() == "modifier" { + modifierText := modifier.Content([]byte(sym.Text)) + if strings.Contains(strings.ToLower(modifierText), "public") { + return true + } + } + } + + // 直接检查modifiers节点文本 + modifiersText := modifiersNode.Content([]byte(sym.Text)) + return strings.Contains(strings.ToLower(modifiersText), "public") + } + + // 如果没有modifiers节点，检查整个符号文本 + // 处理一些特殊情况，如接口方法默认public + symbolText := strings.ToLower(sym.Text) + + // 接口中的方法默认是public的 + if sym.Kind == lsp.SKMethod && strings.Contains(symbolText, "interface") { + return true + } + + // 检查是否包含public关键字 + return strings.Contains(symbolText, "public") +} + +func (c *JavaSpec) HasImplSymbol() bool { + // For Java `implements` and `extends` + return false +} + +func (c *JavaSpec) ImplSymbol(sym lsp.DocumentSymbol) (int, int, int) { + // Java中的继承和接口实现关系识别 + return -1, -1, -1 +} + +func (c *JavaSpec) FunctionSymbol(sym lsp.DocumentSymbol) (int, []int, []int, []int) { + // TODO: Implement for Java + return -1, nil, nil, nil +} + +func (c *JavaSpec) GetUnloadedSymbol(from lsp.Token, define lsp.Location) (string, error) { + return "", nil +} + +// NodeTypeToSymbolKind maps a tree-sitter node type to the corresponding LSP SymbolKind. +// The mapping is based on the official LSP specification and the tree-sitter-java grammar. +// Ref: https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#symbolKind +func NodeTypeToSymbolKind(nodeType string) lsp.SymbolKind { + switch nodeType { + case "class_declaration": + return lsp.SKClass + case "method_declaration": + return lsp.SKMethod + case "constructor_declaration": + return lsp.SKConstructor + case "field_declaration": + return lsp.SKField + case "enum_declaration": + return lsp.SKEnum + case "enum_constant": + return lsp.SKEnumMember + case "interface_declaration", "super_interfaces": + return lsp.SKInterface + case "annotation_type_declaration": + // Annotations are a form of interface in Java. + return lsp.SKInterface + case "module_declaration": + return lsp.SKModule + case "package_declaration": + return lsp.SKPackage + case "variable_declarator": + // This can be a local variable or a field. Context is needed to be more specific. + // Defaulting to SKVariable. + return lsp.SKVariable + case "type_parameter": + return lsp.SKTypeParameter + // Add more mappings as needed for other node types. + case "type_identifier": + return lsp.SKClass + case "generic_type": + return lsp.SKTypeParameter + default: + return lsp.SKUnknown + } +} diff --git a/lang/lsp/client.go b/lang/lsp/client.go index 750c4191..4afe4ad4 100644 --- a/lang/lsp/client.go +++ b/lang/lsp/client.go @@ -22,6 +22,7 @@ import ( "io" "os" "os/exec" + "strings" "time" "github.com/cloudwego/abcoder/lang/log" @@ -37,23 +38,25 @@ type LSPClient struct { tokenModifiers []string hasSemanticTokensRange bool files map[DocumentURI]*TextDocumentItem + provider LanguageServiceProvider ClientOptions } type ClientOptions struct { Server string uniast.Language - Verbose bool + Verbose bool + InitializationOptions interface{} } func NewLSPClient(repo string, openfile string, wait time.Duration, opts ClientOptions) (*LSPClient, error) { // launch golang LSP server - svr, err := startLSPSever(opts.Server) + svr, err := startLSPSever(opts.Server, opts) if err != nil { return nil, err } - cli, err := initLSPClient(context.Background(), svr, NewURI(repo), opts.Verbose) + cli, err := initLSPClient(context.Background(), svr, NewURI(repo), opts.Verbose, opts.InitializationOptions) if err != nil { return nil, err } @@ -61,6 +64,9 @@ func NewLSPClient(repo string, openfile string, wait time.Duration, opts ClientO cli.ClientOptions = opts cli.files = make(map[DocumentURI]*TextDocumentItem) + cli.provider = GetProvider(opts.Language) + cli.Verbose = opts.Verbose + if openfile != "" { _, err := cli.DidOpen(context.Background(), NewURI(openfile)) if err != nil { @@ -110,7 +116,13 @@ type initializeResult struct { Capabilities interface{} `json:"capabilities,omitempty"` } -func initLSPClient(ctx context.Context, svr io.ReadWriteCloser, dir DocumentURI, verbose bool) (*LSPClient, error) { +func (c *LSPClient) InitFiles() { + if c.files == nil { + c.files = make(map[DocumentURI]*TextDocumentItem) + } +} + +func initLSPClient(ctx context.Context, svr io.ReadWriteCloser, dir DocumentURI, verbose bool, InitializationOptions interface{}) (*LSPClient, error) { h := newLSPHandler() stream := jsonrpc2.NewBufferedStream(svr, jsonrpc2.VSCodeObjectCodec{}) conn := jsonrpc2.NewConn(ctx, stream, h) @@ -124,18 +136,25 @@ func initLSPClient(ctx context.Context, svr io.ReadWriteCloser, dir DocumentURI, // NOTICE: some features need to be enabled explicitly cs := map[string]interface{}{ + "workspace": map[string]interface{}{ + "symbol": map[string]interface{}{ + "dynamicRegistration": true, + }, + }, "documentSymbol": map[string]interface{}{ "hierarchicalDocumentSymbolSupport": true, }, } initParams := initializeParams{ - ProcessID: os.Getpid(), - RootURI: lsp.DocumentURI(dir), - Capabilities: cs, - Trace: lsp.Trace(trace), - ClientInfo: lsp.ClientInfo{Name: "vscode"}, + ProcessID: os.Getpid(), + RootURI: lsp.DocumentURI(dir), + Capabilities: cs, + Trace: lsp.Trace(trace), + ClientInfo: lsp.ClientInfo{Name: "vscode"}, + InitializationOptions: InitializationOptions, } + var initResult initializeResult if err := conn.Call(ctx, "initialize", initParams, &initResult); err != nil { return nil, err @@ -215,9 +234,16 @@ func (rwc rwc) Close() error { } // start a LSP process and return its io -func startLSPSever(path string) (io.ReadWriteCloser, error) { - // Launch rust-analyzer - cmd := exec.Command(path) +func startLSPSever(path string, opts ClientOptions) (io.ReadWriteCloser, error) { + + var cmd *exec.Cmd + if uniast.Java == opts.Language { + parts := strings.Fields(path) + cmd = exec.Command(parts[0], parts[1:]...) + } else { + // Launch rust-analyzer + cmd = exec.Command(path) + } stdin, err := cmd.StdinPipe() if err != nil { diff --git a/lang/lsp/lsp.go b/lang/lsp/lsp.go index d1c4bc85..39c6286d 100644 --- a/lang/lsp/lsp.go +++ b/lang/lsp/lsp.go @@ -15,11 +15,14 @@ package lsp import ( + "context" "encoding/json" "fmt" "path/filepath" "strings" + sitter "github.com/smacker/go-tree-sitter" + "github.com/sourcegraph/go-lsp" ) @@ -56,6 +59,13 @@ const ( type SymbolKind = lsp.SymbolKind +type SymbolRole int + +const ( + DEFINITION SymbolRole = 1 + REFERENCE SymbolRole = 2 +) + type Position lsp.Position func (r Position) Less(s Position) bool { @@ -178,8 +188,78 @@ type DocumentSymbol struct { Children []*DocumentSymbol `json:"children"` Text string `json:"text"` Tokens []Token `json:"tokens"` + Node *sitter.Node `json:"-"` + Role SymbolRole `json:"-"` +} + +type TextDocumentPositionParams struct { + /** + * The text document. + */ + TextDocument TextDocumentIdentifier `json:"textDocument"` + + /** + * The position inside the text document. + */ + Position Position `json:"position"` +} + +type TextDocumentIdentifier struct { + /** + * The text document's URI. + */ + URI DocumentURI `json:"uri"` +} + +type Hover struct { + Contents []MarkedString `json:"contents"` + Range *Range `json:"range,omitempty"` +} + +type MarkedString markedString + +type markedString struct { + Language string `json:"language"` + Value string `json:"value"` + + isRawString bool +} + +type WorkspaceSymbolParams struct { + Query string `json:"query"` + Limit int `json:"limit"` +} + +type SymbolInformation struct { + Name string `json:"name"` + Kind SymbolKind `json:"kind"` + Location Location `json:"location"` + ContainerName string `json:"containerName,omitempty"` +} + +// TypeHierarchyItem represents a node in the type hierarchy tree. +// +// @since 3.17.0 +type TypeHierarchyItem struct { + Name string `json:"name"` + Kind SymbolKind `json:"kind"` + Detail string `json:"detail,omitempty"` + URI DocumentURI `json:"uri"` + Range Range `json:"range"` + SelectionRange Range `json:"selectionRange"` + Data interface{} `json:"data,omitempty"` } +func (cli *LSPClient) WorkspaceSymbols(ctx context.Context, query string) ([]DocumentSymbol, error) { + req := WorkspaceSymbolParams{ + Query: query, + } + var resp []DocumentSymbol + if err := cli.Call(ctx, "workspace/symbol", req, &resp); err != nil { + return nil, err + } + return resp, nil +} func (s *DocumentSymbol) MarshalJSON() ([]byte, error) { if s == nil { return []byte("null"), nil @@ -218,3 +298,53 @@ type Token struct { func (t *Token) String() string { return fmt.Sprintf("%s %s %v %s", t.Text, t.Type, t.Modifiers, t.Location) } + +func (cli *LSPClient) Hover(ctx context.Context, uri DocumentURI, line, character int) (*Hover, error) { + if cli.provider != nil { + // The type assertion is safe because the provider is for the specific language. + return cli.provider.Hover(ctx, cli, uri, line, character) + } + // Default hover implementation (or return an error if not supported) + // Default implementation (or return an error if not supported) + return nil, fmt.Errorf("Hover not supported for this language") +} + +func (cli *LSPClient) Implementation(ctx context.Context, uri DocumentURI, pos Position) ([]Location, error) { + if cli.provider != nil { + return cli.provider.Implementation(ctx, cli, uri, pos) + } + // Default implementation (or return an error if not supported) + return nil, fmt.Errorf("implementation not supported for this language") +} + +func (cli *LSPClient) WorkspaceSearchSymbols(ctx context.Context, query string) ([]SymbolInformation, error) { + if cli.provider != nil { + return cli.provider.WorkspaceSearchSymbols(ctx, cli, query) + } + // Default implementation (or return an error if not supported) + return nil, fmt.Errorf("WorkspaceSearchSymbols not supported for this language") +} + +func (cli *LSPClient) PrepareTypeHierarchy(ctx context.Context, uri DocumentURI, pos Position) ([]TypeHierarchyItem, error) { + if cli.provider != nil { + return cli.provider.PrepareTypeHierarchy(ctx, cli, uri, pos) + } + // Default implementation (or return an error if not supported) + return nil, fmt.Errorf("PrepareTypeHierarchy not supported for this language") +} + +func (cli *LSPClient) TypeHierarchySupertypes(ctx context.Context, item TypeHierarchyItem) ([]TypeHierarchyItem, error) { + if cli.provider != nil { + return cli.provider.TypeHierarchySupertypes(ctx, cli, item) + } + // Default implementation (or return an error if not supported) + return nil, fmt.Errorf("TypeHierarchySupertypes not supported for this language") +} + +func (cli *LSPClient) TypeHierarchySubtypes(ctx context.Context, item TypeHierarchyItem) ([]TypeHierarchyItem, error) { + if cli.provider != nil { + return cli.provider.TypeHierarchySubtypes(ctx, cli, item) + } + // Default implementation (or return an error if not supported) + return nil, fmt.Errorf("TypeHierarchySubtypes not supported for this language") +} diff --git a/lang/lsp/provider.go b/lang/lsp/provider.go new file mode 100644 index 00000000..b739b251 --- /dev/null +++ b/lang/lsp/provider.go @@ -0,0 +1,62 @@ +// Copyright 2025 CloudWeGo Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package lsp + +import ( + "context" + "github.com/cloudwego/abcoder/lang/uniast" +) + +// LanguageServiceProvider defines methods for language-specific LSP features. +// It allows for extending the base LSPClient with language-specific capabilities +// without creating circular dependencies. +type LanguageServiceProvider interface { + // Hover provides hover information for a given position. + // Implementations may have custom logic to parse results from different language servers. + Hover(ctx context.Context, cli *LSPClient, uri DocumentURI, line, character int) (*Hover, error) + + // Implementation finds implementations of a symbol. + Implementation(ctx context.Context, cli *LSPClient, uri DocumentURI, pos Position) ([]Location, error) + + // WorkspaceSymbols searches for symbols in the workspace. + WorkspaceSearchSymbols(ctx context.Context, cli *LSPClient, query string) ([]SymbolInformation, error) + + // PrepareTypeHierarchy prepares a type hierarchy for a given position. + PrepareTypeHierarchy(ctx context.Context, cli *LSPClient, uri DocumentURI, pos Position) ([]TypeHierarchyItem, error) + + // TypeHierarchySupertypes gets the supertypes of a type hierarchy item. + TypeHierarchySupertypes(ctx context.Context, cli *LSPClient, item TypeHierarchyItem) ([]TypeHierarchyItem, error) + + // TypeHierarchySubtypes gets the subtypes of a type hierarchy item. + TypeHierarchySubtypes(ctx context.Context, cli *LSPClient, item TypeHierarchyItem) ([]TypeHierarchyItem, error) +} + +var providers = make(map[uniast.Language]LanguageServiceProvider) + +// RegisterProvider makes a LanguageServiceProvider available for a given language. +// This function should be called from the init() function of a language-specific package. +func RegisterProvider(lang uniast.Language, provider LanguageServiceProvider) { + if _, dup := providers[lang]; dup { + // Or maybe log a warning + return + } + providers[lang] = provider +} + +// GetProvider returns the registered LanguageServiceProvider for a given language. +// It returns nil if no provider is registered for the language. +func GetProvider(lang uniast.Language) LanguageServiceProvider { + return providers[lang] +} diff --git a/lang/lsp/spec.go b/lang/lsp/spec.go index fd51bc6c..83feaada 100644 --- a/lang/lsp/spec.go +++ b/lang/lsp/spec.go @@ -26,7 +26,7 @@ type LanguageSpec interface { // give an absolute file path and returns its module name and package path // external path should alse be supported // FIXEM: some language (like rust) may have sub-mods inside a file, but we still consider it as a unity mod here - NameSpace(path string) (string, string, error) + NameSpace(path string, file *uniast.File) (string, string, error) // tells if a file belang to language AST ShouldSkip(path string) bool @@ -67,4 +67,6 @@ type LanguageSpec interface { // Handle a unloaded internal symbol, like `lazy_static!` in rust GetUnloadedSymbol(from Token, define Location) (string, error) + // some language may allow local symbols inside another symbol + ProtectedSymbolKinds() []SymbolKind } diff --git a/lang/parse.go b/lang/parse.go index 0afa5be7..54d77347 100644 --- a/lang/parse.go +++ b/lang/parse.go @@ -20,6 +20,7 @@ import ( "context" "encoding/json" "fmt" + "github.com/cloudwego/abcoder/lang/register" "os" "os/exec" "path/filepath" @@ -28,6 +29,7 @@ import ( "github.com/cloudwego/abcoder/lang/collect" "github.com/cloudwego/abcoder/lang/cxx" "github.com/cloudwego/abcoder/lang/golang/parser" + "github.com/cloudwego/abcoder/lang/java" "github.com/cloudwego/abcoder/lang/log" "github.com/cloudwego/abcoder/lang/lsp" "github.com/cloudwego/abcoder/lang/python" @@ -45,6 +47,8 @@ type ParseOptions struct { // specify the repo id RepoID string + LspOptions map[string]string + // TS options // tsconfig string TSParseOptions @@ -61,7 +65,7 @@ func Parse(ctx context.Context, uri string, args ParseOptions) ([]byte, error) { if !filepath.IsAbs(uri) { uri, _ = filepath.Abs(uri) } - l, lspPath, err := checkLSP(args.Language, args.LSP) + l, lspPath, err := checkLSP(args.Language, args.LSP, args) if err != nil { return nil, err } @@ -74,11 +78,13 @@ func Parse(ctx context.Context, uri string, args ParseOptions) ([]byte, error) { if lspPath != "" { // Initialize the LSP client log.Info("start initialize LSP server %s...\n", lspPath) + register.RegisterProviders() var err error client, err = lsp.NewLSPClient(uri, openfile, opentime, lsp.ClientOptions{ - Server: lspPath, - Language: l, - Verbose: args.Verbose, + Server: lspPath, + Language: l, + Verbose: args.Verbose, + InitializationOptions: args.LspOptions, }) if err != nil { log.Error("failed to initialize LSP server: %v\n", err) @@ -120,6 +126,8 @@ func checkRepoPath(repoPath string, language uniast.Language) (openfile string, openfile, wait = cxx.CheckRepo(repoPath) case uniast.Python: openfile, wait = python.CheckRepo(repoPath) + case uniast.Java: + openfile, wait = java.CheckRepo(repoPath) default: openfile = "" wait = 0 @@ -129,7 +137,7 @@ func checkRepoPath(repoPath string, language uniast.Language) (openfile string, return } -func checkLSP(language uniast.Language, lspPath string) (l uniast.Language, s string, err error) { +func checkLSP(language uniast.Language, lspPath string, args ParseOptions) (l uniast.Language, s string, err error) { switch language { case uniast.Rust: l, s = rust.GetDefaultLSP() @@ -137,6 +145,8 @@ func checkLSP(language uniast.Language, lspPath string) (l uniast.Language, s st l, s = cxx.GetDefaultLSP() case uniast.Python: l, s = python.GetDefaultLSP() + case uniast.Java: + l, s = java.GetDefaultLSP(args.LspOptions) case uniast.Golang: l = uniast.Golang s = "" diff --git a/lang/python/spec.go b/lang/python/spec.go index 5cd1535e..0489cc3d 100644 --- a/lang/python/spec.go +++ b/lang/python/spec.go @@ -35,6 +35,10 @@ type PythonSpec struct { sysPaths []string } +func (c *PythonSpec) ProtectedSymbolKinds() []lsp.SymbolKind { + return []lsp.SymbolKind{} +} + func NewPythonSpec() *PythonSpec { cmd := exec.Command("python", "-c", "import sys ; print('\\n'.join(sys.path))") output, err := cmd.Output() @@ -85,7 +89,7 @@ func (c *PythonSpec) WorkSpace(root string) (map[string]string, error) { } // returns: modName, pkgPath, error -func (c *PythonSpec) NameSpace(path string) (string, string, error) { +func (c *PythonSpec) NameSpace(path string, file *uniast.File) (string, string, error) { if strings.HasPrefix(path, c.topModulePath) { // internal module modName := c.topModuleName diff --git a/lang/register/provider.go b/lang/register/provider.go new file mode 100644 index 00000000..1f89312f --- /dev/null +++ b/lang/register/provider.go @@ -0,0 +1,26 @@ +// Copyright 2025 CloudWeGo Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package register + +import ( + javaLsp "github.com/cloudwego/abcoder/lang/java/lsp" + "github.com/cloudwego/abcoder/lang/lsp" + "github.com/cloudwego/abcoder/lang/uniast" +) + +func RegisterProviders() { + lsp.RegisterProvider(uniast.Java, &javaLsp.JavaProvider{}) + +} diff --git a/lang/rust/rust_test.go b/lang/rust/rust_test.go index 12548141..916c369b 100644 --- a/lang/rust/rust_test.go +++ b/lang/rust/rust_test.go @@ -68,7 +68,7 @@ func TestRustSpec_NameSpaceInternal(t *testing.T) { } // Namespace for _, ns := range tt.nameSpace { - gotMod, gotPkg, err := c.NameSpace(tt.args.root + ns.relPath) + gotMod, gotPkg, err := c.NameSpace(tt.args.root+ns.relPath, nil) if err != nil { t.Errorf("RustSpec.NameSpace() error = %v", err) return diff --git a/lang/rust/spec.go b/lang/rust/spec.go index 36daae38..7193418a 100644 --- a/lang/rust/spec.go +++ b/lang/rust/spec.go @@ -34,6 +34,10 @@ type RustSpec struct { crates []Module // path => name } +func (c *RustSpec) ProtectedSymbolKinds() []lsp.SymbolKind { + return []lsp.SymbolKind{} +} + type Module struct { Name string Path string @@ -327,7 +331,7 @@ func (c *RustSpec) ShouldSkip(path string) bool { return false } -func (c *RustSpec) NameSpace(path string) (string, string, error) { +func (c *RustSpec) NameSpace(path string, file *uniast.File) (string, string, error) { // external lib if !strings.HasPrefix(path, c.repo) { crate, mod := getCrateAndMod(path) diff --git a/lang/uniast/ast.go b/lang/uniast/ast.go index 158aa2e4..12afedae 100644 --- a/lang/uniast/ast.go +++ b/lang/uniast/ast.go @@ -32,6 +32,7 @@ const ( Cxx Language = "cxx" Python Language = "python" TypeScript Language = "typescript" + Java Language = "java" Unknown Language = "" ) @@ -45,6 +46,8 @@ func (l Language) String() string { return "cxx" case Python: return "python" + case Java: + return "java" default: return string(l) } @@ -67,6 +70,8 @@ func NewLanguage(lang string) (l Language) { return Python case "ts", "typescript", "javascript", "js": return TypeScript + case "java": + return Java default: return Unknown } diff --git a/lang/utils/strings.go b/lang/utils/strings.go index 57c99a27..a999f429 100644 --- a/lang/utils/strings.go +++ b/lang/utils/strings.go @@ -94,3 +94,13 @@ func DedupSlice[T comparable](s []T) []T { } return s[:j] } + +// Contains returns true if an T is present in a iteratee. +func Contains[T comparable](s []T, v T) bool { + for _, vv := range s { + if vv == v { + return true + } + } + return false +} diff --git a/main.go b/main.go index 740c7060..83885554 100644 --- a/main.go +++ b/main.go @@ -61,6 +61,7 @@ Language: cxx for c codes (cpp support is on the way) go for golang codes python for python codes + java for java codes ` func main() { @@ -70,6 +71,7 @@ func main() { flagVerbose := flags.Bool("verbose", false, "Verbose mode.") flagOutput := flags.String("o", "", "Output path.") flagLsp := flags.String("lsp", "", "Specify the language server path.") + javaHome := flags.String("java-home", "", "java home") var opts lang.ParseOptions flags.BoolVar(&opts.LoadExternalSymbol, "load-external-symbol", false, "load external symbols into results") @@ -126,6 +128,12 @@ func main() { opts.LSP = *flagLsp } + lspOptions := make(map[string]string) + if javaHome != nil { + lspOptions["java.home"] = *javaHome + } + opts.LspOptions = lspOptions + out, err := lang.Parse(context.Background(), uri, opts) if err != nil { log.Error("Failed to parse: %v\n", err) diff --git a/testdata/java/0_simple/AdvancedFeatures.java b/testdata/java/0_simple/AdvancedFeatures.java new file mode 100644 index 00000000..1e6b9b5e --- /dev/null +++ b/testdata/java/0_simple/AdvancedFeatures.java @@ -0,0 +1,4 @@ +package simple; + +public class AdvancedFeature { +} \ No newline at end of file diff --git a/testdata/java/0_simple/HelloWorld.java b/testdata/java/0_simple/HelloWorld.java new file mode 100644 index 00000000..e4afc856 --- /dev/null +++ b/testdata/java/0_simple/HelloWorld.java @@ -0,0 +1,29 @@ +package simple; + +public class HelloWorld { + public static void main(String[] args) { + System.out.println("Hello, World!"); + P p = new P(); + p.filed = "123"; + HelloWorld helloWorld = new HelloWorld(); + helloWorld.testFunction(p); + testFunction(p,"asda") + } + + + public String testFunction(P args2){ + return args2.getFiled(); + } + + public String testFunction(P args2,String args3){ + return args2.getFiled(); + } + public class P { + public String filed; + + public String getFiled(){ + return filed; + } + } +} + diff --git a/testdata/java/1_advanced/src/main/java/org/example/Animal.java b/testdata/java/1_advanced/src/main/java/org/example/Animal.java new file mode 100644 index 00000000..02e28a1c --- /dev/null +++ b/testdata/java/1_advanced/src/main/java/org/example/Animal.java @@ -0,0 +1,5 @@ +package org.example; + +public interface Animal { + String makeSound(); +} \ No newline at end of file diff --git a/testdata/java/1_advanced/src/main/java/org/example/Cat.java b/testdata/java/1_advanced/src/main/java/org/example/Cat.java new file mode 100644 index 00000000..a9452ac6 --- /dev/null +++ b/testdata/java/1_advanced/src/main/java/org/example/Cat.java @@ -0,0 +1,8 @@ +package org.example; + +public class Cat implements Animal { + @Override + public String makeSound() { + return "Meow!"; + } +} \ No newline at end of file diff --git a/testdata/java/1_advanced/src/main/java/org/example/Dog.java b/testdata/java/1_advanced/src/main/java/org/example/Dog.java new file mode 100644 index 00000000..a7061794 --- /dev/null +++ b/testdata/java/1_advanced/src/main/java/org/example/Dog.java @@ -0,0 +1,15 @@ +package org.example; + +public class Dog implements Animal { + + public String field; + + public void fetch() { + System.out.println("Fetching the ball!"); + } + + @Override + public String makeSound() { + return "Woof!"; + } +} \ No newline at end of file diff --git a/testdata/java/1_advanced/src/main/java/org/example/TestUtf16.java b/testdata/java/1_advanced/src/main/java/org/example/TestUtf16.java new file mode 100644 index 00000000..1d1645ab --- /dev/null +++ b/testdata/java/1_advanced/src/main/java/org/example/TestUtf16.java @@ -0,0 +1,15 @@ +package org.example; + +public class TestUtf16 { + // 测试包含emoji和中文的情况：😀 中文测试 + public void testWithUnicode() { + String emoji = "😀"; // emoji是4字节UTF-8 + String chinese = "中文"; // 中文字符是3字节UTF-8 + String mixed = "a😀中文b"; // 混合字符串 + } + + // 方法参数测试 + public void methodWithParams(String param1, int param2) { + // 测试方法定义位置 + } +} \ No newline at end of file diff --git a/testdata/java/2_inheritance/src/main/java/org/example/Circle.java b/testdata/java/2_inheritance/src/main/java/org/example/Circle.java new file mode 100644 index 00000000..c35ce233 --- /dev/null +++ b/testdata/java/2_inheritance/src/main/java/org/example/Circle.java @@ -0,0 +1,8 @@ +package org.example; + +public class Circle extends Shape { + @Override + public void draw() { + System.out.println("Drawing a circle."); + } +} \ No newline at end of file diff --git a/testdata/java/2_inheritance/src/main/java/org/example/Rectangle.java b/testdata/java/2_inheritance/src/main/java/org/example/Rectangle.java new file mode 100644 index 00000000..34cd89ba --- /dev/null +++ b/testdata/java/2_inheritance/src/main/java/org/example/Rectangle.java @@ -0,0 +1,8 @@ +package org.example; + +public class Rectangle extends Shape { + @Override + public void draw() { + System.out.println("Drawing a rectangle."); + } +} \ No newline at end of file diff --git a/testdata/java/2_inheritance/src/main/java/org/example/Shape.java b/testdata/java/2_inheritance/src/main/java/org/example/Shape.java new file mode 100644 index 00000000..9efd08b4 --- /dev/null +++ b/testdata/java/2_inheritance/src/main/java/org/example/Shape.java @@ -0,0 +1,8 @@ +package org.example; + +public abstract class Shape { + public abstract void draw(); + public void info() { + System.out.println("This is a shape."); + } +} \ No newline at end of file diff --git a/testdata/java/3_java_pom/my-app-sub/pom.xml b/testdata/java/3_java_pom/my-app-sub/pom.xml new file mode 100644 index 00000000..8b81c6b0 --- /dev/null +++ b/testdata/java/3_java_pom/my-app-sub/pom.xml @@ -0,0 +1,15 @@ + + + + my-app + com.example + 1.0.0 + ../pom.xml + + 4.0.0 + + my-app-sub + + \ No newline at end of file diff --git a/testdata/java/3_java_pom/pom.xml b/testdata/java/3_java_pom/pom.xml new file mode 100644 index 00000000..48646839 --- /dev/null +++ b/testdata/java/3_java_pom/pom.xml @@ -0,0 +1,15 @@ + + + 4.0.0 + + com.example + my-app + 1.0.0 + pom + + + my-app-sub + + \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/README.md b/testdata/java/4_full_maven_repo/README.md new file mode 100644 index 00000000..d4cf4d3a --- /dev/null +++ b/testdata/java/4_full_maven_repo/README.md @@ -0,0 +1,68 @@ +# Java测试仓库 + +这是一个用于测试Java解析器的完整Maven多模块项目。 + +## 项目结构 + +``` +test-repo/ +├── pom.xml # 父项目POM +├── core-module/ # 核心业务模块 +├── service-module/ # 服务层模块 +├── web-module/ # Web层模块 +├── common-module/ # 通用工具模块 +└── README.md +``` + +## 模块依赖关系 + +- **common-module**: 基础工具类，被所有其他模块依赖 +- **core-module**: 核心业务逻辑，依赖common-module +- **service-module**: 服务层，依赖core-module和common-module +- **web-module**: Web层，依赖service-module、core-module和common-module + +## 功能特性 + +1. **实体类**: User实体继承BaseEntity基类 +2. **服务层**: UserService提供用户管理功能 +3. **Web API**: RESTful接口通过UserController暴露 +4. **工具类**: StringUtils提供字符串处理功能 +5. **配置**: Spring配置通过AppConfig统一管理 + +## 使用说明 + +### 构建项目 +```bash +mvn clean install +``` + +### 运行应用 +```bash +cd web-module +mvn spring-boot:run +``` + +### 测试API + +- POST /api/users/register - 注册新用户 +- GET /api/users/{id} - 获取用户信息 +- GET /api/users/active - 获取所有活跃用户 +- PUT /api/users/{id}/status - 更新用户状态 +- DELETE /api/users/{id} - 删除用户 +- POST /api/users/reset-password - 重置密码 + +## 代码引用关系 + +项目展示了以下Java解析器测试场景： + +1. **继承关系**: User extends BaseEntity +2. **接口实现**: InMemoryUserRepository implements UserRepository +3. **泛型使用**: Optional, List +4. **枚举定义**: User.UserStatus +5. **注解使用**: @Service, @RestController, @Configuration +6. **依赖注入**: 构造函数注入和@Bean配置 +7. **静态方法**: StringUtils工具类的使用 +8. **包导入**: 跨模块的import语句 +9. **异常处理**: try-catch块和自定义异常 + +这个测试仓库包含了丰富的Java语法特性，适合测试Java解析器的各种场景。 \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/common-module/pom.xml b/testdata/java/4_full_maven_repo/common-module/pom.xml new file mode 100644 index 00000000..30a398d1 --- /dev/null +++ b/testdata/java/4_full_maven_repo/common-module/pom.xml @@ -0,0 +1,24 @@ + + + 4.0.0 + + + com.example.test + test-repo + 1.0.0-SNAPSHOT + + + common-module + jar + + + + junit + junit + test + + + \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/common-module/src/main/java/com/example/common/model/BaseEntity.java b/testdata/java/4_full_maven_repo/common-module/src/main/java/com/example/common/model/BaseEntity.java new file mode 100644 index 00000000..7c740b2e --- /dev/null +++ b/testdata/java/4_full_maven_repo/common-module/src/main/java/com/example/common/model/BaseEntity.java @@ -0,0 +1,52 @@ +package com.example.common.model; + +import java.time.LocalDateTime; + +public abstract class BaseEntity { + + private Long id; + private LocalDateTime createdAt; + private LocalDateTime updatedAt; + private String createdBy; + private String updatedBy; + + public Long getId() { + return id; + } + + public void setId(Long id) { + this.id = id; + } + + public LocalDateTime getCreatedAt() { + return createdAt; + } + + public void setCreatedAt(LocalDateTime createdAt) { + this.createdAt = createdAt; + } + + public LocalDateTime getUpdatedAt() { + return updatedAt; + } + + public void setUpdatedAt(LocalDateTime updatedAt) { + this.updatedAt = updatedAt; + } + + public String getCreatedBy() { + return createdBy; + } + + public void setCreatedBy(String createdBy) { + this.createdBy = createdBy; + } + + public String getUpdatedBy() { + return updatedBy; + } + + public void setUpdatedBy(String updatedBy) { + this.updatedBy = updatedBy; + } +} \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/common-module/src/main/java/com/example/common/utils/StringUtils.java b/testdata/java/4_full_maven_repo/common-module/src/main/java/com/example/common/utils/StringUtils.java new file mode 100644 index 00000000..f253e82e --- /dev/null +++ b/testdata/java/4_full_maven_repo/common-module/src/main/java/com/example/common/utils/StringUtils.java @@ -0,0 +1,31 @@ +package com.example.common.utils; + +import java.util.regex.Pattern; + +public class StringUtils { + + private static final Pattern EMAIL_PATTERN = Pattern.compile("^[A-Za-z0-9+_.-]+@(.+)$"); + + public static boolean isEmpty(String str) { + return str == null || str.trim().isEmpty(); + } + + public static boolean isNotEmpty(String str) { + return !isEmpty(str); + } + + public static String trim(String str) { + return str == null ? "" : str.trim(); + } + + public static boolean isValidEmail(String email) { + return email != null && EMAIL_PATTERN.matcher(email).matches(); + } + + public static String capitalize(String str) { + if (isEmpty(str)) { + return str; + } + return str.substring(0, 1).toUpperCase() + str.substring(1).toLowerCase(); + } +} \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/core-module/pom.xml b/testdata/java/4_full_maven_repo/core-module/pom.xml new file mode 100644 index 00000000..183ad6f8 --- /dev/null +++ b/testdata/java/4_full_maven_repo/core-module/pom.xml @@ -0,0 +1,33 @@ + + + 4.0.0 + + + com.example.test + test-repo + 1.0.0-SNAPSHOT + + + core-module + jar + + + + com.example.test + common-module + ${project.version} + + + org.springframework + spring-context + + + junit + junit + test + + + \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/core-module/src/main/java/com/example/core/model/User.java b/testdata/java/4_full_maven_repo/core-module/src/main/java/com/example/core/model/User.java new file mode 100644 index 00000000..351e8da7 --- /dev/null +++ b/testdata/java/4_full_maven_repo/core-module/src/main/java/com/example/core/model/User.java @@ -0,0 +1,68 @@ +package com.example.core.model; + +import com.example.common.model.BaseEntity; +import com.example.common.utils.StringUtils; + +public class User extends BaseEntity { + + private String username; + private String email; + private String password; + private UserStatus status; + + public enum UserStatus { + ACTIVE, INACTIVE, SUSPENDED + } + + public String getUsername() { + return username; + } + + public void setUsername(String username) { + if (StringUtils.isNotEmpty(username)) { + this.username = username.trim(); + } + } + + public String getEmail() { + return email; + } + + public void setEmail(String email) { + if (StringUtils.isValidEmail(email)) { + this.email = email.toLowerCase(); + } + } + + public String getPassword() { + return password; + } + + public void setPassword(String password) { + if (StringUtils.isNotEmpty(password)) { + this.password = password; + } + } + + public UserStatus getStatus() { + return status; + } + + public void setStatus(UserStatus status) { + this.status = status; + } + + public boolean isActive() { + return status == UserStatus.ACTIVE; + } + + @Override + public String toString() { + return "User{" + + "id=" + getId() + + ", username='" + username + '\'' + + ", email='" + email + '\'' + + ", status=" + status + + '}'; + } +} \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/core-module/src/main/java/com/example/core/repository/InMemoryUserRepository.java b/testdata/java/4_full_maven_repo/core-module/src/main/java/com/example/core/repository/InMemoryUserRepository.java new file mode 100644 index 00000000..f854851b --- /dev/null +++ b/testdata/java/4_full_maven_repo/core-module/src/main/java/com/example/core/repository/InMemoryUserRepository.java @@ -0,0 +1,73 @@ +package com.example.core.repository; + +import com.example.core.model.User; +import java.util.*; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.atomic.AtomicLong; +import java.util.stream.Collectors; + +public class InMemoryUserRepository implements UserRepository { + + private final Map users = new ConcurrentHashMap<>(); + private final Map usersByEmail = new ConcurrentHashMap<>(); + private final AtomicLong idGenerator = new AtomicLong(1); + + @Override + public User save(User user) { + if (user.getId() == null) { + user.setId(idGenerator.getAndIncrement()); + } + + users.put(user.getId(), user); + if (user.getEmail() != null) { + usersByEmail.put(user.getEmail().toLowerCase(), user); + } + + return user; + } + + @Override + public Optional findById(Long id) { + return Optional.ofNullable(users.get(id)); + } + + @Override + public Optional findByEmail(String email) { + return Optional.ofNullable(usersByEmail.get(email.toLowerCase())); + } + + @Override + public List findByStatus(User.UserStatus status) { + return users.values().stream() + .filter(user -> user.getStatus() == status) + .collect(Collectors.toList()); + } + + @Override + public List findAll() { + return new ArrayList<>(users.values()); + } + + @Override + public void deleteById(Long id) { + User user = users.remove(id); + if (user != null && user.getEmail() != null) { + usersByEmail.remove(user.getEmail().toLowerCase()); + } + } + + @Override + public long count() { + return users.size(); + } + + @Override + public boolean existsById(Long id) { + return users.containsKey(id); + } + + @Override + public boolean existsByEmail(String email) { + return usersByEmail.containsKey(email.toLowerCase()); + } +} \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/core-module/src/main/java/com/example/core/repository/UserRepository.java b/testdata/java/4_full_maven_repo/core-module/src/main/java/com/example/core/repository/UserRepository.java new file mode 100644 index 00000000..d8f3f87c --- /dev/null +++ b/testdata/java/4_full_maven_repo/core-module/src/main/java/com/example/core/repository/UserRepository.java @@ -0,0 +1,26 @@ +package com.example.core.repository; + +import com.example.core.model.User; +import java.util.List; +import java.util.Optional; + +public interface UserRepository { + + User save(User user); + + Optional findById(Long id); + + Optional findByEmail(String email); + + List findByStatus(User.UserStatus status); + + List findAll(); + + void deleteById(Long id); + + long count(); + + boolean existsById(Long id); + + boolean existsByEmail(String email); +} \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/core-module/src/main/java/com/example/core/service/UserService.java b/testdata/java/4_full_maven_repo/core-module/src/main/java/com/example/core/service/UserService.java new file mode 100644 index 00000000..f406559f --- /dev/null +++ b/testdata/java/4_full_maven_repo/core-module/src/main/java/com/example/core/service/UserService.java @@ -0,0 +1,69 @@ +package com.example.core.service; + +import com.example.core.model.User; +import com.example.core.repository.UserRepository; +import com.example.common.utils.StringUtils; +import org.springframework.stereotype.Service; +import java.util.List; +import java.util.Optional; + +@Service +public class UserService { + + private final UserRepository userRepository; + + public UserService(UserRepository userRepository) { + this.userRepository = userRepository; + } + + public User createUser(String username, String email, String password) { + if (StringUtils.isEmpty(username)) { + throw new IllegalArgumentException("Username cannot be empty"); + } + + if (!StringUtils.isValidEmail(email)) { + throw new IllegalArgumentException("Invalid email format"); + } + + User user = new User(); + user.setUsername(username); + user.setEmail(email); + user.setPassword(password); + user.setStatus(User.UserStatus.ACTIVE); + + return userRepository.save(user); + } + + public Optional findUserById(Long id) { + return userRepository.findById(id); + } + + public List findAllActiveUsers() { + return userRepository.findByStatus(User.UserStatus.ACTIVE); + } + + public User updateUserStatus(Long userId, User.UserStatus newStatus) { + User user = userRepository.findById(userId) + .orElseThrow(() -> new IllegalArgumentException("User not found: " + userId)); + + user.setStatus(newStatus); + return userRepository.save(user); + } + + public boolean deleteUser(Long userId) { + return userRepository.findById(userId) + .map(user -> { + user.setStatus(User.UserStatus.INACTIVE); + userRepository.save(user); + return true; + }) + .orElse(false); + } + + public boolean validateUserCredentials(String email, String password) { + return userRepository.findByEmail(email) + .filter(user -> user.isActive()) + .filter(user -> user.getPassword().equals(password)) + .isPresent(); + } +} \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/pom.xml b/testdata/java/4_full_maven_repo/pom.xml new file mode 100644 index 00000000..c819deb8 --- /dev/null +++ b/testdata/java/4_full_maven_repo/pom.xml @@ -0,0 +1,61 @@ + + + 4.0.0 + + com.example.test + test-repo + 1.0.0-SNAPSHOT + pom + + + 11 + 11 + UTF-8 + 5.3.21 + + + + core-module + service-module + web-module + common-module + + + + + + org.springframework + spring-context + ${spring.version} + + + org.springframework + spring-web + ${spring.version} + + + junit + junit + 4.13.2 + test + + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + 3.8.1 + + 11 + 11 + + + + + \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/service-module/pom.xml b/testdata/java/4_full_maven_repo/service-module/pom.xml new file mode 100644 index 00000000..9d257787 --- /dev/null +++ b/testdata/java/4_full_maven_repo/service-module/pom.xml @@ -0,0 +1,38 @@ + + + 4.0.0 + + + com.example.test + test-repo + 1.0.0-SNAPSHOT + + + service-module + jar + + + + com.example.test + core-module + ${project.version} + + + com.example.test + common-module + ${project.version} + + + org.springframework + spring-context + + + junit + junit + test + + + \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/service-module/src/main/java/com/example/service/EmailService.java b/testdata/java/4_full_maven_repo/service-module/src/main/java/com/example/service/EmailService.java new file mode 100644 index 00000000..4a9a038b --- /dev/null +++ b/testdata/java/4_full_maven_repo/service-module/src/main/java/com/example/service/EmailService.java @@ -0,0 +1,48 @@ +package com.example.service; + +import com.example.common.utils.StringUtils; +import com.example.core.model.User; +import org.springframework.stereotype.Service; + +@Service +public class EmailService { + + public void sendWelcomeEmail(User user) { + if (user == null || !StringUtils.isValidEmail(user.getEmail())) { + throw new IllegalArgumentException("Invalid user or email"); + } + + String subject = "Welcome to our platform, " + StringUtils.capitalize(user.getUsername()); + String body = String.format( + "Dear %s,\n\nWelcome to our platform! Your account has been successfully created.\n\nBest regards,\nThe Team", + StringUtils.capitalize(user.getUsername()) + ); + + // 模拟发送邮件 + System.out.println("Sending email to: " + user.getEmail()); + System.out.println("Subject: " + subject); + System.out.println("Body: " + body); + } + + public void sendPasswordResetEmail(User user, String resetToken) { + if (user == null || !StringUtils.isValidEmail(user.getEmail())) { + throw new IllegalArgumentException("Invalid user or email"); + } + + if (StringUtils.isEmpty(resetToken)) { + throw new IllegalArgumentException("Reset token cannot be empty"); + } + + String subject = "Password Reset Request"; + String body = String.format( + "Dear %s,\n\nYou have requested a password reset. Please use the following token: %s\n\nThis token will expire in 1 hour.\n\nBest regards,\nThe Team", + StringUtils.capitalize(user.getUsername()), + resetToken + ); + + // 模拟发送邮件 + System.out.println("Sending password reset email to: " + user.getEmail()); + System.out.println("Subject: " + subject); + System.out.println("Body: " + body); + } +} \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/service-module/src/main/java/com/example/service/UserRegistrationService.java b/testdata/java/4_full_maven_repo/service-module/src/main/java/com/example/service/UserRegistrationService.java new file mode 100644 index 00000000..e0a201b4 --- /dev/null +++ b/testdata/java/4_full_maven_repo/service-module/src/main/java/com/example/service/UserRegistrationService.java @@ -0,0 +1,64 @@ +package com.example.service; + +import com.example.core.model.User; +import com.example.core.service.UserService; +import com.example.common.utils.StringUtils; +import org.springframework.stereotype.Service; +import org.springframework.transaction.annotation.Transactional; + +@Service +public class UserRegistrationService { + + private final UserService userService; + private final EmailService emailService; + + public UserRegistrationService(UserService userService, EmailService emailService) { + this.userService = userService; + this.emailService = emailService; + } + + @Transactional + public User registerUser(String username, String email, String password) { + // 验证输入参数 + if (StringUtils.isEmpty(username)) { + throw new IllegalArgumentException("Username is required"); + } + + if (!StringUtils.isValidEmail(email)) { + throw new IllegalArgumentException("Invalid email format"); + } + + if (StringUtils.isEmpty(password)) { + throw new IllegalArgumentException("Password is required"); + } + + // 创建用户 + User user = userService.createUser(username, email, password); + + // 发送欢迎邮件 + emailService.sendWelcomeEmail(user); + + return user; + } + + @Transactional + public boolean initiatePasswordReset(String email) { + if (!StringUtils.isValidEmail(email)) { + throw new IllegalArgumentException("Invalid email format"); + } + + return userService.findAllActiveUsers().stream() + .filter(user -> email.equalsIgnoreCase(user.getEmail())) + .findFirst() + .map(user -> { + String resetToken = generateResetToken(); + emailService.sendPasswordResetEmail(user, resetToken); + return true; + }) + .orElse(false); + } + + private String generateResetToken() { + return "RESET-" + System.currentTimeMillis() + "-" + (int)(Math.random() * 10000); + } +} \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/web-module/pom.xml b/testdata/java/4_full_maven_repo/web-module/pom.xml new file mode 100644 index 00000000..607ae8e6 --- /dev/null +++ b/testdata/java/4_full_maven_repo/web-module/pom.xml @@ -0,0 +1,42 @@ + + + 4.0.0 + + + com.example.test + test-repo + 1.0.0-SNAPSHOT + + + web-module + jar + + + + com.example.test + service-module + ${project.version} + + + com.example.test + core-module + ${project.version} + + + org.springframework + spring-web + + + org.springframework + spring-context + + + junit + junit + test + + + \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/web-module/src/main/java/com/example/web/Application.java b/testdata/java/4_full_maven_repo/web-module/src/main/java/com/example/web/Application.java new file mode 100644 index 00000000..737243a5 --- /dev/null +++ b/testdata/java/4_full_maven_repo/web-module/src/main/java/com/example/web/Application.java @@ -0,0 +1,27 @@ +package com.example.web; + +import com.example.web.config.AppConfig; +import org.springframework.context.ApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; + +public class Application { + + public static void main(String[] args) { + ApplicationContext context = new AnnotationConfigApplicationContext(AppConfig.class); + + System.out.println("Test Repository Application Started!"); + System.out.println("Available beans:"); + + String[] beanNames = context.getBeanDefinitionNames(); + for (String beanName : beanNames) { + System.out.println("- " + beanName); + } + + // 保持应用运行 + try { + Thread.sleep(1000); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + } + } +} \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/web-module/src/main/java/com/example/web/config/AppConfig.java b/testdata/java/4_full_maven_repo/web-module/src/main/java/com/example/web/config/AppConfig.java new file mode 100644 index 00000000..16272cd3 --- /dev/null +++ b/testdata/java/4_full_maven_repo/web-module/src/main/java/com/example/web/config/AppConfig.java @@ -0,0 +1,41 @@ +package com.example.web.config; + +import com.example.core.repository.InMemoryUserRepository; +import com.example.core.repository.UserRepository; +import com.example.core.service.UserService; +import com.example.service.EmailService; +import com.example.service.UserRegistrationService; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.ComponentScan; +import org.springframework.context.annotation.Configuration; + +@Configuration +@ComponentScan(basePackages = { + "com.example.core", + "com.example.service", + "com.example.web" +}) +public class AppConfig { + + @Bean + public UserRepository userRepository() { + return new InMemoryUserRepository(); + } + + @Bean + public UserService userService(UserRepository userRepository) { + return new UserService(userRepository); + } + + @Bean + public EmailService emailService() { + return new EmailService(); + } + + @Bean + public UserRegistrationService userRegistrationService( + UserService userService, + EmailService emailService) { + return new UserRegistrationService(userService, emailService); + } +} \ No newline at end of file diff --git a/testdata/java/4_full_maven_repo/web-module/src/main/java/com/example/web/controller/UserController.java b/testdata/java/4_full_maven_repo/web-module/src/main/java/com/example/web/controller/UserController.java new file mode 100644 index 00000000..856cc8d0 --- /dev/null +++ b/testdata/java/4_full_maven_repo/web-module/src/main/java/com/example/web/controller/UserController.java @@ -0,0 +1,98 @@ +package com.example.web.controller; + +import com.example.core.model.User; +import com.example.service.UserRegistrationService; +import com.example.core.service.UserService; +import com.example.common.utils.StringUtils; +import org.springframework.http.ResponseEntity; +import org.springframework.web.bind.annotation.*; +import java.util.List; +import java.util.Optional; + +@RestController +@RequestMapping("/api/users") +public class UserController { + + private final UserService userService; + private final UserRegistrationService registrationService; + + public UserController(UserService userService, UserRegistrationService registrationService) { + this.userService = userService; + this.registrationService = registrationService; + } + + @PostMapping("/register") + public ResponseEntity registerUser(@RequestBody UserRegistrationRequest request) { + try { + User user = registrationService.registerUser( + request.getUsername(), + request.getEmail(), + request.getPassword() + ); + return ResponseEntity.ok(user); + } catch (IllegalArgumentException e) { + return ResponseEntity.badRequest().build(); + } + } + + @GetMapping("/{id}") + public ResponseEntity getUserById(@PathVariable Long id) { + Optional user = userService.findUserById(id); + return user.map(ResponseEntity::ok) + .orElse(ResponseEntity.notFound().build()); + } + + @GetMapping("/active") + public ResponseEntity> getAllActiveUsers() { + List users = userService.findAllActiveUsers(); + return ResponseEntity.ok(users); + } + + @PutMapping("/{id}/status") + public ResponseEntity updateUserStatus( + @PathVariable Long id, + @RequestParam User.UserStatus status) { + try { + User user = userService.updateUserStatus(id, status); + return ResponseEntity.ok(user); + } catch (IllegalArgumentException e) { + return ResponseEntity.notFound().build(); + } + } + + @DeleteMapping("/{id}") + public ResponseEntity deleteUser(@PathVariable Long id) { + boolean deleted = userService.deleteUser(id); + return deleted ? ResponseEntity.noContent().build() + : ResponseEntity.notFound().build(); + } + + @PostMapping("/reset-password") + public ResponseEntity resetPassword(@RequestBody PasswordResetRequest request) { + boolean initiated = registrationService.initiatePasswordReset(request.getEmail()); + return initiated ? ResponseEntity.ok().build() + : ResponseEntity.notFound().build(); + } + + public static class UserRegistrationRequest { + private String username; + private String email; + private String password; + + public String getUsername() { return username; } + public void setUsername(String username) { this.username = username; } + + public String getEmail() { return email; } + public void setEmail(String email) { this.email = email; } + + public String getPassword() { return password; } + public void setPassword(String password) { this.password = password; } + } + + public static class PasswordResetRequest { + private String email; + + public String getEmail() { return email; } + public void setEmail(String email) { this.email = email; } + } +} \ No newline at end of file