-
Notifications
You must be signed in to change notification settings - Fork 10.7k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[clang-repl] support code completion at a REPL.
This patch enabled code completion for ClangREPL. The feature was built upon three existing Clang components: a list completer for LineEditor, a CompletionConsumer from SemaCodeCompletion, and the ASTUnit::codeComplete method. The first component serves as the main entry point of handling interactive inputs. Because a completion point for a compiler instance has to be unchanged once it is set, an incremental compiler instance is created for each code completion. Such a compiler instance carries over AST context source from the main interpreter compiler in order to obtain declarations or bindings from previous input in the same REPL session. The most important API codeComplete in Interpreter/CodeCompletion is a thin wrapper that calls with ASTUnit::codeComplete with necessary arguments, such as a code completion point and a ReplCompletionConsumer, which communicates completion results from SemaCodeCompletion back to the list completer for the REPL. In addition, PCC_TopLevelOrExpression and CCC_TopLevelOrExpression` top levels were added so that SemaCodeCompletion can treat top level statements like expression statements at the REPL. For example, clang-repl> int foo = 42; clang-repl> f<tab> From a parser's persective, the cursor is at a top level. If we used code completion without any changes, PCC_Namespace would be supplied to Sema::CodeCompleteOrdinaryName, and thus the completion results would not include foo. Currently, the way we use PCC_TopLevelOrExpression and CCC_TopLevelOrExpression is no different from the way we use PCC_Statement and CCC_Statement respectively. Differential revision: https://reviews.llvm.org/D154382
- Loading branch information
1 parent
d00f598
commit eb0e6c3
Showing
20 changed files
with
495 additions
and
28 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| //===----- CodeCompletion.h - Code Completion for ClangRepl ---===// | ||
| // | ||
| // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. | ||
| // See https://llvm.org/LICENSE.txt for license information. | ||
| // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception | ||
| // | ||
| //===----------------------------------------------------------------------===// | ||
| // | ||
| // This file defines the classes which performs code completion at the REPL. | ||
| // | ||
| //===----------------------------------------------------------------------===// | ||
|
|
||
| #ifndef LLVM_CLANG_INTERPRETER_CODE_COMPLETION_H | ||
| #define LLVM_CLANG_INTERPRETER_CODE_COMPLETION_H | ||
| #include <string> | ||
| #include <vector> | ||
|
|
||
| namespace llvm { | ||
| class StringRef; | ||
| } // namespace llvm | ||
|
|
||
| namespace clang { | ||
| class CodeCompletionResult; | ||
| class CompilerInstance; | ||
|
|
||
| void codeComplete(CompilerInstance *InterpCI, llvm::StringRef Content, | ||
| unsigned Line, unsigned Col, const CompilerInstance *ParentCI, | ||
| std::vector<CodeCompletionResult> &CCResults); | ||
|
|
||
| std::vector<std::string> | ||
| convertToCodeCompleteStrings(const std::vector<CodeCompletionResult> &Results); | ||
| } // namespace clang | ||
| #endif |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,229 @@ | ||
| //===------ CodeCompletion.cpp - Code Completion for ClangRepl -------===// | ||
| // | ||
| // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. | ||
| // See https://llvm.org/LICENSE.txt for license information. | ||
| // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception | ||
| // | ||
| //===----------------------------------------------------------------------===// | ||
| // | ||
| // This file implements the classes which performs code completion at the REPL. | ||
| // | ||
| //===----------------------------------------------------------------------===// | ||
|
|
||
| #include "clang/Interpreter/CodeCompletion.h" | ||
| #include "clang/AST/ASTImporter.h" | ||
| #include "clang/AST/DeclarationName.h" | ||
| #include "clang/AST/ExternalASTSource.h" | ||
| #include "clang/Basic/IdentifierTable.h" | ||
| #include "clang/Frontend/ASTUnit.h" | ||
| #include "clang/Frontend/CompilerInstance.h" | ||
| #include "clang/Frontend/FrontendActions.h" | ||
| #include "clang/Interpreter/Interpreter.h" | ||
| #include "clang/Lex/PreprocessorOptions.h" | ||
| #include "clang/Sema/CodeCompleteConsumer.h" | ||
| #include "clang/Sema/CodeCompleteOptions.h" | ||
| #include "clang/Sema/Sema.h" | ||
|
|
||
| namespace clang { | ||
|
|
||
| const std::string CodeCompletionFileName = "input_line_[Completion]"; | ||
|
|
||
| clang::CodeCompleteOptions getClangCompleteOpts() { | ||
| clang::CodeCompleteOptions Opts; | ||
| Opts.IncludeCodePatterns = true; | ||
| Opts.IncludeMacros = true; | ||
| Opts.IncludeGlobals = true; | ||
| Opts.IncludeBriefComments = true; | ||
| return Opts; | ||
| } | ||
|
|
||
| class ReplCompletionConsumer : public CodeCompleteConsumer { | ||
| public: | ||
| ReplCompletionConsumer(std::vector<CodeCompletionResult> &Results) | ||
| : CodeCompleteConsumer(getClangCompleteOpts()), | ||
| CCAllocator(std::make_shared<GlobalCodeCompletionAllocator>()), | ||
| CCTUInfo(CCAllocator), Results(Results){}; | ||
|
|
||
| void ProcessCodeCompleteResults(class Sema &S, CodeCompletionContext Context, | ||
| CodeCompletionResult *InResults, | ||
| unsigned NumResults) final; | ||
|
|
||
| CodeCompletionAllocator &getAllocator() override { return *CCAllocator; } | ||
|
|
||
| CodeCompletionTUInfo &getCodeCompletionTUInfo() override { return CCTUInfo; } | ||
|
|
||
| private: | ||
| std::shared_ptr<GlobalCodeCompletionAllocator> CCAllocator; | ||
| CodeCompletionTUInfo CCTUInfo; | ||
| std::vector<CodeCompletionResult> &Results; | ||
| }; | ||
|
|
||
| void ReplCompletionConsumer::ProcessCodeCompleteResults( | ||
| class Sema &S, CodeCompletionContext Context, | ||
| CodeCompletionResult *InResults, unsigned NumResults) { | ||
| for (unsigned I = 0; I < NumResults; ++I) { | ||
| auto &Result = InResults[I]; | ||
| switch (Result.Kind) { | ||
| case CodeCompletionResult::RK_Declaration: | ||
| if (Result.Declaration->getIdentifier()) { | ||
| Results.push_back(Result); | ||
| } | ||
| break; | ||
| case CodeCompletionResult::RK_Keyword: | ||
| Results.push_back(Result); | ||
| break; | ||
| default: | ||
| break; | ||
| } | ||
| } | ||
| } | ||
|
|
||
| std::vector<std::string> convertToCodeCompleteStrings( | ||
| const std::vector<clang::CodeCompletionResult> &Results) { | ||
| std::vector<std::string> CompletionStrings; | ||
| for (auto Res : Results) { | ||
| switch (Res.Kind) { | ||
| case clang::CodeCompletionResult::RK_Declaration: | ||
| if (auto *ID = Res.Declaration->getIdentifier()) { | ||
| CompletionStrings.push_back(ID->getName().str()); | ||
| } | ||
| break; | ||
| case clang::CodeCompletionResult::RK_Keyword: | ||
| CompletionStrings.push_back(Res.Keyword); | ||
| break; | ||
| default: | ||
| break; | ||
| } | ||
| } | ||
| return CompletionStrings; | ||
| } | ||
|
|
||
| class IncrementalSyntaxOnlyAction : public SyntaxOnlyAction { | ||
| const CompilerInstance *ParentCI; | ||
|
|
||
| public: | ||
| IncrementalSyntaxOnlyAction(const CompilerInstance *ParentCI) | ||
| : ParentCI(ParentCI) {} | ||
|
|
||
| protected: | ||
| void ExecuteAction() override; | ||
| }; | ||
|
|
||
| class ExternalSource : public clang::ExternalASTSource { | ||
| TranslationUnitDecl *ChildTUDeclCtxt; | ||
| ASTContext &ParentASTCtxt; | ||
| TranslationUnitDecl *ParentTUDeclCtxt; | ||
|
|
||
| std::unique_ptr<ASTImporter> Importer; | ||
|
|
||
| public: | ||
| ExternalSource(ASTContext &ChildASTCtxt, FileManager &ChildFM, | ||
| ASTContext &ParentASTCtxt, FileManager &ParentFM); | ||
| bool FindExternalVisibleDeclsByName(const DeclContext *DC, | ||
| DeclarationName Name) override; | ||
| void | ||
| completeVisibleDeclsMap(const clang::DeclContext *childDeclContext) override; | ||
| }; | ||
|
|
||
| // This method is intended to set up `ExternalASTSource` to the running | ||
| // compiler instance before the super `ExecuteAction` triggers parsing | ||
| void IncrementalSyntaxOnlyAction::ExecuteAction() { | ||
| CompilerInstance &CI = getCompilerInstance(); | ||
| ExternalSource *myExternalSource = | ||
| new ExternalSource(CI.getASTContext(), CI.getFileManager(), | ||
| ParentCI->getASTContext(), ParentCI->getFileManager()); | ||
| llvm::IntrusiveRefCntPtr<clang::ExternalASTSource> astContextExternalSource( | ||
| myExternalSource); | ||
| CI.getASTContext().setExternalSource(astContextExternalSource); | ||
| CI.getASTContext().getTranslationUnitDecl()->setHasExternalVisibleStorage( | ||
| true); | ||
|
|
||
| SyntaxOnlyAction::ExecuteAction(); | ||
| } | ||
|
|
||
| ExternalSource::ExternalSource(ASTContext &ChildASTCtxt, FileManager &ChildFM, | ||
| ASTContext &ParentASTCtxt, FileManager &ParentFM) | ||
| : ChildTUDeclCtxt(ChildASTCtxt.getTranslationUnitDecl()), | ||
| ParentASTCtxt(ParentASTCtxt), | ||
| ParentTUDeclCtxt(ParentASTCtxt.getTranslationUnitDecl()) { | ||
| ASTImporter *importer = | ||
| new ASTImporter(ChildASTCtxt, ChildFM, ParentASTCtxt, ParentFM, | ||
| /*MinimalImport : ON*/ true); | ||
| Importer.reset(importer); | ||
| } | ||
|
|
||
| bool ExternalSource::FindExternalVisibleDeclsByName(const DeclContext *DC, | ||
| DeclarationName Name) { | ||
| IdentifierTable &ParentIdTable = ParentASTCtxt.Idents; | ||
|
|
||
| auto ParentDeclName = | ||
| DeclarationName(&(ParentIdTable.get(Name.getAsString()))); | ||
|
|
||
| DeclContext::lookup_result lookup_result = | ||
| ParentTUDeclCtxt->lookup(ParentDeclName); | ||
|
|
||
| if (!lookup_result.empty()) { | ||
| return true; | ||
| } | ||
| return false; | ||
| } | ||
|
|
||
| void ExternalSource::completeVisibleDeclsMap( | ||
| const DeclContext *ChildDeclContext) { | ||
| assert(ChildDeclContext && ChildDeclContext == ChildTUDeclCtxt && | ||
| "No child decl context!"); | ||
|
|
||
| if (!ChildDeclContext->hasExternalVisibleStorage()) | ||
| return; | ||
|
|
||
| for (auto *DeclCtxt = ParentTUDeclCtxt; DeclCtxt != nullptr; | ||
| DeclCtxt = DeclCtxt->getPreviousDecl()) { | ||
| for (auto &IDeclContext : DeclCtxt->decls()) { | ||
| if (NamedDecl *Decl = llvm::dyn_cast<NamedDecl>(IDeclContext)) { | ||
| if (auto DeclOrErr = Importer->Import(Decl)) { | ||
| if (NamedDecl *importedNamedDecl = | ||
| llvm::dyn_cast<NamedDecl>(*DeclOrErr)) { | ||
| SetExternalVisibleDeclsForName(ChildDeclContext, | ||
| importedNamedDecl->getDeclName(), | ||
| importedNamedDecl); | ||
| } | ||
|
|
||
| } else { | ||
| llvm::consumeError(DeclOrErr.takeError()); | ||
| } | ||
| } | ||
| } | ||
| ChildDeclContext->setHasExternalLexicalStorage(false); | ||
| } | ||
| } | ||
|
|
||
| void codeComplete(CompilerInstance *InterpCI, llvm::StringRef Content, | ||
| unsigned Line, unsigned Col, const CompilerInstance *ParentCI, | ||
| std::vector<CodeCompletionResult> &CCResults) { | ||
| std::unique_ptr<llvm::MemoryBuffer> MB = | ||
| llvm::MemoryBuffer::getMemBufferCopy(Content, CodeCompletionFileName); | ||
| llvm::SmallVector<ASTUnit::RemappedFile, 4> RemappedFiles; | ||
|
|
||
| RemappedFiles.push_back(std::make_pair(CodeCompletionFileName, MB.release())); | ||
|
|
||
| auto DiagOpts = DiagnosticOptions(); | ||
| auto consumer = ReplCompletionConsumer(CCResults); | ||
|
|
||
| auto diag = InterpCI->getDiagnosticsPtr(); | ||
| ASTUnit *AU = ASTUnit::LoadFromCompilerInvocationAction( | ||
| InterpCI->getInvocationPtr(), std::make_shared<PCHContainerOperations>(), | ||
| diag); | ||
| llvm::SmallVector<clang::StoredDiagnostic, 8> sd = {}; | ||
| llvm::SmallVector<const llvm::MemoryBuffer *, 1> tb = {}; | ||
| InterpCI->getFrontendOpts().Inputs[0] = FrontendInputFile( | ||
| CodeCompletionFileName, Language::CXX, InputKind::Source); | ||
| auto Act = std::unique_ptr<IncrementalSyntaxOnlyAction>( | ||
| new IncrementalSyntaxOnlyAction(ParentCI)); | ||
| AU->CodeComplete(CodeCompletionFileName, 1, Col, RemappedFiles, false, false, | ||
| false, consumer, | ||
| std::make_shared<clang::PCHContainerOperations>(), *diag, | ||
| InterpCI->getLangOpts(), InterpCI->getSourceManager(), | ||
| InterpCI->getFileManager(), sd, tb, std::move(Act)); | ||
| } | ||
|
|
||
| } // namespace clang |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.