New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
✨ [Sema, Lex, Parse] Preprocessor embed in C and C++ (and Obj-C and Obj-C++ by-proxy) #68620
Open
ThePhD
wants to merge
136
commits into
llvm:main
Choose a base branch
from
ThePhD:thephd/embed-speed
base: main
Could not load branches
Branch not found: {{ refName }}
Could not load tags
Nothing to show
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+3,322
−108
Open
Changes from 124 commits
Commits
Show all changes
136 commits
Select commit
Hold shift + click to select a range
7050c93
✨ [Sema, Driver, Lex, Frontend] Implement naive #embed for C23 and C+…
ThePhD 6a7a4c9
✨ Speedy #embed implementation
ThePhD 324ff1d
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman 77aad07
Update based on API changes in community
AaronBallman da636a2
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman 1cca725
We don't yet expose a libclang cursor for embed expressions
AaronBallman cd6142d
Update preprocessor tests for new builtin macros
AaronBallman 495f1d4
Fix logical think-o with the test
AaronBallman 680c379
Fix -Wreorder diagnostics; NFC
AaronBallman a0f8278
Clean up these constructors to take a SmallVectorImpl
AaronBallman 4d9ed9e
Fix a crash with argument parsing
AaronBallman 8a466f3
Back out unrelated CMake changes
AaronBallman a3d4b13
Remove a spurious #undef; NFC
AaronBallman 7dad1be
Backing out more unnecessary CMake changes
AaronBallman 29ac376
Correct the logic for this diagnostic checking function
AaronBallman e4e28eb
Fix think-o with test to get it to pass
AaronBallman ab5f8c2
Restore previous behavior; fixes two more failing test cases
AaronBallman 0cda935
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman 9d5eadf
Clean up the way we expose the __STDC_EMBED_*__ macros; NFC
AaronBallman f88a1ae
Fix a broken pp-trace test
AaronBallman e7ef292
Remove __builtin_pp_embed as a builtin function; NFC
AaronBallman 7c6bc7b
Add a test for feature testing the builtin
AaronBallman 038c90d
Correct parsing behavior and add tests
AaronBallman c204b73
No longer expose the embed driver options to Flang
AaronBallman ec01bec
Fix type mismatch that was upsetting the precommit CI bot
AaronBallman f57334a
Fix misuse of Twine and add a test
AaronBallman 8ef8da3
Remove unused variable; NFC
AaronBallman a5517cb
Fix a typo that snuck into this test
AaronBallman 7f856dd
Fix another type mismatch that was upsetting the precommit CI bot
AaronBallman 6d49cc8
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman b8a57c2
Fix new compile error from rebase; NFC
AaronBallman 6a6f813
Replace __builtin_pp_embed with annotation tokens
AaronBallman c7e1304
Formatting changes; NFC
AaronBallman 8c1a8fb
Change how we handle prefix and suffix tokens
AaronBallman 14d08b6
Add some FIXME comments about AST fidelity; NFC
AaronBallman 16cfd31
Backing out an unneeded change; NFC
AaronBallman 528077e
Correct diagnostic behavior for ext and compat warnings
AaronBallman 23eaf98
Fix formatting; NFC
AaronBallman 626b1e4
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman a10d9d6
Fix the way we preprocess to a file
AaronBallman a245899
Fix formatting; NFC
AaronBallman 00c6ff9
Attempt to appease clang-format; NFC
AaronBallman f36c95f
Improve the __has_embed tests somewhat
AaronBallman a76fa14
Use named constants in test instead of magic values; NFC
AaronBallman 1dcc449
Start reworking the way we represent a PPEmbedExpr
AaronBallman 1bc8f7b
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman aee2f34
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman 8433fd5
Rename `-embed-dir` to `--embed-dir` and only accept the = form
AaronBallman d6c3c90
Remove the embed path group; it's not necessary
AaronBallman 6abddb6
Replace some uses of string literals in diagnostics; NFC
AaronBallman 08004b8
Simplify the implementation somewhat; NFC intended
AaronBallman 257fc01
Correct the way we look up files with __has_embed vs #embed
AaronBallman 137961a
Test and fix the behavior for dependency file generation
AaronBallman 7dbd5f7
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman a91723c
Remove changes for clang-format
AaronBallman 0d6ea7d
No longer pass around base64 encoded data
AaronBallman 178cd3e
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman fbb0ff2
Fix the diagnostic group for unknown directives
AaronBallman 250ec6b
Remove a newly-added unused diagnostic group, add test coverage
AaronBallman 102d683
Remove more diagnostics related to CHAR_BIT
AaronBallman 659e6b4
Back out unrelated changes
AaronBallman 7f83440
Correct header file comments; NFC
AaronBallman df7aeae
Use a single annotation token rather than play parsing games
AaronBallman bab4ffb
Remove unused data member; NFC
AaronBallman 830ab5d
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman 013bb3e
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman b8c08af
Correct and simplify embed file lookup logic
AaronBallman 0a185ab
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman 41c3927
No longer copy file contents to a buffer
AaronBallman 928846c
Don't need a const size_t &; NFC
AaronBallman eb46ed0
Refactor code for clarity based on review feedback; NFC
AaronBallman 828e497
Improve test coverage
AaronBallman 69bb3bd
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman 7cf6156
Rework the way we parse embed parameters, add tests
AaronBallman 5499603
Fix parsing for __has_embed, add comprehensive tests
AaronBallman 37cbc63
Remove copy pasta tests; NFC
AaronBallman 7f793f2
Formatting fixes; NFC
AaronBallman e3a1a32
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman ce1fc30
Add test coverage for unknown quoted file paths
AaronBallman c631f08
Fix handling of the limit() parameter and add tests
AaronBallman 456201d
Correct the behavior of clang::offset and add more tests
AaronBallman 2675f69
Properly diagnose duplicate parameters
AaronBallman dd62b3f
Emit an extension warning for clang::offset
AaronBallman ec97562
Support parameters with leading and trailing double underscores
AaronBallman 0dd04f2
Add test coverage from examples in the standard
AaronBallman aa86a45
Update the C feature status page.
AaronBallman 64ad63c
Add documentation
AaronBallman efd51ef
Diagnose use of 'defined' within clang::offset and limit
AaronBallman 3bc8fac
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman b594f41
Remove unused include; NFC
AaronBallman 6cfb532
Back out unrelated change; NFC
AaronBallman 99e7828
Remove incomplete code that isn't needed
AaronBallman 83d7bbc
Add this diagnostic to a warning group
AaronBallman f0ece93
Remove unnecessary changes; NFC
AaronBallman 35461ea
Fix behavior of offset in __has_embed
AaronBallman b43d16c
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman 140b65e
Fix comments based on review feedback; NFC
AaronBallman 548bed4
Remove C++ feature test macro
AaronBallman 578259a
Use a 64-bit bit-field instead of a 32-bit one
AaronBallman bdac0d8
Refactor the element iterator based on review feedback; NFC
AaronBallman 117814b
Unify these diagnostics; NFC
AaronBallman aa0a08d
Use SourceRange rather than a pair of SourceLocations
AaronBallman 0ce7899
Document this interface better; NFC
AaronBallman 56cc314
Merge branch 'main' into thephd/embed-speed
AaronBallman 03669ce
Merge remote-tracking branch 'origin/main' into thephd/embed-speed
AaronBallman e886b04
Merge branch 'main' into thephd/embed-speed
cor3ntin 5432f41
Fix some diagnostics; NFC
AaronBallman 8148041
Merge remote-tracking branch 'main' into thephd/embed-speed
h-vetinari a7a1638
Merge branch 'main' into thephd/embed-speed
Fznamznon 0273781
Fix test
Fznamznon fc6c6a4
Improve generic #embed usage case
Fznamznon 2b1c0a8
Merge branch 'main' into thephd/embed-speed
Fznamznon ff94a01
Fix format, remove unused variable
Fznamznon 87dd643
Fix a couple of minor problems
Fznamznon b1b50c5
Remove ExpandSinglePPEmbedExpr
Fznamznon cc97fcd
Remove Expand* and Modify* functions
Fznamznon 1509ffd
Merge branch 'main' into thephd/embed-speed
Fznamznon cfda22f
Merge PPEmbedExpr and EmbedSubscriptExpr into EmbedExpr
Fznamznon 25c98d2
Merge branch 'main' into thephd/embed-speed
Fznamznon 31a967b
Remove CheckExprListForPPEmbedExpr
Fznamznon 757b54f
Minor cleanup
Fznamznon ca665b2
Add a couple of test cases
Fznamznon 4fd08eb
Fix broken test case
Fznamznon 8629e19
Cleanups
Fznamznon a17b0d0
Fix format concerns
Fznamznon ac6cc3b
Merge branch 'main' into thephd/embed-speed
Fznamznon 4bd9965
Merge branch 'main' into thephd/embed-speed
Fznamznon b4513aa
Apply comments, cleanup EmbedExpr a bit
Fznamznon 00990de
A couple of comments applied
Fznamznon 179485d
Merge branch 'main' into thephd/embed-speed
Fznamznon 2521ae3
Improve performance of constexpr evaluation
Fznamznon becb80b
Resolve review comments
Fznamznon 27ce354
Revert "Emit an extension warning for clang::offset"
Fznamznon a39062e
Fix format
Fznamznon 9f3d527
Merge branch 'main' into thephd/embed-speed
Fznamznon a3615ed
Expose -dE only as cc1 option, fix a couple of warnings
Fznamznon File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4799,6 +4799,167 @@ class SourceLocExpr final : public Expr { | |
friend class ASTStmtReader; | ||
}; | ||
|
||
/// Stores data related to single #embed directive. | ||
Fznamznon marked this conversation as resolved.
Show resolved
Hide resolved
|
||
struct EmbedDataStorage { | ||
StringLiteral *Filename; | ||
StringLiteral *BinaryData; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think I'm fine shipping this-as is, but I am not sure a |
||
size_t getDataElementCount() const { return BinaryData->getByteLength(); } | ||
}; | ||
|
||
/// Represents a reference to #emded data. By default references the whole | ||
Fznamznon marked this conversation as resolved.
Show resolved
Hide resolved
|
||
/// range. Otherwise epresents a subrange of data imported by #embed directive. | ||
Fznamznon marked this conversation as resolved.
Show resolved
Hide resolved
|
||
/// Needed to handle nested initializer lists with #embed directives. | ||
/// Example: | ||
/// struct S { | ||
/// int x, y; | ||
/// }; | ||
/// | ||
/// struct T { | ||
/// int x[2]; | ||
/// struct S s | ||
/// }; | ||
/// | ||
/// struct T t[] = { | ||
/// #embed "data" // data contains 10 elements; | ||
/// }; | ||
/// The resulting semantic form of initializer list will contain (EE stands | ||
AaronBallman marked this conversation as resolved.
Show resolved
Hide resolved
|
||
/// for EmbedExpr): | ||
/// { {EE(first two data elements), {EE(3rd element), EE(4th element) }}, | ||
/// { {EE(5th and 6th element), {EE(7th element), EE(8th element) }}, | ||
/// { {EE(9th and 10th element), { zeroinitializer }}} | ||
/// | ||
/// EmbedExpr inside of a semantic initializer list and referencing more than | ||
/// element can only appear for arrays of scalars. | ||
class EmbedExpr final : public Expr { | ||
SourceLocation BuiltinLoc, RParenLoc; | ||
Fznamznon marked this conversation as resolved.
Show resolved
Hide resolved
|
||
DeclContext *ParentContext; | ||
Fznamznon marked this conversation as resolved.
Show resolved
Hide resolved
|
||
IntegerLiteral *FakeChildNode = nullptr; | ||
const ASTContext *Ctx = nullptr; | ||
EmbedDataStorage *Data; | ||
unsigned Begin = 0; | ||
unsigned NumOfElements; | ||
|
||
public: | ||
EmbedExpr(const ASTContext &Ctx, SourceLocation BLoc, | ||
SourceLocation RParenLoc, DeclContext *ParentContext, | ||
EmbedDataStorage *Data, unsigned Begin, unsigned NumOfElements); | ||
explicit EmbedExpr(EmptyShell Empty) : Expr(SourceLocExprClass, Empty) {} | ||
|
||
const DeclContext *getParentContext() const { return ParentContext; } | ||
DeclContext *getParentContext() { return ParentContext; } | ||
|
||
SourceLocation getLocation() const { return BuiltinLoc; } | ||
SourceLocation getBeginLoc() const { return BuiltinLoc; } | ||
SourceLocation getEndLoc() const { return RParenLoc; } | ||
|
||
StringLiteral *getFilenameStringLiteral() const { return Data->Filename; } | ||
StringLiteral *getDataStringLiteral() const { return Data->BinaryData; } | ||
EmbedDataStorage *getData() const { return Data; } | ||
|
||
unsigned getStartingElementPos() const { return Begin; } | ||
size_t getDataElementCount() const { | ||
return NumOfElements; | ||
} | ||
|
||
template <bool Const> | ||
class ChildElementIter | ||
AaronBallman marked this conversation as resolved.
Show resolved
Hide resolved
|
||
: public llvm::iterator_facade_base< | ||
ChildElementIter<Const>, std::random_access_iterator_tag, | ||
std::conditional_t<Const, const IntegerLiteral *, | ||
IntegerLiteral *>> { | ||
friend class EmbedExpr; | ||
|
||
EmbedExpr *EExpr = nullptr; | ||
unsigned long long CurOffset = ULLONG_MAX; | ||
using BaseTy = typename ChildElementIter::iterator_facade_base; | ||
|
||
ChildElementIter(EmbedExpr *E) : EExpr(E) { | ||
if (E) | ||
CurOffset = E->getStartingElementPos(); | ||
} | ||
|
||
public: | ||
ChildElementIter() : CurOffset(ULLONG_MAX) {} | ||
typename BaseTy::reference operator*() const { | ||
assert(EExpr && CurOffset != ULLONG_MAX && | ||
"trying to dereference an invalid iterator"); | ||
IntegerLiteral *N = EExpr->FakeChildNode; | ||
StringRef DataRef = EExpr->Data->BinaryData->getBytes(); | ||
N->setValue(*EExpr->Ctx, | ||
llvm::APInt(N->getValue().getBitWidth(), DataRef[CurOffset], | ||
N->getType()->isSignedIntegerType())); | ||
// We want to return a reference to the fake child node in the | ||
// EmbedExpr, not the local variable N. | ||
return const_cast<typename BaseTy::reference>(EExpr->FakeChildNode); | ||
} | ||
typename BaseTy::pointer operator->() const { return **this; } | ||
using BaseTy::operator++; | ||
ChildElementIter &operator++() { | ||
assert(EExpr && "trying to increment an invalid iterator"); | ||
assert(CurOffset != ULLONG_MAX && | ||
"Already at the end of what we can iterate over"); | ||
if (++CurOffset >= | ||
EExpr->getDataElementCount() + EExpr->getStartingElementPos()) { | ||
CurOffset = ULLONG_MAX; | ||
EExpr = nullptr; | ||
} | ||
return *this; | ||
} | ||
bool operator==(ChildElementIter Other) const { | ||
return (EExpr == Other.EExpr && CurOffset == Other.CurOffset); | ||
} | ||
}; // class ChildElementIter | ||
|
||
public: | ||
using fake_child_range = llvm::iterator_range<ChildElementIter<false>>; | ||
using const_fake_child_range = llvm::iterator_range<ChildElementIter<true>>; | ||
|
||
fake_child_range underlying_data_elements() { | ||
return fake_child_range(ChildElementIter<false>(this), | ||
ChildElementIter<false>()); | ||
} | ||
|
||
const_fake_child_range underlying_data_elements() const { | ||
return const_fake_child_range( | ||
ChildElementIter<true>(const_cast<EmbedExpr *>(this)), | ||
ChildElementIter<true>()); | ||
} | ||
|
||
child_range children() { | ||
return child_range(child_iterator(), child_iterator()); | ||
} | ||
|
||
const_child_range children() const { | ||
return const_child_range(const_child_iterator(), const_child_iterator()); | ||
} | ||
|
||
static bool classof(const Stmt *T) { | ||
return T->getStmtClass() == EmbedExprClass; | ||
} | ||
|
||
ChildElementIter<false> begin() { | ||
return ChildElementIter<false>(this); | ||
} | ||
|
||
ChildElementIter<true> begin() const { | ||
return ChildElementIter<true>(const_cast<EmbedExpr *>(this)); | ||
} | ||
|
||
template <typename Foo, typename... Targs> | ||
bool doForEachDataElement(Foo F, unsigned &StartingIndexInArray, | ||
Targs... Fargs) const { | ||
for (auto It : underlying_data_elements()) { | ||
if (!F(const_cast<IntegerLiteral *>(It), StartingIndexInArray, Fargs...)) | ||
return false; | ||
StartingIndexInArray++; | ||
} | ||
return true; | ||
} | ||
Fznamznon marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
private: | ||
friend class ASTStmtReader; | ||
}; | ||
|
||
/// Describes an C or C++ initializer list. | ||
/// | ||
/// InitListExpr describes an initializer list, which can be used to | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My changes in SemaInit.cpp likely caused this, however the outcome seems correct to me, so I just changed the test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not certain this change makes sense to me --
int
is sufficiently small that you typically would not use a reference for it.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I rolled it back.
Though I'm not certain I agree with size argument (in the same test I see references used for int in other places), I don't want to alter clang-tidy behavior unexpectedly with
#embed
implementation.