-
Notifications
You must be signed in to change notification settings - Fork 62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add some utils into LexAccessor and StyleContext #54
Comments
void BackTo(Sci_PositionU startPos) {
assert(startPos <= styler.GetStartSegment());
if (startPos < styler.GetStartSegment()) {
styler.Flush();
styler.StartAt(startPos);
styler.StartSegment(startPos);
}
SeekTo(startPos);
} These function are useful to write advanced lexers with try parse rewind on failure approach, e.g. my enhanced lexer for GitHub Flavored Markdown, GitLab Flavored Markdown and Pandoc’s Markdown (though it still needs further work to handle indented code block). |
The implementation of
|
for It should documented
if (pWidth && *pWidth < 0) {
position = MovePositionOutsideChar(position, -1, false);
} |
…fy logic and improve performance.
Pondered the name a bit and went for |
The proposed const FullCharacter ch = styler.CharacterAt(nextPos);
const auto [ch, width] = styler.CharacterAt(nextPos); --- a/lexlib/LexAccessor.h
+++ b/lexlib/LexAccessor.h
@@ -12,6 +12,11 @@ namespace Lexilla {
enum class EncodingType { eightBit, unicode, dbcs };
+struct FullCharacter {
+ int character = 0;
+ Sci_Position width = 1;
+};
+
class LexAccessor {
private:
Scintilla::IDocument *pAccess;
@@ -80,6 +85,11 @@ public:
Scintilla::IDocument *MultiByteAccess() const noexcept {
return pAccess;
}
+ FullCharacter CharacterAt(Sci_Position position) const {
+ FullCharacter fullCharacter;
+ fullCharacter.character = pAccess->GetCharacterAndWidth(position, &fullCharacter.width);
+ return fullCharacter;
+ }
/** Safe version of operator[], returning a defined value for invalid position. */
char SafeGetCharAt(Sci_Position position, char chDefault=' ') {
if (position < startPos || position >= endPos) { |
OK then. Maybe |
Updated |
There also needs better names for these method, may be rename |
The Linked is a patch that does that. The patch does some of the same things as the above patches. It also makes fields const where possible, makes lineDocEnd private, initializes with field assignments where trivial, initializes using the initializer list when dependent on data. |
… for fields that do not change, initialize in field declaration or initializer list instead of constructor.
The seeking methods are the main outstanding feature of this proposal. I think something similar could be useful but its difficult to understand when and how to use them (particularly the single line / multiple line split) so its less likely lexer developers will incorporate these techniques. Perhaps there is a subset that is strongly motivated and could be presented as a simple technique to use. An approach could be to implement the feature inside a lexer (possibly with a StyleContext subclass) then refine it until it appears finished then move it up into StyleContext. |
The code is located at https://github.com/zufuliu/notepad2/blob/main/scintilla/lexlib/StyleContext.h#L218, multiline rewind is used by the Markdown lexer (with |
lexlib-0114.zip
code extract from https://github.com/zufuliu/notepad2/blob/main/scintilla/lexlib/LexAccessor.h and https://github.com/zufuliu/notepad2/blob/main/scintilla/lexlib/StyleContext.h
LexAccessor
addedGetCharacterAndWidth()
andStyleAtEx()
(to get cache style without callFlush()
).StyleContext
addedSeekTo()
,Rewind()
,BackTo()
andAdvance()
.GetDocNextChar()
andGetLineNextChar()
(currently only skip ASCII whitespaces) may also useful for writing lexers.The text was updated successfully, but these errors were encountered: