-
-
Notifications
You must be signed in to change notification settings - Fork 62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP: Fix blocks (Lexer/Parser rework) #104
Conversation
I'm finding it difficult to work with Using a version of Trying to figure out the nicest way to handle this that gives us good type safety but doesn't require external API changes. Specifically, the issue arises when using functions from |
Re For now, I'll keep thinking about how we can have good type safety internally, whilst allowing users to choose the less safe variant of |
Edit: the fix didn't work - propagated too quickly. However, discussions are going well so we might be able to use Edit: |
@xiaoxinghu update on block parsing. There are three main cases of block contents we need to watch out for:
I've updated the parser/lexer so it can now handle "greater" and "element" blocks, and to a degree it can handle "verse" blocks. However, as it currently stands, I'm needing to add exceptions for lexing objects for the verse block in a way that we will likely need to be able to apply in other areas, but currently isn't very maintainable. As a result, I'm looking into one of these two methods:
Point 1 has the advantage of simplifying some of the parsing, and means we don't have to worry about converting tokens in the parser. Point 2 has the advantage of simplifying lexing to be more clearly token-based without too much lookahead, but means that we may need to do some re-parsing depending on how we handle the lexer. I think point 2 currently feels a bit more daunting beacuse the lexer needs to be more atomic - once the lexer is more atomic then it should actually be easier. I'll look into point 2 and let you know how it goes - it should make life a lot easier in updating parsing in the future. |
08c69f3
to
a94a406
Compare
a94a406
to
a092485
Compare
Added some documentation too.
Much safer this way, managed to catch a bunch of potential crashes.
a092485
to
d39ae90
Compare
👍 Wow, what a refactor! |
Yep. I'm trying to be as comprehensive with tests as possible to make it easier to review, but it's always gonna be a bit of a beast. Once the backend stuff is done w/ the lexer and parser, I'll need to make sure that all the frontend packages work correctly with this, which might be harder. |
@GuiltyDolphin , this looks dangerous to me, I suggest that we break these changes down into smaller pieces, because I am doing some refactor work myself which is pretty major. Also, I am changing the way tests are written, so I think it'd be easier to do these tests after mine, I will try my best to get it out of my computer. |
@xiaoxinghu Alright! I'll close this for now until your changes are done, and then see how things look after that. |
WORK IN PROGRESS
Outline
Treatment of blocks
org-element-parse-buffer
parses this as a paragraph (with text like#+BEGIN_QUOTE
) inside a quote block - if you try to HTML export the document, Org produces entirely text in no blocks. Which option do we want here? I think allowing quote blocks inside quote blocks might be the best option, then the user can decide whether to render these as text or notTodo
@types/unist
version2.0.5
orgajs
packages to account for changes