Skip to content

Major rewrite of treegen, add % operator, refactoring, syntax highlighting, tests [mega-PR] #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
MarcellPerger1 merged 89 commits into from
Sep 28, 2024
Merged

Major rewrite of treegen, add % operator, refactoring, syntax highlighting, tests [mega-PR] #21

MarcellPerger1 merged 89 commits into from
Sep 28, 2024

Conversation

@MarcellPerger1
Copy link
Owner

@MarcellPerger1 MarcellPerger1 commented Aug 8, 2024 •
edited
Loading

  • Big rewrite of treegen (much more readable now, and not a complete incomprehensible probably-buggy mess) using recursion
  • More tests, more snapshot tests
  • Added fuzzing using pythonfuzz (although it's a bit outdated and a bit broken on Windows) (no CI yet)
  • Our filenames end in .st (Scratch Text)
  • Custom Textmate bundle for syntax highlighting 🎉 🚀
  • Other fixes (including crashes/internal errors, timeouts (a lot of them caught by pythonfuzz))
  • fuzzing on CI (it's free!)
  • ( Make fuzzing add new branch and commit #22 is for this) Make fuzzing add new branch (see 61c3f42)
  • TODO: fix all the TODOs in the code relating to this PR

@MarcellPerger1
Add '..' to operators.py and other minor fixes
6e99546
@MarcellPerger1
refactor: Start reimplementing expr parsing, do abstract grammar
783f168
@MarcellPerger1
refactor: Reimplement level 3-5
6b3124b
@MarcellPerger1
refactor: Reimplement level 6-11
ab64b4f
@MarcellPerger1
refactor: Integrate new impl, fix a few bugs
9be4b37 
(Still may be a few bugs I think)
@MarcellPerger1
test: Add more tests
440f311
@MarcellPerger1
fix: Fix the snapshot (old code was wrong!)
71cc722
@MarcellPerger1
fix: Fix bad snapshot due to old code being bad
b4a5491
@MarcellPerger1
feat: Our filenames end with .st now
f41191e 
(Scratch Text, <u>S</u>cratch <u>T</u>ext)
@MarcellPerger1
fix: Fix bugs
b484126
@MarcellPerger1
test: Test main.py to check no exceptions thrown.
b3eabe2 
TODO: snapshot test them
@MarcellPerger1
fix: Fix working dir issues
6c02a01
@MarcellPerger1
test: Test methods calls in lvalue
ce1ae00
@MarcellPerger1
fix: Fix underscores in numbers
36e7ed8
@MarcellPerger1
fix: Fix augmented assignment
406a413
@MarcellPerger1
refactor: Remove stuff conditional on !USE_NEW
b318a08
@MarcellPerger1
refactor: Remove old code
47e1e45
@MarcellPerger1
refactor: Remove a bunch of unused stuff (thanks Vulture)
3414b62
@MarcellPerger1
refactor: Remove old brk_reason
4640c95
@MarcellPerger1
refactor: Remove old partial* args to _parse_expr
33df6f4
@MarcellPerger1
perf: Add more timing to main.py
f7d5c7f 
Note: the printing takes ages (73ms/86ms = 85%)
@MarcellPerger1
test: Add some tests
9c5cb57 
Some of them are failing!
@MarcellPerger1
fix: Remove old code, finish checking self[idx] stuff
d246ea3 
Also, added tests for EOF stuff
@MarcellPerger1
refactor: Add actual token to expected * error messages
38df0aa
@MarcellPerger1
fix: Fix decl without value
705928d
@MarcellPerger1
fix: Fix EOF errors in tokenizer
3458bd2
@MarcellPerger1
test: Add fuzzing
b66ad9c
@MarcellPerger1
fix: Fix the bug that fuzzing has caught
1a5f54d
@MarcellPerger1
refactor: Add only-slightly-fragile timeout decorator
d51aef3
@MarcellPerger1
fix: Fix pickling errors
b8b6d5f
@MarcellPerger1 MarcellPerger1 mentioned this pull request Sep 28, 2024
Closed
@MarcellPerger1 MarcellPerger1 mentioned this pull request Sep 28, 2024
Closed
@MarcellPerger1
refactor: Unify shared worker functions
d224f6e
@MarcellPerger1
refactor: Remove unneeded del
7e5f819
@MarcellPerger1
refactor: Use cleaner dict.pop
8f72b34
@MarcellPerger1
refactor: Add option to include waiting for process in timeout
98d3acf
@MarcellPerger1
refactor: Extract _create_task function in SimpleProcessPool
8839e52
@MarcellPerger1
refactor: Extract to more methods for clearer code
0631c20
@MarcellPerger1
refactor: Extract to more methods for clearer code
ae44bff
@MarcellPerger1
refactor: Add .kill() method to _ProcessWrapper
d938811
@MarcellPerger1
refactor: Rename variable for clarity
dec47ae
@MarcellPerger1
refactor: Add internal put_task_result to SimpleProcessPool
ec1f93f
@MarcellPerger1
refactor: Rename _ProcessWrapper.waiting for clarity, only update sta…
385ccf7 
…tus if not just created
@MarcellPerger1
refactor: Set default interval to 0
596be6c
@MarcellPerger1
docs: Add some docstring about pool arg to run_with_timeout_async
8f2af01
@MarcellPerger1
refactor: Finish _PickleWrapper explanation
15ef499
@MarcellPerger1
Copy link
Owner Author

MarcellPerger1 commented Sep 28, 2024

All, review comments resolved, I'll merge this.

@MarcellPerger1 MarcellPerger1 merged commit 26584e9 into main Sep 28, 2024
@MarcellPerger1 MarcellPerger1 deleted the tidy-treegen branch September 28, 2024 19:13
@MarcellPerger1 MarcellPerger1 added T-refactor Refactor T-tests Type: Tests (e.g. add more tests) A-CST Area: Concrete Syntax Tree Generation (CST) A-Tests Area: Tests (e.g. improving test infrastructure) A-TextmateBundle Area: The TextMate Bundle (textmate_bundles/st) labels Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

A-CST Area: Concrete Syntax Tree Generation (CST) A-Tests Area: Tests (e.g. improving test infrastructure) A-TextmateBundle Area: The TextMate Bundle (textmate_bundles/st) T-enhancement Improvement to an existing feature T-refactor Refactor T-tests Type: Tests (e.g. add more tests)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@MarcellPerger1