Bug fix for python demo #708
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sfc-gh-sikram
added a commit
to sfc-gh-sabdelrahman/piranha
that referenced
this pull request
Jun 25, 2025
* Add a new argument for custom internal rules (uber#714) * Run polyglot_release.yml workflow on new tag pushes (uber#715) * Release v0.3.28 (uber#716) * Include LICENSE and MANIFEST in package (uber#717) * Release v0.3.29 (uber#718) * Add delete_comments Flag to Rules (uber#720) Allows rules to optionally preserve comments during code transformations. Defaults to current behavior (deletion), but can be disabled per rule * Release v0.3.30 (uber#721) * Bump ubuntu version (uber#722) * API changes for deciding whether or not to keep a comment at the rule level (uber#724) API changes for deciding whether or not to keep a comment at the rule level. Now instead of booleans, we take a set of regex * Changes the kotlin grammar dependency to my version (uber#723) Kotlin's grammar parses all expressions wrong. For example `-a + b` gets parsed to `-(a+b)`. This PR changes the grammar dependency to my version, which fixes that problem * [PolyglotPiranha] Prepare for release 0.3.32. (uber#725) * Changes the Kotlin grammar dependency to a custom branch due to problems with parsing expressions * API changes for deciding whether or not to keep a comment at the rule level * Releasing w/ Debian and select between PyPI or TestPyPI (uber#726) Adds: - debian container for our linux release (to be tested once merged) - workflow_dispatch to easily select between releasing to PyPI or TestPyPI --------- Co-authored-by: Yuxin Wang <yuxinwang.dev@gmail.com> * Fix kotlin grammar dependency after upgrade (uber#727) The latest update to the kotlin grammar made string literal non anonymous, which causes concrete syntax matches to fail. This update fixes that * Reverting release runs-on to unblock new versions (uber#728) Reverts the changes on `runs-on` but keeps the workflow_dispatch changes. Still doesn't solve our problems with Debian 11, but unblocks releasing new versions * Use CIBuildWheel to release polyglot-piranha to avoid linking to new glibc/glibcxx (uber#729) Our release workflow is heavily dependent on the OS we are running our release job in. Specifically, we need to build rust core using pyo3, which link against glibc the current OS provides. Due to the deprecation of ubuntu 20.04 in github actions, we have recently bumped our OS version in the release job, which also bumped the glibc version we linked against in the built wheels. This generally poses problems for older OSes (especially for the older OSes we still use internally). This PR uses [cibuildwheel](https://cibuildwheel.pypa.io/en/stable/) to build instead, which uses [manylinux](https://github.com/pypa/manylinux) container under the hood for building that use a relatively old version of glibc to ensure the built wheels have best compatibility. I tried this in my private fork and it works (only failing at the upload step due to lack of API token): https://github.com/yuxincs/piranha/actions/runs/15383618174 * Prepare for release 0.3.34 (uber#730) * Include `polyglot_piranha.pyi` file in the source distribution (uber#731) This would make source distribution fully-typed as well even when users are building from source instead of getting prebuilt wheels. * New simplified release process (uber#732) As title suggests, since we now have automated release workflow. We allow admins to update `Cargo.toml` and then push the tags to trigger the release process. * Fix typo in RELEASING.md (uber#733) * Update go to my version to fix new line issue (uber#734) The go grammar is broken because of new line issues. I forked the grammar that we had, and fixed it there * Release 0.3.35 * Build a tree-sitter playground with the grammars that Piranha uses (uber#736) Piranha uses several grammar repositories with custom patches to support the transformations. While these patches are being upstreamed, there may be discrepancies between the grammars in this repository and the upstream grammars. This PR adds a build script to (1) instantiate the `index.html.template` (slightly modified from [tree-sitter-cli-playground-html]) and copy it to the given `dist` directory. (2) clone the (custom) grammar repositories that we use in Piranha (by parsing the `Cargo.toml` file), check out the specific versions, and then build the WASM files for the grammars and copy them to the `dist/assets` directory. (3) the `dist` directory can be served by any web server (e.g., `python -m http.server`). We also add a GitHub Actions job to automatically deploy to GitHub Pages whenever the grammars have been updated in Piranha (or the build script / GitHub Actions configuration chanegs). This is tested in my fork: https://yuxincs.github.io/piranha/tree-sitter-playground/ [tree-sitter-cli-playground-html]: https://github.com/tree-sitter/tree-sitter/blob/eaa10b279f208b47f65e77833d65763f072f3030/crates/cli/src/playground.html#L13 * Build Tree-sitter playground on PRs as well (uber#737) This PR changes the tree-sitter playground github actions to also build on PRs to catch breakages, but only deploy on master branch. * Add asterisk (*) support for zero-or-more matching in concrete syntax (uber#735) ## Summary Added support for `:[var*]` pattern matching in concrete syntax templates, allowing zero-or-more node matching similar to regex quantifiers. ## Changes - Added `MatchMode` enum to replace boolean parameters for cleaner API - Implemented `ZeroPlus` mode in `handle_template_variable_matching` - Added unit tests for zero-or-more matching behavior ## Implementation Details The asterisk implementation first attempts to match zero nodes, then falls back to one-or-more matching if zero match fails. This ensures `:[var*]` behaves as expected: - `class Example { }` matches `class :[name] { :[body*] }` with empty body - `class Example { int x = 1; }` matches same pattern with body containing the field Like before, concrete syntax continues to be lazy during matching; (otherwise there is a state space explosion) * Parse concrete syntax with a proper grammar (uber#741) Concrete syntax has a hacky implementation based on string matching. If we want to expand the grammar to support constraints and other constructs, it's worthwhile to build a proper grammar. This will allow us to expand it in the future to use things like: - `where len(:[name]) > 2` for example to select nodes only with two or more children; - `where :[name] matches regex` with regex to add constraints on template variables. In this PR I essentially wrote the paper, but didn't touch the matching algorithm. In summary: - Added Pest grammar (concrete_syntax.pest) - Added parser that converts strings to build an actual AST - Moved the matching algorithm to [interpreter.rs](https://github.com/danieltrt/piranha/blob/a993eb15fd9f2317f928c6a94c96577ffb73336d/src/models/concrete_syntax/interpreter.rs#L44) * Modernize CI build configs (uber#744) * Upgrade the minimum python version to 3.9 * Upgrade several GH action versions * Upgrade pre-commit hook versions * Pin maturin to 1.8.7 since 1.9.0 introduced a packaging bug when it comes to including LICENSE files. * Minor refactors here and there. * Bug fix for python demo (uber#708) For some reason one of the rules is not working in the python demo (uber#707). I'm not sure if the problem is with the grammar. Nonetheless, the rule is much easier to write / understand in cs Co-authored-by: Nikhil Parasaram <tommycjniko@gmail.com> * Remove experimental directory (uber#745) Per discussion w/ @danieltrt , the Piranha playground is no longer being used and is safe to delete. If we need this in the future, we can always reference it in git history, and we can properly refactor and maintain it outside of experimental dir. Removing the dir helps with maintainability. * Set maturin constraint in pyproject.toml instead of github action script (uber#746) * Support CPP * Fix stupid rules * reorder if rules * remove usages of unsupported variable_declarator * Add scopes for cpp * more declarator fixes * Update README.md * Keep {} when removing unnecessary blocks * Fix remove_unnecessary_nested_block and delete_all_statements_after_return rules + bump version * Add CGPattern pattern getter * Split impl blocks --------- Co-authored-by: Nikhil Parasaram <tommycjniko@gmail.com> Co-authored-by: Daniel Ramos <danielrr@uber.com> Co-authored-by: Diego Marcilio <diego.marcilio@uber.com> Co-authored-by: Yuxin Wang <yuxinwang.dev@gmail.com> Co-authored-by: Sara Abdelrahman <sara.abdelrahman@snowflake.com> Co-authored-by: Corbin McElhanney <corbin.mcelhanney@snowflake.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For some reason one of the rules is not working in the python demo (#707). I'm not sure if the problem is with the grammar. Nonetheless, the rule is much easier to write / understand in cs