-
Notifications
You must be signed in to change notification settings - Fork 102
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DO NOT MERGE] Server side snowpark #1538
base: main
Are you sure you want to change the base?
Commits on May 3, 2024
-
[private branch] Maintain the AST as we build up Snowpark expressions…
…: table(), filter() (#1468) A very basic initial attempt at serializing the AST. I'm trying to maintain a parallel codebase for phases 0 and 1 for now, since it would be a shame to do this work twice. Once we complete and ship phase 0, we'll be able to drastically simplify the phase 1 client. Unlike what I mentioned before, this implementation doesn't flush dependencies of eagerly evaluated expressions. Instead, any client-side value is appended to the pending batch. This is simpler to implement and will likely work well, although we may need to do some dependency analysis on the server to ensure we don't issue unnecessary queries.
Configuration menu - View commit details
-
Copy full SHA for 7a92099 - Browse repository at this point
Copy the full SHA 7a92099View commit details -
Configuration menu - View commit details
-
Copy full SHA for e13fa28 - Browse repository at this point
Copy the full SHA e13fa28View commit details -
Merge branch 'server-side-snowpark' of https://github.com/snowflakedb…
…/snowpark-python into server-side-snowpark
Configuration menu - View commit details
-
Copy full SHA for ae3469d - Browse repository at this point
Copy the full SHA ae3469dView commit details
Commits on May 8, 2024
-
[SERVER] Refresh dev branch with recent snowpark changes (#1537)
Updates our server branch with recent snowpark changes.
Configuration menu - View commit details
-
Copy full SHA for 4305cf2 - Browse repository at this point
Copy the full SHA 4305cf2View commit details
Commits on May 14, 2024
-
Update ast_pb2.py and include it in the Snowpark wheel (#1564)
<!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-0 2. Fill out the following pre-review checklist: - [ ] I am adding a new automated test(s) to verify correctness of my new code - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [x] I am adding a new dependency 3. Please describe how your code solves the related issue. Update `ast_pb2.py` (already present in the repository). Add the `setuptools` dependencies required for development. Include the module path for `ast_pb2.py` in the manifest, so that the file makes it into the Snowpark wheel.
Configuration menu - View commit details
-
Copy full SHA for e6ce3bb - Browse repository at this point
Copy the full SHA e6ce3bbView commit details
Commits on May 15, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 7fbbc51 - Browse repository at this point
Copy the full SHA 7fbbc51View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0cc4fa8 - Browse repository at this point
Copy the full SHA 0cc4fa8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2a9b50b - Browse repository at this point
Copy the full SHA 2a9b50bView commit details
Commits on May 16, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 2e54569 - Browse repository at this point
Copy the full SHA 2e54569View commit details
Commits on May 21, 2024
-
[feature branch] Async session request + hacked up delay to avoid rac…
…es until we send a response on TCM creation (#1617)
Configuration menu - View commit details
-
Copy full SHA for 1446f51 - Browse repository at this point
Copy the full SHA 1446f51View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7657d40 - Browse repository at this point
Copy the full SHA 7657d40View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3c144f8 - Browse repository at this point
Copy the full SHA 3c144f8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6537943 - Browse repository at this point
Copy the full SHA 6537943View commit details
Commits on May 24, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 29684a4 - Browse repository at this point
Copy the full SHA 29684a4View commit details -
Configuration menu - View commit details
-
Copy full SHA for b086fd2 - Browse repository at this point
Copy the full SHA b086fd2View commit details
Commits on May 28, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 8de5d28 - Browse repository at this point
Copy the full SHA 8de5d28View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7275dfa - Browse repository at this point
Copy the full SHA 7275dfaView commit details
Commits on May 30, 2024
-
[Snowpark Server] Helper script to fetch protobuf from devvm (#1700)
Run `update-from-devvm.sh` from within `src/snowflake/snowpark/_internal/proto/` with a running local devvm to update the proto file on the thin client.
Configuration menu - View commit details
-
Copy full SHA for f16f1cb - Browse repository at this point
Copy the full SHA f16f1cbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 037fc60 - Browse repository at this point
Copy the full SHA 037fc60View commit details
Commits on Jun 5, 2024
-
Configuration menu - View commit details
-
Copy full SHA for e4a10f9 - Browse repository at this point
Copy the full SHA e4a10f9View commit details
Commits on Jun 10, 2024
-
Configuration menu - View commit details
-
Copy full SHA for d30822a - Browse repository at this point
Copy the full SHA d30822aView commit details
Commits on Jun 12, 2024
-
Configuration menu - View commit details
-
Copy full SHA for bce97c0 - Browse repository at this point
Copy the full SHA bce97c0View commit details -
[ThinClient] Use connector with server-side-snowpark changes in setup…
….py (#1766) Modifies `setup.py` to use the latest HEAD of https://github.com/snowflakedb/snowflake-connector-python/tree/server-side-snowpark which includes connector changes (most notable adding the `_dataframe_ast` field for phase 0). To update your local dev environment run ``` pip uninstall snowflake-connector-python -y python -m pip install --no-cache -e ".[development,pandas]" ``` Running the pip command should show `git clone` in the logs.
Configuration menu - View commit details
-
Copy full SHA for e20a430 - Browse repository at this point
Copy the full SHA e20a430View commit details -
[SNOW-1473697] Support show client-side with modified TCM (#1649)
This is the thin-client PR complementary to https://github.com/snowflakedb/snowflake/pull/183143/files.
Configuration menu - View commit details
-
Copy full SHA for 922881b - Browse repository at this point
Copy the full SHA 922881bView commit details
Commits on Jun 14, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 385ac60 - Browse repository at this point
Copy the full SHA 385ac60View commit details
Commits on Jun 17, 2024
-
Configuration menu - View commit details
-
Copy full SHA for ce81f68 - Browse repository at this point
Copy the full SHA ce81f68View commit details -
[SERVER-SIDE] Add test infrastructure to track all requests have data…
…frameAST field. (#1794) Vendors snowflake vcrpy from https://github.com/Snowflake-Labs/snowflake-vcrpy (could not get install working, therefore vendoring it) with custom Snowflake changes to track requests in vendored urllib3 within the snowflake python connector. Adds decorator `check_ast_encode_invoked` (applied with `autouse=True` to all tests) which checks that every query send contains `dataframeAst` property for phase 0, and errors out together with traceback information whenever tests need to be fixed / APIs are missing that need to be encoded within the AST.
Configuration menu - View commit details
-
Copy full SHA for 9eef249 - Browse repository at this point
Copy the full SHA 9eef249View commit details
Commits on Jun 18, 2024
-
[SERVER-SIDE] Remove temporarily Modin tests as Modin is incompatible…
… with Python 3.8 (#1796) Remove temporarily Modin tests as Modin is incompatible with Python 3.8.
Configuration menu - View commit details
-
Copy full SHA for 8d29d02 - Browse repository at this point
Copy the full SHA 8d29d02View commit details
Commits on Jun 20, 2024
-
[SERVER-SIDE CI] Exclude protobuf file in flake8 precommit (#1812)
Exclude protobuf file in precommit (flake8), as protobuf does not adhere to good coding standards and a protobuf update will consequently fail precommit.
Configuration menu - View commit details
-
Copy full SHA for 718fed4 - Browse repository at this point
Copy the full SHA 718fed4View commit details -
[SERVER-SIDE CI] In update-unparser.sh script use bazel targets inste…
…ad of sbt (#1811) Convenience script `update-unparser.sh` uses sbt build currently. With introduction of the bazel scala targets dependencies (i.e., IR changes) switch to using them to create an updates unparser.sh as IR/protobuf updates will be reflected.
Configuration menu - View commit details
-
Copy full SHA for c92e511 - Browse repository at this point
Copy the full SHA c92e511View commit details
Commits on Jun 25, 2024
-
[SERVER-SIDE CI] Temporary fix for unparser fat jar (#1817)
Fixes issue of bazel not producing a fat jar in `scala_binary` by calling both the bazel build and the sbt build. This is a temporary workaround till we fix in bazel the rules to emit a fat jar.
Configuration menu - View commit details
-
Copy full SHA for 58eaa3c - Browse repository at this point
Copy the full SHA 58eaa3cView commit details
Commits on Jun 27, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 4cecfdc - Browse repository at this point
Copy the full SHA 4cecfdcView commit details
Commits on Jun 28, 2024
-
Configuration menu - View commit details
-
Copy full SHA for f513ea9 - Browse repository at this point
Copy the full SHA f513ea9View commit details
Commits on Jul 1, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 424bab1 - Browse repository at this point
Copy the full SHA 424bab1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6834c36 - Browse repository at this point
Copy the full SHA 6834c36View commit details
Commits on Jul 2, 2024
-
Configuration menu - View commit details
-
Copy full SHA for f7f969d - Browse repository at this point
Copy the full SHA f7f969dView commit details
Commits on Jul 3, 2024
-
[SNOW-1491328] Support functions.py for AST/Snowpark IR. (#1810)
[SNOW-1491328](https://snowflakecomputing.atlassian.net/browse/SNOW-1491328) Support AST encoding for functions (up to table functions, UDFx) in `functions.py`. Details: - Fix `update_from_devvm.sh` script to invoke both bazel builds required. - Add `check=True` to `test_ast_driver.py` to error out when subprocess fails. - Removes cls method `._fill_ast` from `Column` class and refactors into free method within `ast_utils.py`. - Introduces new `build_fn_apply` helper function to create `ApplyExpr` in IR. [SNOW-1491328]: https://snowflakecomputing.atlassian.net/browse/SNOW-1491328?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: Arthur Zwiegincew <arthur.zwiegincew@snowflake.com>
Configuration menu - View commit details
-
Copy full SHA for 5ce96c1 - Browse repository at this point
Copy the full SHA 5ce96c1View commit details