Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update surround_substring to surround all substrings #209

Closed
MrPowers opened this issue Feb 23, 2024 · 2 comments · Fixed by #212
Closed

Update surround_substring to surround all substrings #209

MrPowers opened this issue Feb 23, 2024 · 2 comments · Fixed by #212

Comments

@MrPowers
Copy link
Owner

It works properly when there is only one substring like here: surround_substring("spark rdd stuff", "rdd", "**", "||").

It does not work properly when there are two substrings, like here: surround_substring("spark rdd stuff with rdd", "rdd", "**", "||").

Screenshot 2024-02-23 at 10 54 28 AM
@SemyonSinchenko
Copy link
Collaborator

@MrPowers What do you think about the case when we want to branch depending on SparkConnect? For example:

if hasattr(spark, "_jvm"):
    spark_code(...)
else:
    spark_connect_code(...)

@MrPowers
Copy link
Owner Author

@SemyonSinchenko - the purpose of this code is to run on other libraries and identify any files that contain potentially problematic code.

I am considering creating a separate project with the intention of making Spark Connect less breaking. That library would use the design pattern you outlined above. It would be great if we could monkey patch in the "old way" and just do the new "Spark Connect way" under the hood (when possible of course).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants