Skip to content

Conversation

@xuyangzhong
Copy link
Contributor

@xuyangzhong xuyangzhong commented Jun 5, 2024

What is the purpose of the change

Currently, we depend on asm to extract method parameter names. We custom a MethodVisitor to visit local variable tables in file .class, and then cut the first N local variable names as the method parameter names.

However, if there are multi blocks about one local variable, the first N local variable names could be same, and then wrongly be extracted, and crashed when validating the conflict of them(the new logic added in FLINK-1.19).

This pr will use the slot index in .class file to extract the method parameter names, because method parameter names are always at the head in the 'slot index' list(https://docs.oracle.com/javase/specs/jvms/se17/html/jvms-3.html)

Take the test function ExtractionUtilsTest#MultiLocalVariableBlocksWithoutInitializationClass as an example:

  • Before fix:
  1. extract parameter names from asm:
    [localVariable, localVariable, localVariable, this, generic, genericFuture, listOfGenericFuture, array, localVariable]
  2. get the first 4 after offset 1(expected this in index 0)
    [localVariable, localVariable, this, generic]

This is because in local variable table, there are multi localVariable with different lifecycle in different blocks.

  • After fix:
  1. extract parameter names from asm:
    [this, generic, genericFuture, listOfGenericFuture, array, localVariable]
  2. get the first 4 after offset 1(expected this in index 0)
    [generic, genericFuture, listOfGenericFuture, array]

That's what we expected.

Further more, if the local variable has been initialized before if statement, its lifecycle is across all if blocks, and there is no need to init new local variable localVariable for it. Otherwize, localVariable in multi blocks are actually not visible although it is declared at first, and jvm will create multi localVariable for different blocks and add it in local variable table(You can see in local variable table, although these localVariable's slots are same, but their start label and end label are different).

Brief change log

  • Fix the logic in ExtractionUtils
  • Add tests to verify this fix

Verifying this change

Some tests are added to verity it.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented?

@flinkbot
Copy link
Collaborator

flinkbot commented Jun 5, 2024

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Copy link
Contributor

@lincoln-lil lincoln-lil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xuyangzhong Thanks for fixing this! Overall looks good to me, I've left some comments, PTAL.

@xuyangzhong
Copy link
Contributor Author

Hi, @lincoln-lil I have updated this pr to resolve your comments and rebase the master. I would be grateful if you have time to review it again.

Copy link
Contributor

@lincoln-lil lincoln-lil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xuyangzhong Thanks for your updates! LGTM +1
Could you also create backport pr to both 1.20 & 1.19 branches as this affects the availability of some UDFs?

@lincoln-lil lincoln-lil merged commit 84165b2 into apache:master Jul 2, 2024
xuyangzhong added a commit to xuyangzhong/flink that referenced this pull request Jul 4, 2024
…acting method parameter names from UDF

This closes apache#24890

(cherry picked from commit 84165b2)
xuyangzhong added a commit to xuyangzhong/flink that referenced this pull request Jul 4, 2024
…acting method parameter names from UDF

This closes apache#24890

(cherry picked from commit 84165b2)
superdiaodiao pushed a commit to superdiaodiao/flink that referenced this pull request Jul 4, 2024
snuyanzin pushed a commit to snuyanzin/flink that referenced this pull request Jul 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants