Skip to content

Improve precision of the remove_task_decorator function#34509

Closed
SamWheating wants to merge 2 commits intoapache:mainfrom
SamWheating:sw-fix-code-modification-pythonoperator
Closed

Improve precision of the remove_task_decorator function#34509
SamWheating wants to merge 2 commits intoapache:mainfrom
SamWheating:sw-fix-code-modification-pythonoperator

Conversation

@SamWheating
Copy link
Contributor

@SamWheating SamWheating commented Sep 20, 2023

closes: #34154

The remove_task_decorator can incorrectly truncate function bodies if there are specific comments in the function body present, which is a rare edge case but can lead to really misleading behaviour where only a part of the intended code is executed.

I think that the correct way to do code manipulation like this would be to lean on the ast library, but the ability to go from ast back to code was only added in python 3.9 with the ast.unparse() function.

Anyways, this PR is an attempt to improve the accuracy of this method. While I don't think that this is perfect, it at least solves the issue for the cases in the linked issue with the following changes:

  • only early exiting if a line starts with the decorator name
  • splitting the code at most one time

Any suggestions welcome.

If you want to discuss this IRL, come find me at Airflow Summit :)

@SamWheating SamWheating force-pushed the sw-fix-code-modification-pythonoperator branch from e0084e5 to 0a45403 Compare September 28, 2023 00:03
@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Nov 12, 2023
@github-actions github-actions bot closed this Nov 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale Stale PRs per the .github/workflows/stale.yml policy file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

"@task.virtualenv" cannot appear in a comment inside a @task.virtualenv task

1 participant