Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Status of testing of Apache Airflow 2.7.2rc1 #34830

Closed
43 of 99 tasks
ephraimbuddy opened this issue Oct 9, 2023 · 15 comments
Closed
43 of 99 tasks

Status of testing of Apache Airflow 2.7.2rc1 #34830

ephraimbuddy opened this issue Oct 9, 2023 · 15 comments
Labels
kind:meta High-level information important to the community testing status Status of testing releases

Comments

@ephraimbuddy
Copy link
Contributor

ephraimbuddy commented Oct 9, 2023 •
edited by hussein-awala
Loading

Body

We are kindly requesting that contributors to Apache Airflow RC 2.7.2rc1 help test the RC.

Please let us know by commenting if the issue is addressed in the latest RC.

Thanks to all who contributed to the release (probably not a complete list!):
@dkalamar @jedcunningham @Raul824 @yiannis-had @Xiroo @erdos2n @drobert-bfm @dstandish @ephraimbuddy @Usiel @potiuk @hussein-awala @eumiro @mbarugelCA @jens-scheffler-bosch @utkarsharma2 @yermalov-here @Taragolis @Abhishek-kumar-samsung @csp33 @pankajkoti @mj-dd @gbonazzoli @fritz-astronomer @sa1 @ldacey @SamWheating @risicle @Calder-Ty @Bisk1 @bolkedebruin @uranusjr @alartum @Jayden-Chiu

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.
@ephraimbuddy ephraimbuddy added the kind:meta High-level information important to the community label Oct 9, 2023
@jedcunningham
Copy link
Member

#34011 and #34636 both should have been skipped - providers and dev env respectively.

@ephraimbuddy
Copy link
Contributor Author

#34011 and #34636 both should have been skipped - providers and dev env respectively.

Mistake

@Calder-Ty
Copy link
Contributor

#34568 Is working as expected

@utkarsharma2
Copy link
Contributor

utkarsharma2 commented Oct 10, 2023 via email

@Xiroo
Copy link
Contributor

Xiroo commented Oct 10, 2023

#34359 is working as expected.

@Bisk1
Copy link
Contributor

Bisk1 commented Oct 11, 2023

#33746, #33893 and #34027 work as expected

@Usiel
Copy link
Contributor

Usiel commented Oct 11, 2023 •
edited
Loading

#34072 and #34136 work as expected.

I also tested a few other PRs:

Details on #34120 issue (cc @hussein-awala)

Tested both

  • locally with Breeze (Posgres 11, Python 3.8, git rev c8b25cb3ee aka 2.7.2rc1 tag) and
  • on a test environment (MySQL 5.7, Python 3.9, PyPi package 2.7.2rc1).

In both cases, for different DAGs, I get the error below. #34656 seems to have broken this for me.

[2023-10-11T02:22:36.229+0000] {cli_action_loggers.py:85} DEBUG - Calling callbacks: []
Traceback (most recent call last):
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/bin/airflow", line 8, in <module>
    sys.exit(main())
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/airflow/__main__.py", line 60, in main
    args.func(args)
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/airflow/cli/cli_config.py", line 49, in command
    return func(*args, **kwargs)
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/airflow/utils/cli.py", line 113, in wrapper
    return f(*args, **kwargs)
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 634, in task_test
    session.delete(ti.dag_run)
  File "/opt/.pyenv/versions/3.9.8/lib/python3.9/contextlib.py", line 126, in __exit__
    next(self.gen)
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/airflow/utils/session.py", line 37, in create_session
    session.commit()
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1454, in commit
    self._transaction.commit(_to_root=self.future)
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 832, in commit
    self._prepare_impl()
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 811, in _prepare_impl
    self.session.flush()
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3449, in flush
    self._flush(objects)
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3589, in _flush
    transaction.rollback(_capture_exception=True)
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    compat.raise_(
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3549, in _flush
    flush_context.execute()
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
    rec.execute(self)
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 667, in execute
    util.preloaded.orm_persistence.delete_obj(
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 330, in delete_obj
    table_to_mapper = base_mapper._sorted_tables
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 1184, in __get__
    obj.__dict__[self.__name__] = result = self.fget(obj)
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/orm/mapper.py", line 3386, in _sorted_tables
    sorted_ = sql_util.sort_tables(
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/sql/ddl.py", line 1217, in sort_tables
    for (t, fkcs) in sort_tables_and_constraints(
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/sql/ddl.py", line 1289, in sort_tables_and_constraints
    filtered = filter_fn(fkc)
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/sql/ddl.py", line 1207, in _skip_fn
    if skip_fn(fk):
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/orm/mapper.py", line 3369, in skip
    dep = table_to_mapper.get(fk.column.table)
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 1113, in __get__
    obj.__dict__[self.__name__] = result = self.fget(obj)
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/sql/schema.py", line 2532, in column
    return self._resolve_column()
  File "/opt/.pyenv/versions/3.9.8/envs/airflow/lib/python3.9/site-packages/sqlalchemy/sql/schema.py", line 2543, in _resolve_column
    raise exc.NoReferencedTableError(
sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'dag_run_note.user_id' could not find table 'ab_user' with which to generate a foreign key to target column 'id'
[2023-10-11T02:22:36.234+0000] {settings.py:353} DEBUG - Disposing DB connection pool (PID 5345)

@ephraimbuddy
Copy link
Contributor Author

ephraimbuddy commented Oct 11, 2023 •
edited
Loading

#34072 and #34136 work as expected.

I also tested a few other PRs:

I don't think the failing one should be considered a blocker since no new issue was introduced. cc @dstandish @hussein-awala

@hussein-awala
Copy link
Member

#34072 and #34136 work as expected.
I also tested a few other PRs:

I don't think the failing one should be considered a blocker since no new issue was introduced. cc @dstandish @hussein-awala

Yes it should not considered as blocker, but if there is a different blocker for the RC (I hope not), we can cherry-pick #34656 in the next RC

@hussein-awala
Copy link
Member

I tested my changes, they are present in the RC. I tested only the bug fixes, and all looks good, except #34120 as discussed above.

@ephraimbuddy
Copy link
Contributor Author

#34072 and #34136 work as expected.
I also tested a few other PRs:

I don't think the failing one should be considered a blocker since no new issue was introduced. cc @dstandish @hussein-awala

Yes it should not considered as blocker, but if there is a different blocker for the RC (I hope not), we can cherry-pick #34656 in the next RC

#34656 is in the release, but not on the release note. You can see the milestone and changelog:skip label on it. I wanted to combine it with #34120 on the release note but forgot that.
We probably need another fix. Also, you can go to the v2-7-stable to confirm if the change is there

@jscheffl
Copy link
Contributor

Checked for #34248 and can confirm that (1) code is correctly merged and (2) fix is working! Thanks for making the release!

@ephraimbuddy
Copy link
Contributor Author

Airflow 2.7.2 has been released. Thank you all for testing the release candidate

@yermalov-here
Copy link
Contributor

#34274 works as expected

@hussein-awala hussein-awala mentioned this issue Oct 13, 2023
Open
2 tasks
@sa1
Copy link
Contributor

sa1 commented Oct 18, 2023

Only verified after release but #34531 is working.

@potiuk potiuk added the testing status Status of testing releases label Dec 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind:meta High-level information important to the community testing status Status of testing releases
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests