enum: _generate_next_value_ is not called if its definition occurs after calls to auto()

[edd07]

BPO	40025
Nosy	@warsaw, @ethanfurman, @srinivasreddy, @hongweipeng, @miss-islington, @edd07, @ethanonstott, @jonnyhsu, @ankeshsaha
PRs	bpo-40025: Require _generate_next_value_ to be defined before members #19098 [3.7] bpo-40025: Require _generate_next_value_ to be defined before members (GH-19098) #19762 [3.8] bpo-40025: Require _generate_next_value_ to be defined before members (GH-19098) #19763 bpo-40025:The behavior is the same when _generate_next_value_ is defined afte… #19904 bpo-40025: acknowledge Weipeng Hong's contributions #22284 bpo-40025: _auto_called cleanup #22285 [3.9] bpo-40025: _auto_called cleanup (GH-22285) #22287 [3.8] bpo-40025: _auto_called cleanup (GH-22285) #22286
Files	Issue40025.PNG: Screenshot of the code and output

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ethanfurman'
closed_at = <Date 2020-09-17.00:32:48.974>
created_at = <Date 2020-03-20.11:25:03.987>
labels = ['easy', 'type-bug', 'library', '3.10']
title = 'enum: _generate_next_value_ is not called if its definition occurs after calls to auto()'
updated_at = <Date 2020-09-17.00:32:48.972>
user = 'https://github.com/edd07'

bugs.python.org fields:

activity = <Date 2020-09-17.00:32:48.972>
actor = 'ethan.furman'
assignee = 'ethan.furman'
closed = True
closed_date = <Date 2020-09-17.00:32:48.974>
closer = 'ethan.furman'
components = ['Library (Lib)']
creation = <Date 2020-03-20.11:25:03.987>
creator = 'edd07'
dependencies = []
files = ['49014']
hgrepos = []
issue_num = 40025
keywords = ['patch', 'easy']
message_count = 13.0
messages = ['364665', '364707', '364776', '364784', '364967', '365321', '367538', '367547', '370127', '371324', '372002', '372010', '377032']
nosy_count = 11.0
nosy_names = ['barry', 'eli.bendersky', 'docs@python', 'ethan.furman', 'thatiparthy', 'hongweipeng', 'miss-islington', 'edd07', 'Ethan Onstott', 'Jonathan Hsu', 'ankeshsaha']
pr_nums = ['19098', '19762', '19763', '19904', '22284', '22285', '22287', '22286']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40025'
versions = ['Python 3.10']

[edd07]

I ran into this issue when attempting to add a custom _generate_next_value_ method to an existing Enum. Adding the method definition to the bottom of the class causes it to not be called at all:

from enum import Enum, auto

class E(Enum):
	A = auto()
	B = auto()
	def _generate_next_value_(name, *args):
		return name

E.B.value # Returns 2, E._generate_next_value_ is not called

class F(Enum):
	def _generate_next_value_(name, *args):
		return name
	A = auto()
	B = auto()
	
	
F.B.value  # Returns 'B', as intended

I do not believe that the order of method/attribute definition should affect the behavior of the class, or at least it should be mentioned in the documentation.

[ethanfurman]

Immediate solution is to raise an exception if _generate_next_value_ is defined after members.

Possible future solution is to save all member definitions until after class is defined.

The exception-raising solution would require a check in _EnumDict where _generate_next_value_ is saved -- if any members already exist, raise.

[jonnyhsu]

While the current behavior may be initially unexpected, it does match the way that python normally behaves when defining class variables. For example, the following class will throw an exception because the function number_two() is called before it is defined:

class Numbers:
    one = 1
    two = number_two()

    def number_two():
        return 2

# NameError: name 'number_two' is not defined

However, this version is fine:

class Numbers:
    one = 1

    def number_two():
        return 2

    two = number_two()

[ethanfurman]

Jonathan Hsu, you are correct -- and "don't do that" was my initial response; but Enum takes many pains to make sure the user doesn't shoot themselves in the foot, so raising a TypeError is appropriate instead of silently doing the wrong thing.

[jonnyhsu]

Thank you for the explanation.

[ankeshsaha]

Hi,

I have ran the code with with _generate_next_value_ method at the bottom of the class and didn't run into any exceptions. Please refer my python file. Please let me know if I am missing something.

from enum import Enum, auto

class E(Enum):
	A = auto()
	B = auto()
	def _generate_next_value_(name, *args):
		return name

for l in (E):
	print(l.name)
	print(l.value)

[ethanonstott]

Ankesh, that is the expected behavior as no patch has been merged yet.

[ethanfurman]

New changeset d9a43e2 by Ethan Onstott in branch 'master':
bpo-40025: Require _generate_next_value_ to be defined before members (GH-19098)
d9a43e2

[ethanfurman]

New changeset b5ecbf0 by Miss Islington (bot) in branch '3.8':
bpo-40025: Require _generate_next_value_ to be defined before members(GH-19763)
b5ecbf0

[ethanfurman]

New changeset ebd4400 by Miss Islington (bot) in branch '3.7':
bpo-40025: Require _generate_next_value_ to be defined before members (GH-19762)
ebd4400

[srinivasreddy]

We can close this?

[ethanfurman]

Not yet. I want to investigate the idea Ankesh Saha had some more.

[ethanfurman]

There was an effort to make it so _generate_next_value_ could be defined last and still work correctly -- unfortunately, it could not handle the more common case of using auto() with the default _generate_next_value_:

  class I(Enum):
      first = auto()
      second = first + 2    # this line would fail

Closing the ticket. Thank you everyone!