`atypes` not defined in python `inline`

[Gremious]

Problem

beet fields lists albumtype, albumtypes, and atypes as available tags.

albumtype and albumtypes, are well and usable within the inline python script, however, atypes is not, and crashes with:

NameError: name 'atypes' is not defined

example:

album_fields:
   test: |
     import platform
     log = open("log.txt", "a")
     log.write("type:" + albumtype + '\n') // ok
     log.write("types:" + albumtypes + '\n') // ok
     log.write("atypes:" + atypes + '\n') // NameError: name 'atypes' is not defined

I have tried using it first e.g.

paths:
  default: ${atypes}${test}/...

just to see if it makes a difference, but no go.

${atypes} itself in path does work completely fine.

I don't know if this is intentional, but it would be nice to be able to use the renamed values, e.g. compilation: Anthology / dj-mix: DJ-Mix inline without doing a match.

Setup

• OS: Debian
  beets version 1.6.0
  Python version 3.9.2
  plugins: albumtypes, bandcamp, chroma, copyartifacts, discogs, edit, fromfilename, inline, lastgenre, the
• Turning off plugins made problem go away (yes/no): N/A

My configuration (output of beet config) is:

Click to expand

directory: ~/data/user/gremious/public/beets_music
library: ~/data/user/gremious/public/beets_music/library.db

plugins: albumtypes fromfilename inline discogs chroma copyartifacts lastgenre bandcamp edit the
languages: en
per_disc_numbering: yes
chroma:
    auto: yes
copyartifacts:
    print_ignored: yes
    extensions: .*
lastgenre:
    count: 99
    force: no
    whitelist: yes
    min_weight: 10
    fallback:
    canonical: no
    source: album
    auto: yes
    separator: ', '
    prefer_specific: no
    title_case: yes
albumtypes:
    types:
    -   album: Album
    -   ep: EP
    -   single: Single
    -   soundtrack: OST
    -   remix: Remix
    -   live: Live
    -   mixtape: Mixtape
    -   compilation: Compilation
    -   dj-mix: DJ-Mix
    -   demo: Demo
    -   other: Other
    ignore_va: ''
    bracket: '  '

album_fields:
    test: "import platform\nlog = open(\"log.txt\", \"a\")\nlog.write(\"type:\" + albumtype + '\\n')\nlog.write(\"types:\" + albumtypes + '\\n')\nlog.write(\"atypes:\" + atypes + '\\n')\n\nret = \"\"\ntypes = albumtypes.split(' ')\n\nif len(types) <= 1:\n  return albumtypes\n\n# if types[0] == \"Album\":\n    # types.pop(0)\n    # return types\n# else:\n    # return albumtypes\n"

paths:
    default: ${albumartist}/${atypes}$album/${track}. $title
    comp: ${test}/${track}. $title
    singleton: Misc Tracks/${artist} - ${title}
    albumtype:soundtrack: Soundtracks/${albumartist} - ${album}/${track}. $artist - $title
discogs:
    apikey: REDACTED
    apisecret: REDACTED
    tokenfile: discogs_token.json
    source_weight: 0.5
    user_token: REDACTED
    separator: ', '
    index_tracks: no
edit:
    albumfields: album albumartist
    itemfields: track title artist album
    ignore_fields: id path
the:
    the: yes
    a: yes
    format: '{0}, {1}'
    strip: no
    patterns: []
bandcamp:
    include_digital_only_tracks: yes
    search_max: 2
    art: no
    exclude_extra_fields: []
    genre:
        capitalize: no
        maximum: 0
        mode: progressive
        always_include: []
    comments_separator: '

        ---

        '
pathfields: {}
item_fields: {}

[sampsyo]

Thanks for reporting!

For a little additional context, the atypes field comes from the albumtypes plugin:
https://beets.readthedocs.io/en/stable/plugins/albumtypes.html

That field is a computed field, i.e., it runs a function every time it's looked up. The underlying problem here is that the inline plugin doesn't expose these computed fields. One reason we don't currently do this is that inline fields themselves are computed—and trying to eagerly compute them to provide them to other inline fields can easily cause infinite loops.

If you're curious about where this happens in the code, here's where we get the list of fields in the inline plugin:

beets/beetsplug/inline.py

Line 101 in 36454a3

out = dict(obj)

And here is where the computed fields are excluded by default:

beets/beets/dbcore/db.py

Lines 492 to 501 in 36454a3

	def keys(self, computed: bool = False):
	"""Get a list of available field names for this object. The
	`computed` parameter controls whether computed (plugin-provided)
	fields are included in the key list.
	"""
	base_keys = list(self._fields) + list(self._values_flex.keys())
	if computed:
	return base_keys + list(self._getters().keys())
	else:
	return base_keys

It's possible we could think of a brilliant way to resolve this and expose fields like atypes to the inline plugin, but it will require some kind of creativity… I don't see an obvious route forward at the moment.

[Gremious]

Ah, yeah, I see the problem.
I'm gonna make some assumptions so forgive me if I ask nonsense.

I think I understand the "get data -> compile -> evaluate" flow. But would you kindly elaborate on exactly where does dbcore::db::keys connect with inline to provide the fn the keys? It's a little hard for me to read this, as obj seems to come from a mystery place with no obvious call to _expr_func.

My naive thought would be getting computed fields, but filtering out all inline keys, e.g. eagerly evaluating all computed fields but only those from other plugins. Though I don't know if that's easily doable in this setup?

---

Also, how are inline fields are evaluated, anyway? Hard to phrase, but like, assuming this setup:

item_fields:
  foo: "cool"
  bar: "good"

If one were to fetch and evaluate all computed fields while parsing the python for bar - would foo already be in that list? I assume that this is true based on the problem, but, if so, is it possible for the update to the fields list to be delayed until after all inline fields are computed - that way they just do not see each other.

[sampsyo]

If I understand the first question: here, obj is an Album or Item object (i.e., a subclass of dbcore.Model). Calling dict(obj) iterates over the keys of obj, i.e., it calls something like obj.__iter__(), which in turn invokes iter(self.keys()) as listed there. I admit this is kind of convoluted!

The way inline fields work in general is that they attach custom "getter" functions to the model classes. That is, it doesn't work this way:

1. Compute the value for foo, which is "cool". Assign the foo field of the object to "cool".
2. Do the same for bar, producing the string "good" and putting it in the bar field.

Instead, foo and bar correspond to functions that, on every lookup of item.foo or item.bar, execute code to compute results on the fly. This is important so that computed fields like this can get "fresh" values, especially when they are based on other fields that can be updated.

This registration mechanism happens here, FWIW:

beets/beets/dbcore/db.py

Line 328 in 2032729

def _getters(cls: Type["Model"]):

[stale]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.