add virtual fields list to schema reflection

[azhi]

Schema.__schema__(:virtual_fields) should return list of all virtual
fields defined in Schema

As far as I can see, most of the other schema reflection data is used within ecto. List of virtual fields is not used within ecto, but it might be useful for users of ecto (at least it would be useful for me :)).

[josevalim]

Can you please expand in the use case? You can always get the list of all fields by simply looking at the struct. So if it is not a field but it is in the struct, it is by consequence a virtual field. -- *José Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D

[azhi]

Yeah, there is a way to solve this outside of Ecto, but minimal working implementation for test schema looks like this:
((Schema.__struct__ |> Map.delete(:__meta__) |> Map.delete(:__struct__) |> Map.keys()) -- Schema.__schema__(:fields)) -- Schema.__schema__(:associations)

It looks like you should be able just to filter out fields and get a list of virtual fields, but in practice you need to deal with stuff like __meta__, __struct__, associations, embeds - and it ends up not looking good.
Another argument is that a list of things in struct you need to filter out depends heavily on Ecto. Right now it's __meta__, __struct__, associations and embeds (if I didn't miss anything), but if Ecto introduces new hypothetical field type that isn't DB field or one of the above - suggested code will break and consider it a virtual field.
Having this inside Ecto will add a nice explicit way of retrieving actual list of virtual fields.

As about broader use case without going in too much detail - I'm writing graphql API in elixir using absinthe, and want to write a middleware that would resolve virtual and non-virtual fields differently.

[josevalim]

Do you need to have a list of all fields upfront or do you need to know if a field is a virtual field or not for a given struct? Those are slightly different. Another way to phrase this is: do you need to know this list at compile time or runtime?

[azhi]

For my specific use case runtime is good enough - it will have the cost of doing this check for every field, but it shouldn't be a big problem. If, however, this info is available at compile time, I can use it at compile time and optimize that way (absinthe has a nice feature that allows easily moving this stuff to compile time).

[josevalim]

For compile time, you can then use what you proposed. I wouldn’t worry about the underscores fields per se, simply consider anything double (or single) underscores as private. For runtime, you can make it faster simply by checking if the fields is on the map, and it is not a field nor an association. Given there are solvable ways to address this, we won’t add a new Ecto API for now (and a new Ecto release is likely several months away anyway). In any case, thanks for the proposal! -- *José Valimwww.plataformatec.com.br <http://www.plataformatec.com.br/>Founder and Director of R&D*

[v0idpwn]

Could we add this now? I'm on need. I'm willing to implement it if you think it is a good idea.

I'm working with a library that has an abstraction over embedded schemas + changesets. It relies on reflection in runtime and it would benefit from reading virtual fields.

[josevalim]

Is the __changeset__ function not enough?

[v0idpwn]

It works! I thought it was private API.