Skip to content

Conversation

@lcmen
Copy link
Contributor

@lcmen lcmen commented Feb 6, 2025

This PR kicks start the work I suggested in proposal to add field names to identities.

Please keep in mind the feature is not complete yet and it also relies on adding field_names option to Ash.Resource.Identity that I plan to address after this one is completed.

Once the overall direction is approved, I will work on tests to cover this new functionality.

Please let me know what you think!

PS. It's my first constribution to any Elixir open source project. I only use Elixir in my spare time so please forgive all my mistakes. I treat is as an opportunity to learn more about Elixir and Ash ecosystem.

Contributor checklist

  • Bug fixes include regression tests
  • Features include unit/acceptance tests

]
)
identity = Enum.find(identities, fn identity ->
String.contains?(constraint, to_string(identity.name))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably not the right approach to match violated constraint with an identity but it was the quickest way to test this feature in my local env.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have some logic elsewhere to get the name for a given identity, so we should reverse engineer it here. I believe we do this in the migration generator and elsewhere when we actually add in the constraint. So we can search the identities for a match on that name instead of just containing the identity name 😄

@lcmen lcmen force-pushed the add-support-for-field-names branch from ab8b966 to 7486066 Compare February 6, 2025 20:20
@lcmen
Copy link
Contributor Author

lcmen commented Feb 7, 2025

@zachdaniel please let me know if this is what you meant by:

We would then need to adjust code in ash_postgres that raises these errors to consider those fields.

In your original response. Thanks!

@zachdaniel
Copy link
Contributor

Hey there! Haven't forgotten about you, will review soon, hopefully today.

@zachdaniel
Copy link
Contributor

Looks great! As you pointed out, the main issue is how we get the relevant identity. It should be doable, search in the data_layer.ex file for where we add identities to the Ecto changeset to see how we determine identity names, and use that to match identities to the constraint name.

@lcmen lcmen force-pushed the add-support-for-field-names branch from 7486066 to f137692 Compare February 9, 2025 19:29
@lcmen
Copy link
Contributor Author

lcmen commented Feb 9, 2025

Looks great! As you pointed out, the main issue is how we get the relevant identity. It should be doable, search in the data_layer.ex file for where we add identities to the Ecto changeset to see how we determine identity names, and use that to match identities to the constraint name.

Thank you! I followed your suggestions and I've updated the logic to match identity to constraint.

I would like to work on tests now but I have problems running tests for the main branches. I tried to run them with main branches like this

ASH_SQL_VERSION=main ASH_VERSION=main mix deps.get
ASH_SQL_VERSION=main ASH_VERSION=main mix test test/calculation_test.exs

but I have many failures across multiple files, as an example here is the output from test/calculations_test.exs:

==> ash
Compiling 507 files (.ex)
Generated ash app
==> ash_postgres
Compiling 87 files (.ex)
Generated ash_postgres app
Running ExUnit with seed: 249364, max_cases: 16

.............

  1) test calculations that refer to aggregates can be authorized (AshPostgres.CalculationTest)
     test/calculation_test.exs:456
     match (=) failed
     code:  assert %{has_future_comment: true} =
              Post
              |> Ash.Query.load([:has_future_comment, :latest_comment_created_at])
              |> Ash.Query.for_read(:allow_any, %{})
              |> Ash.read_one!(authorize?: false)
     left:  %{has_future_comment: true}
     right: %AshPostgres.Test.Post{
              has_future_comment: false,
              __lateral_join_source__: nil,
              __meta__: #Ecto.Schema.Metadata<:loaded, "posts">,
              __metadata__: %{
                selected: [
                  :uniq_custom_two,
                  :category,
                  :point,
                  :price,
                  :constrained_int,
                  :uniq_if_contains_foo,
                  :version,
                  :updated_at,
                  :status_enum_no_cast,
                  :author_id,
                  :list_of_stuff,
                  :limited_score,
                  :datetime,
                  :status_enum,
                  :uniq_one,
                  :status,
                  :uniq_on_upper,
                  :organization_id,
                  :uniq_two,
                  :ltree_unescaped,
                  :score,
                  :ltree_escaped,
                  :id,
                  :stuff,
                  :title,
                  :decimal,
                  :public,
                  :type,
                  :created_at,
                  :uniq_custom_one,
                  :composite_point,
                  :list_containing_nils,
                  :parent_post_id
                ],
                keyset: "g2o="
              },
              __order__: nil,
              active_followers: #Ash.NotLoaded<:relationship, field: :active_followers>,
              active_followers_assoc: #Ash.NotLoaded<:relationship, field: :active_followers_assoc>,
              agg_map: #Ash.NotLoaded<:calculation, field: :agg_map>,
              aggregates: %{},
              author: #Ash.NotLoaded<:relationship, field: :author>,
              author_count_of_posts: #Ash.NotLoaded<:calculation, field: :author_count_of_posts>,
              author_count_of_posts_agg: #Ash.NotLoaded<:calculation, field: :author_count_of_posts_agg>,
              author_first_name: #Ash.NotLoaded<:aggregate, field: :author_first_name>,
              author_first_name_calc: #Ash.NotLoaded<:calculation, field: :author_first_name_calc>,
              author_first_name_ref_agg_calc: #Ash.NotLoaded<:calculation, field: :author_first_name_ref_agg_calc>,
              author_has_post_with_follower_named_fred: #Ash.NotLoaded<:calculation, field: :author_has_post_with_follower_named_fred>,
              author_id: nil,
              author_profile_description: #Ash.NotLoaded<:aggregate, field: :author_profile_description>,
              author_profile_description_from_agg: #Ash.NotLoaded<:calculation, field: :author_profile_description_from_agg>,
              avg_comment_rating: #Ash.NotLoaded<:aggregate, field: :avg_comment_rating>,
              c_times_p: #Ash.NotLoaded<:calculation, field: :c_times_p>,
              calc_returning_json: #Ash.NotLoaded<:calculation, field: :calc_returning_json>,
              calculations: %{},
              category: nil,
              category_label: #Ash.NotLoaded<:calculation, field: :category_label>,
              co_author_posts: #Ash.NotLoaded<:relationship, field: :co_author_posts>,
              co_authors: #Ash.NotLoaded<:relationship, field: :co_authors>,
              co_authors_unfiltered: #Ash.NotLoaded<:relationship, field: :co_authors_unfiltered>,
              comment_authors: #Ash.NotLoaded<:aggregate, field: :comment_authors>,
              comment_title: #Ash.NotLoaded<:calculation, field: :comment_title>,
              comment_titles: #Ash.NotLoaded<:aggregate, field: :comment_titles>,
              comment_titles_with_5_likes: #Ash.NotLoaded<:aggregate, field: :comment_titles_with_5_likes>,
              comment_titles_with_nils: #Ash.NotLoaded<:aggregate, field: :comment_titles_with_nils>,
              comments: #Ash.NotLoaded<:relationship, field: :comments>,
              comments_containing_title: #Ash.NotLoaded<:relationship, field: :comments_containing_title>,
              comments_matching_post_title: #Ash.NotLoaded<:relationship, field: :comments_matching_post_title>,
              comments_with_high_rating: #Ash.NotLoaded<:relationship, field: :comments_with_high_rating>,
              composite_origin: #Ash.NotLoaded<:calculation, field: :composite_origin>,
              composite_point: nil,
              constrained_int: 2,
              count_comment_titles: #Ash.NotLoaded<:aggregate, field: :count_comment_titles>,
              count_of_comment_ratings: #Ash.NotLoaded<:aggregate, field: :count_of_comment_ratings>,
              count_of_comments: #Ash.NotLoaded<:aggregate, field: :count_of_comments>,
              count_of_comments_called_baz: #Ash.NotLoaded<:calculation, field: :count_of_comments_called_baz>,
              count_of_comments_called_match: #Ash.NotLoaded<:aggregate, field: :count_of_comments_called_match>,
              count_of_comments_containing_title: #Ash.NotLoaded<:aggregate, field: :count_of_comments_containing_title>,
              count_of_comments_that_have_a_post: #Ash.NotLoaded<:aggregate, field: :count_of_comments_that_have_a_post>,
              count_of_comments_that_have_a_post_with_exists: #Ash.NotLoaded<:aggregate, field: :count_of_comments_that_have_a_post_with_exists>,
              count_of_linked_posts: #Ash.NotLoaded<:aggregate, field: :count_of_linked_posts>,
              count_of_popular_comment_ratings: #Ash.NotLoaded<:aggregate, field: :count_of_popular_comment_ratings>,
              count_of_popular_comments: #Ash.NotLoaded<:aggregate, field: :count_of_popular_comments>,
              count_of_ratings: #Ash.NotLoaded<:aggregate, field: :count_of_ratings>,
              count_of_recent_popular_comments: #Ash.NotLoaded<:aggregate, field: :count_of_recent_popular_comments>,
              count_uniq_comment_titles: #Ash.NotLoaded<:aggregate, field: :count_uniq_comment_titles>,
              created_at: ~U[2025-02-09 19:32:43.786445Z],
              current_user_author: #Ash.NotLoaded<:relationship, field: :current_user_author>,
              datetime: nil,
              decimal: Decimal.new("0"),
              first_3_followers: #Ash.NotLoaded<:relationship, field: :first_3_followers>,
              first_comment: #Ash.NotLoaded<:aggregate, field: :first_comment>,
              first_comment_nils_first: #Ash.NotLoaded<:aggregate, field: :first_comment_nils_first>,
              first_comment_nils_first_called_stuff: #Ash.NotLoaded<:aggregate, field: :first_comment_nils_first_called_stuff>,
              first_comment_nils_first_include_nil: #Ash.NotLoaded<:aggregate, field: :first_comment_nils_first_include_nil>,
              first_three_followers_assoc: #Ash.NotLoaded<:relationship, field: :first_three_followers_assoc>,
              followers: #Ash.NotLoaded<:relationship, field: :followers>,
              followers_join_assoc: #Ash.NotLoaded<:relationship, field: :followers_join_assoc>,
              foo_bar_from_stuff: #Ash.NotLoaded<:calculation, field: :foo_bar_from_stuff>,
              has_author: #Ash.NotLoaded<:calculation, field: :has_author>,
              has_comment_called_match: #Ash.NotLoaded<:aggregate, field: :has_comment_called_match>,
              has_comments: #Ash.NotLoaded<:calculation, field: :has_comments>,
              has_follower_named_fred: #Ash.NotLoaded<:calculation, field: :has_follower_named_fred>,
              has_future_arbitrary_timestamp: #Ash.NotLoaded<:calculation, field: :has_future_arbitrary_timestamp>,
              has_no_followers: #Ash.NotLoaded<:calculation, field: :has_no_followers>,
              high_ratings: #Ash.NotLoaded<:relationship, field: :high_ratings>,
              highest_comment_rating: #Ash.NotLoaded<:aggregate, field: :highest_comment_rating>,
              highest_rating: #Ash.NotLoaded<:aggregate, field: :highest_rating>,
              id: "5a1cad97-2c50-4f97-b260-618623942484",
              last_comment: #Ash.NotLoaded<:aggregate, field: :last_comment>,
              latest_arbitrary_timestamp: #Ash.NotLoaded<:aggregate, field: :latest_arbitrary_timestamp>,
              latest_comment: #Ash.NotLoaded<:relationship, field: :latest_comment>,
              latest_comment_created_at: ~U[2025-02-09 19:32:43.787908Z],
              limited_score: nil,
              linked_multitenant_posts: #Ash.NotLoaded<:relationship, field: :linked_multitenant_posts>,
              linked_multitenant_posts_join_assoc: #Ash.NotLoaded<:relationship, field: :linked_multitenant_posts_join_assoc>,
              linked_posts: #Ash.NotLoaded<:relationship, field: :linked_posts>,
              list_containing_nils: nil,
              list_of_stuff: nil,
              lowest_comment_rating: #Ash.NotLoaded<:aggregate, field: :lowest_comment_rating>,
              ltree_escaped: nil,
              ltree_unescaped: nil,
              max_comment_similarity: #Ash.NotLoaded<:calculation, field: :max_comment_similarity>,
              negative_score: #Ash.NotLoaded<:calculation, field: :negative_score>,
              not_selected_by_default: #Ash.NotLoaded<:attribute, field: :not_selected_by_default>,
              organization: #Ash.NotLoaded<:relationship, field: :organization>,
              organization_id: nil,
              parent_post: #Ash.NotLoaded<:relationship, field: :parent_post>,
              parent_post_id: nil,
              permalinks: #Ash.NotLoaded<:relationship, field: :permalinks>,
              point: nil,
              popular_comments: #Ash.NotLoaded<:relationship, field: :popular_comments>,
              post_followers: #Ash.NotLoaded<:relationship, field: :post_followers>,
              post_links: #Ash.NotLoaded<:relationship, field: :post_links>,
              posts_with_matching_title: #Ash.NotLoaded<:relationship, field: :posts_with_matching_title>,
              price: nil,
              price_string: #Ash.NotLoaded<:calculation, field: :price_string>,
              price_string_with_currency_sign: #Ash.NotLoaded<:calculation, field: :price_string_with_currency_sign>,
              price_times_2: #Ash.NotLoaded<:calculation, field: :price_times_2>,
              public: nil,
              query: #Ash.NotLoaded<:calculation, field: :query>,
              ratings: #Ash.NotLoaded<:relationship, field: :ratings>,
              score: nil,
              score_after_winning: #Ash.NotLoaded<:calculation, field: :score_after_winning>,
              score_map: #Ash.NotLoaded<:calculation, field: :score_map>,
              score_plus: #Ash.NotLoaded<:calculation, field: :score_plus>,
              score_with_score: #Ash.NotLoaded<:calculation, field: :score_with_score>,
              similarity: #Ash.NotLoaded<:calculation, field: :similarity>,
              sorted_followers: #Ash.NotLoaded<:relationship, field: :sorted_followers>,
              start_of_day: #Ash.NotLoaded<:calculation, field: :start_of_day>,
              stateful_followers: #Ash.NotLoaded<:relationship, field: :stateful_followers>,
              stateful_followers_join_assoc: #Ash.NotLoaded<:relationship, field: :stateful_followers_join_assoc>,
              status: nil,
              status_enum: nil,
              status_enum_no_cast: nil,
              stuff: nil,
              sum_of_author_count_of_posts: #Ash.NotLoaded<:calculation, field: :sum_of_author_count_of_posts>,
              sum_of_comment_likes: #Ash.NotLoaded<:aggregate, field: :sum_of_comment_likes>,
              sum_of_comment_likes_called_match: #Ash.NotLoaded<:aggregate, field: :sum_of_comment_likes_called_match>,
              sum_of_comment_likes_with_default: #Ash.NotLoaded<:aggregate, field: :sum_of_comment_likes_with_default>,
              sum_of_comment_ratings_calc: #Ash.NotLoaded<:aggregate, field: :sum_of_comment_ratings_calc>,
              sum_of_popular_comment_rating_scores: #Ash.NotLoaded<:aggregate, field: :sum_of_popular_comment_rating_scores>,
              sum_of_popular_comment_rating_scores_2: #Ash.NotLoaded<:aggregate, field: :sum_of_popular_comment_rating_scores_2>,
              sum_of_recent_popular_comment_likes: #Ash.NotLoaded<:aggregate, field: :sum_of_recent_popular_comment_likes>,
              ten_most_popular_comments: #Ash.NotLoaded<:aggregate, field: :ten_most_popular_comments>,
              title: "title",
              title_twice: #Ash.NotLoaded<:calculation, field: :title_twice>,
              title_twice_with_calc: #Ash.NotLoaded<:calculation, field: :title_twice_with_calc>,
              type: :sponsored,
              uniq_comment_titles: #Ash.NotLoaded<:aggregate, field: :uniq_comment_titles>,
              uniq_custom_one: nil,
              uniq_custom_two: nil,
              uniq_if_contains_foo: nil,
              uniq_on_upper: nil,
              uniq_one: nil,
              uniq_two: nil,
              updated_at: ~U[2025-02-09 19:32:43.786445Z],
              upper_thing: #Ash.NotLoaded<:calculation, field: :upper_thing>,
              upper_title: #Ash.NotLoaded<:calculation, field: :upper_title>,
              version: 1,
              views: #Ash.NotLoaded<:relationship, field: :views>,
              was_created_in_the_last_month: #Ash.NotLoaded<:calculation, field: :was_created_in_the_last_month>
            }
     stacktrace:
       test/calculation_test.exs:473: (test)

...................
20:32:43.848 [debug] QUERY OK db=0.0ms
SELECT timezone('UTC', date_trunc('day', '2025-02-09 19:32:43.846949Z' AT TIME ZONE 'UTC' AT TIME ZONE 'EST')) ::timestamp FROM (VALUES(1)) AS f0


  2) test start_of_day functions the same as Elixir's start of day (AshPostgres.CalculationTest)
     test/calculation_test.exs:54
     Assertion with == failed
     code:  assert Ash.calculate!(Post, :start_of_day, data_layer?: true) ==
              Ash.Expr.eval!(Ash.Expr.expr(start_of_day(now(), "EST")))
     left:  ~U[2025-02-09 23:00:00Z]
     right: ~U[2025-02-09 05:00:00Z]
     stacktrace:
       test/calculation_test.exs:57: (test)

.............
Finished in 0.6 seconds (0.00s async, 0.6s sync)
47 tests, 2 failures

Are those failures related to my setup? Thanks!


"main" ->
[git: "https://github.com/ash-project/ash.git"]
[git: "https://github.com/ash-project/ash.git", override: true]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I don't add override, then I'm not able to use main option for ASH_VERSION variable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, I haven't had that problem but adding that should fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without override: true, I was getting a following error:

Dependencies have diverged:
* ash (https://github.com/ash-project/ash.git)
  the dependency ash in mix.exs is overriding a child dependency:

  > In mix.exs:
    {:ash, [env: :prod, git: "https://github.com/ash-project/ash.git"]}

  > In deps/ash_sql/mix.exs:
    {:ash, "~> 3.0 and >= 3.4.60", [env: :prod, hex: "ash", repo: "hexpm"]}

  Ensure they match or specify one of the above in your deps and set "override: true"
** (Mix) Can't continue due to errors on dependencies

identities do
identity(:uniq_one_and_two, [:uniq_one, :uniq_two])
identity(:uniq_on_upper, [:upper_thing])
identity(:uniq_on_upper_title, [:upper_title], field_names: [:title])
Copy link
Contributor Author

@lcmen lcmen Feb 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to use this identity to test field_names option.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided to use Organization resource as it's not heavily used in other tests. When I added a new constraint to the Post resource, some other tests started to fail.

@zachdaniel
Copy link
Contributor

zachdaniel commented Feb 10, 2025

Don't worry about those tests 😄 They are related to other things that have been fixed elsewhere.

@lcmen lcmen marked this pull request as ready for review February 10, 2025 19:24
@lcmen
Copy link
Contributor Author

lcmen commented Feb 10, 2025

It depends on ash-project/ash#1786

@lcmen lcmen force-pushed the add-support-for-field-names branch from a16e7d5 to 0344284 Compare February 13, 2025 17:42
@lcmen
Copy link
Contributor Author

lcmen commented Feb 13, 2025

Dependent PR has been merged. @zachdaniel can you give it a final review, please?

@zachdaniel
Copy link
Contributor

Looks great, however ash_postgres needs to now explicitly depend on the version of ash that introduced this new feature.

@lcmen
Copy link
Contributor Author

lcmen commented Feb 13, 2025

Looks great, however ash_postgres needs to now explicitly depend on the version of ash that introduced this new feature.

Thank you! Just to make sure I understand. Now I need to wait for new ash release (either 3.4.64 or 3.5.0) and update mix.exs for ash_postgres to require ash with version at least3.4.64?

@zachdaniel
Copy link
Contributor

Yeah we just need to pin to at least 3.4.64

@lcmen
Copy link
Contributor Author

lcmen commented Feb 17, 2025

Yeah we just need to pin to at least 3.4.64

Done. I believe now they can be released together.

@zachdaniel zachdaniel merged commit 64d768c into ash-project:main Feb 17, 2025
@zachdaniel
Copy link
Contributor

🚀 Thank you for your contribution! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants