Fix and simplify overlays #3766

chrzaszcz · 2022-09-21T06:44:20Z

As stated in the rebar3 docs:

There should be only one overlay file. I checked with the docs and the code, it just supports one file now.
You can include one file from another. I checked that if a variable is declared more than once, the last value wins (https://github.com/erlware/relx/blob/main/src/rlx_overlay.erl#L149).

This is why we can just include the overlay files:

configure.vars.config into vars-toml.config
vars-toml.config into *.vars-toml.config

This way the overlays work correctly with the latest Rebar3, and there is no need to concatenate the files with make.
We have some templating utilities for tests, and they needed support for file inclusion as well.

codecov · 2022-09-21T12:28:34Z

Codecov Report

Base: 82.83% // Head: 17.87% // Decreases project coverage by -64.95% ⚠️

Coverage data is based on head (654e815) compared to base (2fcd129).
Patch has no changes to coverable lines.

❗ Current head 654e815 differs from pull request most recent head efee290. Consider uploading reports for the commit efee290 to get more accurate results

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #3766       +/-   ##
===========================================
- Coverage   82.83%   17.87%   -64.96%     
===========================================
  Files         531      531               
  Lines       34083    34066       -17     
===========================================
- Hits        28233     6090    -22143     
- Misses       5850    27976    +22126

Impacted Files	Coverage Δ
src/mam/mam_jid.erl	`0.00% <0.00%> (-100.00%)`	⬇️
src/mod_last_api.erl	`0.00% <0.00%> (-100.00%)`	⬇️
src/mam/mam_decoder.erl	`0.00% <0.00%> (-100.00%)`	⬇️
src/mam/mam_encoder.erl	`0.00% <0.00%> (-100.00%)`	⬇️
src/mam/mam_jid_rfc.erl	`0.00% <0.00%> (-100.00%)`	⬇️
src/mam/mam_message.erl	`0.00% <0.00%> (-100.00%)`	⬇️
src/gen_iq_component.erl	`0.00% <0.00%> (-100.00%)`	⬇️
src/ejabberd_c2s_state.erl	`0.00% <0.00%> (-100.00%)`	⬇️
src/mam/mod_mam_params.erl	`0.00% <0.00%> (-100.00%)`	⬇️
src/sasl/cyrsasl_oauth.erl	`0.00% <0.00%> (-100.00%)`	⬇️
... and 465 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

This way there is only one overlay file, and the overlays work correctly with the latest Rebar3. As we do some templating in our tools, they need to support file inclusion as well.

It is easier to modify vars-toml.config directly

mongoose-im · 2022-09-21T13:29:16Z

small_tests_24 / small_tests / efee290
Reports root / small

small_tests_25 / small_tests / efee290
Reports root / small

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / efee290
Reports root/ big
OK: 3826 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

ldap_mnesia_24 / ldap_mnesia / efee290
Reports root/ big
OK: 1992 / Failed: 0 / User-skipped: 742 / Auto-skipped: 0

ldap_mnesia_25 / ldap_mnesia / efee290
Reports root/ big
OK: 1992 / Failed: 0 / User-skipped: 742 / Auto-skipped: 0

dynamic_domains_mysql_redis_25 / mysql_redis / efee290
Reports root/ big
OK: 3800 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / efee290
Reports root/ big
OK: 3826 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / efee290
Reports root/ big
OK: 3826 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

internal_mnesia_25 / internal_mnesia / efee290
Reports root/ big
OK: 2113 / Failed: 0 / User-skipped: 621 / Auto-skipped: 0

elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / efee290
Reports root/ big
OK: 2452 / Failed: 1 / User-skipped: 616 / Auto-skipped: 0

bosh_SUITE:essential:accept_higher_hold_value

{error,
  {{assertEqual,
     [{module,bosh_SUITE},
      {line,251},
      {expression,"get_bosh_sessions ( )"},
      {expected,[]},
      {value,
        [{bosh_session,<<"c94942b635deb2b239aff10c32b1b797265469dc">>,
           <9159.5104.0>}]}]},
   [{bosh_SUITE,accept_higher_hold_value,1,
      [{file,"/home/circleci/project/big_tests/tests/bosh_SUITE.erl"},
       {line,251}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1291}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1223}]}]}}

Report log

pgsql_mnesia_24 / pgsql_mnesia / efee290
Reports root/ big
OK: 4200 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

pgsql_mnesia_25 / pgsql_mnesia / efee290
Reports root/ big
OK: 4200 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

mysql_redis_25 / mysql_redis / efee290
Reports root/ big
OK: 4186 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0

mssql_mnesia_25 / odbc_mssql_mnesia / efee290
Reports root/ big
OK: 4199 / Failed: 1 / User-skipped: 97 / Auto-skipped: 0

push_integration_SUITE:pubsub_ful:pm_notifications_with_inbox:inbox_msg_unread_count_apns

{error,
  {{assertMatch,
     [{module,push_integration_SUITE},
      {line,662},
      {expression,"Data"},
      {pattern,"# { << \"message-count\" >> := ExpectedCount }"},
      {value,
        #{<<"last-message-body">> => <<"Private message">>,
        <<"last-message-sender">> =>
          <<"kate_inbox_msg_unread_count_apns_2726@localhost">>,
        <<"message-count">> => 1}}]},
   [{push_integration_SUITE,check_notification,2,
      [{file,
         "/home/circleci/project/big_tests/tests/push_integration_SUITE.erl"},
       {line,662}]},
    {escalus_story,story,4,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,72}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1291}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1223}]}]}}

Report log

riak_mnesia_24 / riak_mnesia / efee290
Reports root/ big
OK: 2310 / Failed: 1 / User-skipped: 609 / Auto-skipped: 0

pubsub_SUITE:dag+basic:subscribe_options_test

{error,{{badmatch,{{<<"pubsub#deliver">>,<<"boolean">>,<<"false">>},
           <<"pubsub#deliver">>}},
    [{pubsub_tools,'-verify_form_values/2-fun-0-',2,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
            {line,650}]},
     {lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
     {pubsub_SUITE,'-subscribe_options_test/1-fun-1-',3,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_SUITE.erl"},
            {line,455}]},
     {escalus_story,story,4,
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
             {line,72}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1292}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1224}]}]}}

Report log

elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / efee290
Reports root/ big
OK: 2453 / Failed: 0 / User-skipped: 616 / Auto-skipped: 0

mssql_mnesia_25 / odbc_mssql_mnesia / efee290
Reports root/ big
OK: 4199 / Failed: 1 / User-skipped: 97 / Auto-skipped: 0

inbox_extensions_SUITE:async_pools:one_to_one:mute_muted_entry_gets_unmuted

{error,
  {{assert,
     [{module,inbox_extensions_SUITE},
      {line,798},
      {expression,"escalus_pred : is_message ( Message )"},
      {expected,true},
      {value,false}]},
   [{inbox_extensions_SUITE,check_message_with_properties,4,
      [{file,
         "/home/circleci/project/big_tests/tests/inbox_extensions_SUITE.erl"},
       {line,798}]},
    {inbox_extensions_SUITE,set_inbox_properties,4,
      [{file,
         "/home/circleci/project/big_tests/tests/inbox_extensions_SUITE.erl"},
       {line,792}]},
    {inbox_extensions_SUITE,'-mute_muted_entry_gets_unmuted/1-fun-2-',2,
      [{file,
         "/home/circleci/project/big_tests/tests/inbox_extensions_SUITE.erl"},
       {line,568}]},
    {escalus_story,story,4,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,72}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1291}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1223}]}]}}

Report log

mssql_mnesia_25 / odbc_mssql_mnesia / efee290
Reports root/ big
OK: 4211 / Failed: 1 / User-skipped: 97 / Auto-skipped: 0

pep_SUITE:pep_tests:unsubscribe_after_presence_unsubscription

{error,
  {{badmatch,
     [{xmlel,<<"message">>,
        [{<<"from">>,
        <<"alice_unsubscribe_after_presence_unsubscription_2461@localhost">>},
         {<<"to">>,
        <<"bob_unsubscribe_after_presence_unsubscription_2461@localhost/res1">>},
         {<<"type">>,<<"headline">>}],
        [{xmlel,<<"event">>,
           [{<<"xmlns">>,
           <<"http://jabber.org/protocol/pubsub#event">>}],
           [{xmlel,<<"items">>,
            [{<<"node">>,<<"/vBoHP2UkFGPh/qvprm6qA==">>}],
            [{xmlel,<<"item">>,
               [{<<"id">>,<<"salmon">>}],
               [{xmlel,<<"entry">>,
                  [{<<"xmlns">>,
                  <<"http://www.w3.org/2005/Atom">>}],
                  []}]}]}]},
         {xmlel,<<"headers">>,
           [{<<"xmlns">>,<<"http://jabber.org/protocol/shim">>}],
           []}]}]},
   [{pep_SUITE,'-unsubscribe_after_presence_unsubscription/1-fun-0-',2,
      [{file,"/home/circleci/project/big_tests/tests/pep_SUITE.erl"},
       {line,384}]},
    {escalus_story,story,4,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
       {line,72}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1291}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1223}]}]}}

Report log

arcusfelis

okk

chrzaszcz changed the title ~~Include overlay vars from vars-toml.config~~ Fix and simplify overlays Sep 21, 2022

chrzaszcz force-pushed the fix-overlay-vars branch 2 times, most recently from 7c75c05 to 654e815 Compare September 21, 2022 12:19

This comment was marked as outdated.

Sign in to view

chrzaszcz force-pushed the fix-overlay-vars branch from 654e815 to eaa6bfb Compare September 21, 2022 12:28

mongoose-im mentioned this pull request Sep 21, 2022

CI build failures summary - part 2 #3685

Closed

This comment was marked as outdated.

Sign in to view

Paweł Chrząszcz added 2 commits September 21, 2022 15:20

Include overlay vars from vars-toml.config

bf99f2a

This way there is only one overlay file, and the overlays work correctly with the latest Rebar3. As we do some templating in our tools, they need to support file inclusion as well.

Get rid of rel/vars-toml.config.in

efee290

It is easier to modify vars-toml.config directly

chrzaszcz force-pushed the fix-overlay-vars branch from eaa6bfb to efee290 Compare September 21, 2022 13:20

chrzaszcz marked this pull request as ready for review September 22, 2022 05:40

chrzaszcz requested a review from arcusfelis September 22, 2022 05:42

arcusfelis approved these changes Sep 22, 2022

View reviewed changes

arcusfelis merged commit 6c3819f into master Sep 22, 2022

arcusfelis deleted the fix-overlay-vars branch September 22, 2022 12:09

chrzaszcz added this to the 6.0.0 milestone Dec 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix and simplify overlays #3766

Fix and simplify overlays #3766

chrzaszcz commented Sep 21, 2022 •

edited

This comment was marked as outdated.

This comment was marked as outdated.

codecov bot commented Sep 21, 2022 •

edited

This comment was marked as outdated.

mongoose-im commented Sep 21, 2022 •

edited

arcusfelis left a comment

Fix and simplify overlays #3766

Fix and simplify overlays #3766

Conversation

chrzaszcz commented Sep 21, 2022 • edited

This comment was marked as outdated.

This comment was marked as outdated.

codecov bot commented Sep 21, 2022 • edited

Codecov Report

This comment was marked as outdated.

mongoose-im commented Sep 21, 2022 • edited

arcusfelis left a comment

Choose a reason for hiding this comment

chrzaszcz commented Sep 21, 2022 •

edited

codecov bot commented Sep 21, 2022 •

edited

mongoose-im commented Sep 21, 2022 •

edited