Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate DOAP with XEP list for xmpp.org #4021

Merged
merged 5 commits into from
May 16, 2023
Merged

Generate DOAP with XEP list for xmpp.org #4021

merged 5 commits into from
May 16, 2023

Conversation

chrzaszcz
Copy link
Member

@chrzaszcz chrzaszcz commented May 12, 2023

The goal is to have automatically generated information about MongooseIM features and implemented XEP's on the xmpp.org page. To do this, a DOAP file needs to be provided.

To make the process automatic, I decided to use the already existing -xep(...) attributes and adapt the XEP Tool to produce the DOAP file in doc/mongooseim.doap using a Mustache template. This way the file is automatically deployed with MongooseDocs and is available at https://esl.github.io/MongooseDocs/latest/mongooseim.doap.

The XEP tool is also updated, because it wasn't working due to xmpp.org website changes. I also changed it to prevent all the lengthy screen scraping of all the individual XEP pages. The script is now using the XML XEP list instead. The links are changed to point to the latest version, because the DOAP needs them like this, and most of the links for previous versions were dead anyway. The new script is cleaner and works in a few seconds instead of minutes. There is also better checking for version conflicts in the declared XEPs, and a new way to declare supported legacy versions.

I've already pull-requested the change and manually supplied a DOAP file to the MongooseDocs repo, so you can check out how the MongooseIM page looks like.

Excluded from this PR

  • Adding attributes for ProtoXEPs like muc-light
  • Improvement of XEP coverage, adding missing already implemented XEPs etc. will be done separately.
  • Possible attributes for RFCs like for XEPs.
  • Tests for the XEP tool... not easy because it is an escript. Btw I checked it with dialyzer by temporarily converting it to a module.

@mongoose-im
Copy link
Collaborator

small_tests_24 / small_tests / d0b206a
Reports root / small

@codecov
Copy link

codecov bot commented May 12, 2023

Codecov Report

Patch coverage has no change and project coverage change: -0.02 ⚠️

Comparison is base (7c57e02) 82.21% compared to head (1785139) 82.19%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4021      +/-   ##
==========================================
- Coverage   82.21%   82.19%   -0.02%     
==========================================
  Files         535      535              
  Lines       33841    33841              
==========================================
- Hits        27823    27817       -6     
- Misses       6018     6024       +6     
Impacted Files Coverage Δ
src/mam/mod_mam.erl 97.40% <ø> (ø)
src/mam/mod_mam_muc.erl 75.46% <ø> (ø)
src/mam/mod_mam_pm.erl 89.60% <ø> (ø)
src/mod_amp.erl 96.51% <ø> (ø)
src/mod_carboncopy.erl 90.07% <ø> (ø)
src/sasl/cyrsasl_external.erl 88.60% <ø> (ø)

... and 8 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@mongoose-im
Copy link
Collaborator

mongoose-im commented May 12, 2023

small_tests_24 / small_tests / 069a190
Reports root / small


small_tests_25_arm64 / small_tests / 069a190
Reports root / small


small_tests_25 / small_tests / 069a190
Reports root / small


ldap_mnesia_24 / ldap_mnesia / 069a190
Reports root/ big
OK: 2222 / Failed: 0 / User-skipped: 834 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 069a190
Reports root/ big
OK: 4194 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


ldap_mnesia_25 / ldap_mnesia / 069a190
Reports root/ big
OK: 2222 / Failed: 0 / User-skipped: 834 / Auto-skipped: 0


dynamic_domains_mysql_redis_25 / mysql_redis / 069a190
Reports root/ big
OK: 4168 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 069a190
Reports root/ big
OK: 4194 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 069a190
Reports root/ big
OK: 4191 / Failed: 0 / User-skipped: 91 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / 069a190
Reports root/ big
OK: 4567 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0


elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 069a190
Reports root/ big
OK: 2728 / Failed: 0 / User-skipped: 667 / Auto-skipped: 0


internal_mnesia_25 / internal_mnesia / 069a190
Reports root/ big
OK: 2368 / Failed: 0 / User-skipped: 688 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 069a190
Reports root/ big
OK: 4567 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0


mssql_mnesia_25 / odbc_mssql_mnesia / 069a190
Reports root/ big
OK: 4564 / Failed: 0 / User-skipped: 100 / Auto-skipped: 0


mysql_redis_25 / mysql_redis / 069a190
Reports root/ big
OK: 4553 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0

@mongoose-im
Copy link
Collaborator

mongoose-im commented May 12, 2023

small_tests_24 / small_tests / c0605d9
Reports root / small


small_tests_25_arm64 / small_tests / c0605d9
Reports root / small


small_tests_25 / small_tests / c0605d9
Reports root / small


ldap_mnesia_24 / ldap_mnesia / c0605d9
Reports root/ big
OK: 2222 / Failed: 0 / User-skipped: 834 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / c0605d9
Reports root/ big
OK: 4194 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


ldap_mnesia_25 / ldap_mnesia / c0605d9
Reports root/ big
OK: 2222 / Failed: 0 / User-skipped: 834 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / c0605d9
Reports root/ big
OK: 4194 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


dynamic_domains_mysql_redis_25 / mysql_redis / c0605d9
Reports root/ big
OK: 4168 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / c0605d9
Reports root/ big
OK: 4567 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0


elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / c0605d9
Reports root/ big
OK: 2728 / Failed: 0 / User-skipped: 667 / Auto-skipped: 0


internal_mnesia_25 / internal_mnesia / c0605d9
Reports root/ big
OK: 2368 / Failed: 0 / User-skipped: 688 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / c0605d9
Reports root/ big
OK: 4567 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0


mysql_redis_25 / mysql_redis / c0605d9
Reports root/ big
OK: 4553 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0


mssql_mnesia_25 / odbc_mssql_mnesia / c0605d9
Reports root/ big
OK: 4564 / Failed: 0 / User-skipped: 100 / Auto-skipped: 0

@mongoose-im
Copy link
Collaborator

mongoose-im commented May 15, 2023

small_tests_24 / small_tests / 440dc17
Reports root / small


small_tests_25 / small_tests / 440dc17
Reports root / small


small_tests_25_arm64 / small_tests / 440dc17
Reports root / small


ldap_mnesia_24 / ldap_mnesia / 440dc17
Reports root/ big
OK: 2222 / Failed: 0 / User-skipped: 834 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 440dc17
Reports root/ big
OK: 4194 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


ldap_mnesia_25 / ldap_mnesia / 440dc17
Reports root/ big
OK: 2222 / Failed: 0 / User-skipped: 834 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 440dc17
Reports root/ big
OK: 4194 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


dynamic_domains_mysql_redis_25 / mysql_redis / 440dc17
Reports root/ big
OK: 4168 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / 440dc17
Reports root/ big
OK: 4567 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 440dc17
Reports root/ big
OK: 4191 / Failed: 0 / User-skipped: 91 / Auto-skipped: 0


elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 440dc17
Reports root/ big
OK: 2728 / Failed: 0 / User-skipped: 667 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 440dc17
Reports root/ big
OK: 4567 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0


mysql_redis_25 / mysql_redis / 440dc17
Reports root/ big
OK: 4553 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0


mssql_mnesia_25 / odbc_mssql_mnesia / 440dc17
Reports root/ big
OK: 4564 / Failed: 0 / User-skipped: 100 / Auto-skipped: 0

@mongoose-im
Copy link
Collaborator

mongoose-im commented May 15, 2023

small_tests_24 / small_tests / df520f7
Reports root / small


small_tests_25 / small_tests / df520f7
Reports root / small


small_tests_25_arm64 / small_tests / df520f7
Reports root / small


ldap_mnesia_24 / ldap_mnesia / df520f7
Reports root/ big
OK: 2222 / Failed: 0 / User-skipped: 834 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / df520f7
Reports root/ big
OK: 4194 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


dynamic_domains_mysql_redis_25 / mysql_redis / df520f7
Reports root/ big
OK: 4168 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


ldap_mnesia_25 / ldap_mnesia / df520f7
Reports root/ big
OK: 2222 / Failed: 0 / User-skipped: 834 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / df520f7
Reports root/ big
OK: 4194 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / df520f7
Reports root/ big
OK: 4191 / Failed: 0 / User-skipped: 91 / Auto-skipped: 0


internal_mnesia_25 / internal_mnesia / df520f7
Reports root/ big
OK: 2368 / Failed: 0 / User-skipped: 688 / Auto-skipped: 0


elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / df520f7
Reports root/ big
OK: 2728 / Failed: 0 / User-skipped: 667 / Auto-skipped: 0


mysql_redis_25 / mysql_redis / df520f7
Reports root/ big
OK: 4553 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / df520f7
Reports root/ big
OK: 4585 / Failed: 1 / User-skipped: 97 / Auto-skipped: 0

jingle_SUITE:all:mongoose_sends_reINVITE_on_source_add_action
{error,
  {{assertion_failed,assert,is_iq_result,
     {xmlel,<<"iq">>,
       [{<<"from">>,
         <<"alice_mongoose_sends_reinvite_on_source_add_action_1606@localhost/res1">>},
        {<<"to">>,
         <<"bob_mongoose_sends_reinvite_on_source_add_action_1606@localhost/res1">>},
        {<<"id">>,<<"a01063e0-c896-4810-9588-d1ee8bcafb10">>},
        {<<"type">>,<<"error">>}],
       [{xmlel,<<"error">>,
          [{<<"code">>,<<"404">>},{<<"type">>,<<"cancel">>}],
          [{xmlel,<<"item-not-found">>,
             [{<<"xmlns">>,
             <<"urn:ietf:params:xml:ns:xmpp-stanzas">>}],
             []}]}]},
     "<iq from='alice_mongoose_sends_reinvite_on_source_add_action_1606@localhost/res1' to='bob_mongoose_sends_reinvite_on_source_add_action_1606@localhost/res1' id='a01063e0-c896-4810-9588-d1ee8bcafb10' type='error'><error code='404' type='cancel'><item-not-found xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/></error></iq>"},
   [{escalus_new_assert,assert_true,2,
      [{file,
         "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_new_assert.erl"},
       {line,84}]},
    {jingle_SUITE,accept_jingle_session,4,
      [{file,"/home/circleci/project/big_tests/tests/jingle_SUITE.erl"},
       {line,370}]},
    {jingle_SUITE,'-mongoose_sends_reINVITE_on_source_add_action/1-fun-0-',
      2,
      [{file,"/home/circleci/project/big_tests/tests/jingle_SUITE.erl"},
       {line,338}]},
    {escalus_story,story,4,
      [{file,
         "/home/circ...

Report log


mssql_mnesia_25 / odbc_mssql_mnesia / df520f7
Reports root/ big
OK: 4564 / Failed: 0 / User-skipped: 100 / Auto-skipped: 0

- Add the DOAP output format required by xmpp.org
- Support the new annotations
  - 'status' (complete or partial) is used instead of arbitrary comments
  - 'legacy_versions' is used to highlight the main version
- Add conflict detection when the same XEP is declared multiple times
- Improve coding style
- Resolve version conflicts.
- Use 'status' and 'legacy_versions'.
@mongoose-im
Copy link
Collaborator

mongoose-im commented May 15, 2023

small_tests_25_arm64 / small_tests / 8b985ba
Reports root / small


small_tests_25 / small_tests / 8b985ba
Reports root / small


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 8b985ba
Reports root/ big
OK: 4194 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


dynamic_domains_mysql_redis_25 / mysql_redis / 8b985ba
Reports root/ big
OK: 4168 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


internal_mnesia_25 / internal_mnesia / 8b985ba
Reports root/ big
OK: 2368 / Failed: 0 / User-skipped: 688 / Auto-skipped: 0


ldap_mnesia_25 / ldap_mnesia / 8b985ba
Reports root/ big
OK: 2222 / Failed: 0 / User-skipped: 834 / Auto-skipped: 0


elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 8b985ba
Reports root/ big
OK: 2728 / Failed: 0 / User-skipped: 667 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 8b985ba
Reports root/ big
OK: 4191 / Failed: 0 / User-skipped: 91 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 8b985ba
Reports root/ big
OK: 4567 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0


mysql_redis_25 / mysql_redis / 8b985ba
Reports root/ big
OK: 4553 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0


mssql_mnesia_25 / odbc_mssql_mnesia / 8b985ba
Reports root/ big
OK: 4564 / Failed: 0 / User-skipped: 100 / Auto-skipped: 0

@mongoose-im
Copy link
Collaborator

mongoose-im commented May 15, 2023

small_tests_24 / small_tests / 1785139
Reports root / small


small_tests_25_arm64 / small_tests / 1785139
Reports root / small


small_tests_25 / small_tests / 1785139
Reports root / small


ldap_mnesia_24 / ldap_mnesia / 1785139
Reports root/ big
OK: 2222 / Failed: 0 / User-skipped: 834 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 1785139
Reports root/ big
OK: 4194 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


ldap_mnesia_25 / ldap_mnesia / 1785139
Reports root/ big
OK: 2222 / Failed: 0 / User-skipped: 834 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 1785139
Reports root/ big
OK: 4194 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0


dynamic_domains_mysql_redis_25 / mysql_redis / 1785139
Reports root/ big
OK: 4168 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0


elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 1785139
Reports root/ big
OK: 2728 / Failed: 0 / User-skipped: 667 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 1785139
Reports root/ big
OK: 4191 / Failed: 0 / User-skipped: 91 / Auto-skipped: 0


pgsql_mnesia_24 / pgsql_mnesia / 1785139
Reports root/ big
OK: 4567 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0


mysql_redis_25 / mysql_redis / 1785139
Reports root/ big
OK: 4553 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0


internal_mnesia_25 / internal_mnesia / 1785139
Reports root/ big
OK: 2368 / Failed: 0 / User-skipped: 688 / Auto-skipped: 0


pgsql_mnesia_25 / pgsql_mnesia / 1785139
Reports root/ big
OK: 4567 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0


mssql_mnesia_25 / odbc_mssql_mnesia / 1785139
Reports root/ big
OK: 4563 / Failed: 1 / User-skipped: 100 / Auto-skipped: 0

service_domain_db_SUITE:db:db_event_could_appear_with_lower_id
{error,
  {{badmatch,1331},
   [{service_domain_db_SUITE,db_event_could_appear_with_lower_id,1,
      [{file,
         "/home/circleci/project/big_tests/tests/service_domain_db_SUITE.erl"},
       {line,712}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1291}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1223}]}]}}

Report log


mssql_mnesia_25 / odbc_mssql_mnesia / 1785139
Reports root/ big
OK: 4564 / Failed: 0 / User-skipped: 100 / Auto-skipped: 0

@chrzaszcz chrzaszcz marked this pull request as ready for review May 15, 2023 16:07
Copy link
Contributor

@jacekwegr jacekwegr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! 👍

@jacekwegr jacekwegr merged commit 7c0303d into master May 16, 2023
4 checks passed
@jacekwegr jacekwegr deleted the xep-list branch May 16, 2023 08:26
@chrzaszcz chrzaszcz added this to the 6.2.0 milestone Dec 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants