Replication of the schema fails 'master branch' -> 1.2.11 or 1.3.1

[389-ds-bot]

Cloned from Pagure issue: https://pagure.io/389-ds-base/issue/47676

• Created at 2014-01-17 21:54:57 by tbordaz (@tbordaz)
• Closed as Fixed
• Assigned to tbordaz (@tbordaz)

---

This bug is possibly a duplicate of https://fedorahosted.org/389/ticket/47633

The 1.2.11 schema contains the following definition in 60rfc3712.ldif

objectClasses: (
  
  NAME 'printer-uri'
  DESC 'A URI supported by this printer.'
  X-ORIGIN 'rfc3712'
  )

This definition does not exist in 1.3.1 and 'master branch'.

Since https://fedorahosted.org/389/ticket/47490, the schema of a consumer (master/hub/consumer) is not overwritten if the consumer schema is a superset of the supplier schema.
If we have a supplier 1.3.2.1 (and after) and a consumer 1.2.11, and the schema is updated on the supplier. Then the schema is never pushed.

To reproduce, install 2 masters: M1:master branch, M2: 1.2.11
Update the schema on M1, and check the following message in M1 error logs:

[17/Jan/2014:15:56:14 +0100] NSMMReplicationPlugin - Schema agmt="cn=meTo_localhost.localdomain:45389" (localhost:45389) must not be overwritten (set replication log for additional info)

Enabling replication log: we can see the detail

[17/Jan/2014:15:56:14 +0100] schema - Fail to retrieve in the local supplier schema [printer-uri or printer-uri-oid]

[389-ds-bot]

Comment from rmeggins (@richm) at 2014-01-17 22:03:38

Looks like I caused this bug :-( with the fix for https://fedorahosted.org/389/ticket/47647

I guess we have to add back the bogus schema entry to make schema replication work :P This means our schema tests in master will fail. I'm not sure how we can fix this and keep backwards compatability.

[389-ds-bot]

Comment from tbordaz (@tbordaz) at 2014-01-17 22:55:15

There are two issues:

since https://fedorahosted.org/389/ticket/47490 and https://fedorahosted.org/389/ticket/47647, it fails to push the schema 'master' -> 1.3.1 and 'master' -> 1.2.11 because of 'printer-uri'
A workaround is to update 60rfc3712.ldif under <1.3.1 or 1.2.11 install>/etc/dirsrv/schema, and update already created instances

since https://fedorahosted.org/389/ticket/47541, it fails to push the schema 'master' -> 1.2.11 because of unhashed#user#password pseudo attribute

[389-ds-bot]

Comment from tbordaz (@tbordaz) at 2014-01-22 22:11:37

attachment
0001-Ticket-47676-Replication-of-the-schema-fails-master-.patch

[389-ds-bot]

Comment from tbordaz (@tbordaz) at 2014-01-29 22:23:23

attachment
0002-Ticket-ticket47676-Replication-of-the-schema-fails-m.patch

[389-ds-bot]

Comment from rmeggins (@richm) at 2014-01-30 22:34:05

In create_repl_schema_policy() - you should clear or free the pblock after every add operation. It may cause strange problems to re-use the same pblock for multiple operations (even though it may seem to work fine in your testing).

Your return -1 will leak memory. Don't do a short-circuit return:

 	672	        if (return_value != LDAP_SUCCESS && return_value != LDAP_ALREADY_EXISTS) { 
 	673	                slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Warning: unable to " 
 	674	                        "create configuration entry %s: %s\n", repl_schema_top, 
 	675	                        ldap_err2string(return_value)); 
 	676	                return -1; 
 	677	        }

• instead, have a done: label that does all of the cleanup:

     int rc = 0;
...
 	672	        if (return_value != LDAP_SUCCESS && return_value != LDAP_ALREADY_EXISTS) { 
 	673	                slapi_log_error(SLAPI_LOG_FATAL, repl_plugin_name, "Warning: unable to " 
 	674	                        "create configuration entry %s: %s\n", repl_schema_top, 
 	675	                        ldap_err2string(return_value)); 
 	676	                rc = -1;
                                goto done;
 	677	        }
...
done:
     if (pb) {
         slapi_pblock_free(pb);
         pb = NULL;
     }
     return rc;

You will also need to free Slapi_Entry *e if the add fails - if the add is successful, set e = NULL so you will know that you don't need to free it.

 	219	repl_schema_policy_t supplier_policy = {0}; 
 	220	repl_schema_policy_t consumer_policy = {0};

Do these need to be globals? If not, please define them as static.

Also, the parsing, structs, etc. - do you need to have a separate name and oid? Can't you just have a char *name_or_oid?

Finally, I think you can get rid of a lot of the parsing, etc. by just having 4 different multi-valued attributes:
schemaUpdateAccept: acceptattr
schemaUpdateAccept: acceptoid
...
schemaUpdateReject: rejectattr
schemaUpdateReject: rejectoid
...

I don't know if you even need to keep a separate list of objectclasses and attributetypes - I suppose if we allow you to have an objectclass and an attribute with the same name, then you might want to reject objectclass "foo" but allow attribute "foo". As far as OIDs go, you will never have an objectclass or attribute with the same OID as any other schema element.

[389-ds-bot]

Comment from tbordaz (@tbordaz) at 2014-02-04 20:20:04

attachment
0002-Ticket-47676-test-case.patch

[389-ds-bot]

Comment from tbordaz (@tbordaz) at 2014-02-04 20:20:18

attachment
0003-Ticket-47676-Replication-of-the-schema-fails-master-.patch

[389-ds-bot]

Comment from tbordaz (@tbordaz) at 2014-02-05 17:08:01

git merge

Updating 01cb401..c44ea92
Fast-forward
dirsrvtests/tickets/ticket47676_test.py | 485 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
ldap/schema/01core389.ldif | 5 +
ldap/servers/plugins/replication/repl5_init.c | 98 +++++++++++++++-
ldap/servers/slapd/schema.c | 304 +++++++++++++++++++++++++++++++++++++++++++++++--
ldap/servers/slapd/slap.h | 36 ++++++
ldap/servers/slapd/slapi-private.h | 3 +
6 files changed, 922 insertions(+), 9 deletions(-)
create mode 100644 dirsrvtests/tickets/ticket47676_test.py

git push origin master

Counting objects: 30, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (16/16), done.
Writing objects: 100% (16/16), 11.80 KiB, done.
Total 16 (delta 12), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
01cb401..c44ea92 master -> master

commit c44ea92
Author: Thierry bordaz (tbordaz) tbordaz@redhat.com
Date: Tue Feb 4 11:50:45 2014 +0100

[389-ds-bot]

Comment from tbordaz (@tbordaz) at 2014-02-05 21:58:06

attachment
0001-Ticket-47676-cont.-Replication-of-the-schema-fails-m.patch

[389-ds-bot]

Comment from tbordaz (@tbordaz) at 2014-02-05 22:16:39

I did pick up a wrong OID for nsSchemaPolicy OID. Just OLC reviewed by Rich

git merge ticket47676_entries
Updating c44ea92..ce43858
Fast-forward
ldap/schema/01core389.ldif | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

git push origin master

Counting objects: 9, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 691 bytes, done.
Total 5 (delta 4), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
c44ea92..ce43858 master -> master

commit ce43858
Author: Thierry bordaz (tbordaz) tbordaz@redhat.com
Date: Wed Feb 5 16:56:10 2014 +0100

[389-ds-bot]

Comment from tbordaz (@tbordaz) at 2014-05-20 15:17:49

In order to backport ​https://fedorahosted.org/389/ticket/47721 in 1.3.2, this bug needs to be backported first

git push origin '''389-ds-base-1.3.2''' (core fix)

Counting objects: 30, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (15/15), done.
Writing objects: 100% (16/16), 11.76 KiB, done.
Total 16 (delta 12), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
dd543aa..e35d201 389-ds-base-1.3.2 -> 389-ds-base-1.3.2

commit e35d201
Author: Thierry bordaz (tbordaz) tbordaz@redhat.com
Date: Tue Feb 4 11:50:45 2014 +0100

git push origin '''389-ds-base-1.3.2''' (wrong OID)

Counting objects: 9, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 689 bytes, done.
Total 5 (delta 4), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
e35d201..d26a4a0 389-ds-base-1.3.2 -> 389-ds-base-1.3.2

commit d26a4a0
Author: Thierry bordaz (tbordaz) tbordaz@redhat.com
Date: Wed Feb 5 16:56:10 2014 +0100

[389-ds-bot]

Comment from tbordaz (@tbordaz) at 2017-02-11 23:04:38

Metadata Update from @tbordaz:

• Issue assigned to tbordaz
• Issue set to the milestone: 1.3.3 - 1/14 (January)