Taxonomy Fields is not created correctly with Samples/Taxonomies.xml

[csharp01]

I have had issues with the taxonomy field when they are created with this example (Samples/Taxonomies.xml) by creating both the TaxonomyFieldType and the Note field results in a corrupted field under the hood in SharePoint Online.
MS told me to let the platform create all necessary references.

The issue when also provisioning the Note field result in that the index will not contains crawled content for the ows_taxId_FieldName

I'm not sure exactly what is wrong by provisioning both fields, but taxonomy fieldst works as expected if I provisioing this type of field only with PnP.Core and only specify the TaxonomyFieldType.

When using the the provisioning schema the taxonomy fields will not be created correct.

[PaoloPia]

Hi, thanks for your feedback and for your contribution. We will double-check the sample and the provisioning engine. Stay tuned.

[PaoloPia]

Hi, we double-checked the demo and we were not able to reproduce the issue. We tested it on two different tenants in Office 365. Could you be so kind to expand a little bit more about your issue? Thanks.

[csharp01]

Hi,

I was using the 2015-08 schema and provisioning both TaxonomyFieldType and TaxonomyFieldTypeMulti with corresponding Note fields, exactly as in the example, of course apart from other guid values for the term set and so on.

First all seems to work as expected. You can tag content with some terms. You also may get content crawled when you are testing the first time. Then if you change some items to i.e. tag an item with more terms (multi-value) or create a new items in the same list. The tax-field will be broken and the crawler will not index the managed metadata field anymore.

In previous version in the cloud SPO automatically creates the managed property fields ows_FieldName and owstaxIdFieldName, and the last one was automatically mapped to the crawled field ows_taxId_FieldName. These MM fields will still be created but will not be autmaticaly mapped to the crawled property (ows_taxId_FieldName). The MM are grayed out in the Managed Properties view in Search Schemas. If you view the Crawled Properties view it will display that the crawled property are mapped to the managed property. But this is wrong.

For new site collections the MM owstaxiIdFieldName will be created but not automatically mapped to the crawled propety. This are a changen Microsof has done.

So in the future you must manually map the ows_taxId_FieldName to a RefinabelStringXX MM property. This was the new recommendation I have received from MS now.

Back to the issue.

If the MM owstaxIdFieldName does not contains any value is because either the MM is not mapped or the tax-field has some issues.

MS analyzed the logs under the hood in our tentant and saw that the ows_taxId_FieldName are not indexed becaucse there are some issues with the reference against the Note field. These fields Tax- or Note field are in some way corrupted.

I asked him to see if I could get some information what he saw on his side to maybe I have did something wrong in the provisioning xml.

The only thing he could inform back to me is that these fields are note created correct.

The only difference I can see from my side between creating these fields through the xml or manually is that the Name property and StaticName property of the Note field will be some kind of guid when let the platform create the Note field instead of try to create and link it to the tax field through the framework.

I have named the field as shown in the example.

The MS guy was a search engineer and not a developer but he told me to let the platform create all necessary fields, like the Note field and the link between them because there are more stuff going on under the hood.

Br
Dan

[VesaJuvonen]

Hi Dan,
since you are referring to discussions with MS in these notes, could you send me for example private Yammer message with support case information (case ID or contact), so that can see the full thread internally. Just need to confirm to whom you talked to and to ensure that we are aligned fully internally on the messaging. Thx.

[csharp01]

Hi Vesa,
I have sent information to your Hotmail.

Br.
Dan

[VesaJuvonen]

Hi Dan,
Just to follow up on this. Seems that email never arrived to my Hotmail account. Could you send that again to vesa.juvonen at Hotmail for checking the status or alternatively with private message in Yammer.

[csharp01]

I have sent you a new mail now.

[christopher-walker]

Hi,
Not sure if you resolved this issue, but I seem to have something similar. Working with SP 2013 initially I created a large number of Taxonomy fields by hand everything seemed to work well we could tag fields the fields worked as expected.
At a later stage in the project we started to look into search, then checking the crawled properties some of the fields had ows_taxid prefix and some did not. (when i say they did not get the ows_tax id prefix they were populated on a document / listitem and received a ows_ prefix but no managed property associated).
The next thing we did was to create a column using the UI in our test environment and export it. We then deployed this taxonomy field and its related note field to the test environment ... it worked ... strange ... we then tried to follow the same process for another field and it failed. This seems to intermittently succeed and fail, not sure if this is to do with the provisioning engine, csom or ther crawler.
It just seems like there are so many different things we need to look at in order to debug this :(

[csharp01]

Hi Christopher,
From my point of view the provisioning engine are still creating taxonomy field in a wrong way when using the xml provisioning.
When I was working with MS to resolve the provisioning of taxonomy fields we found that when provisioning with PnP Core (CSOM) the fields are created correct.
I still use xml provisioning for content types but bofore apply the xml template I first create the taxonomy fields through CSOM/PnP Core.
In the xml template I only ref the internal name of the field in the content types and let the platform (SP/SPO) add the related note filed automatically.

Last week I updated to lates version of PnP Core and 2016 schema but it still not working with taxonomy fields through the xml provisioning.

I have been busy and have not had time to follow up this with the PnP team.
Note. The taxonomy fields are created through the xml provisioning, but not 100% correct. You will notice the difference when you are wokring with the managed properties and refinables. The crawler will not crawl tha taxonomy field if the refference with the Note field are not 100% correct. More information of what exactly are the difference of the non working field and working fields did I not received from the search team.

[csharp01]

I'm using the following code to provisioning taxonomy fields.

TaxonomyFields.xml

<?xml version="1.0" encoding="utf-8" ?>
<Fields>
  <Field ID="{xxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx}" StaticName="ContosoOrganization" Name="ContosoOrganization" DisplayName="Organiszation" Type="TaxonomyFieldTypeMulti" Group="Contoso Core">
        <TaxonomyGroup ID="xxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx">People</TaxonomyGroup>
        <TermSet ID="xxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx">Organization</TermSet>
    </Field>

    <Field ID="{xxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx}" StaticName="ContosoLinkCategory" Name="ContosoLinkCategory" DisplayName="Link Category" Type="TaxonomyFieldType" Group="Contoso Core">
        <TaxonomyGroup ID="xxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx">Contoso</TaxonomyGroup>
        <TermSet ID="xxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx">Link Categories</TermSet>
    </Field>
</Fields>

CSOM/PnP Core

/// <summary>
/// Create site collection taxonomy fields.
/// </summary> 
public void CreateTaxonomyFields(ClientContext ctx)
        {
            var taxonomyFieldsConfiguration = XDocument.Load(@".\Resources\TaxonomyFields.xml").Element("Fields");
            var fields = taxonomyFieldsConfiguration.Descendants("Field");
            if (fields != null && fields.Count() > 0)
            {
                foreach (XElement field in fields)
                {
                    // Create the taxonomy field.
                    Guid fieldId = Guid.Parse(field.Attribute("ID").Value);
                    string fieldName = field.Attribute("Name").Value;
                    string fieldDisplayName = field.Attribute("DisplayName").Value;
                    string fieldGroupName = field.Attribute("Group").Value;
                    string fieldType = field.Attribute("Type").Value;
                    bool taxonomyFieldMultiValue = (fieldType == "TaxonomyFieldTypeMulti");
                    Guid taxonomyGroupId = Guid.Parse(field.Descendants("TaxonomyGroup").First().Attribute("ID").Value);
                    Guid termSetId = Guid.Parse(field.Descendants("TermSet").First().Attribute("ID").Value);

                    CreateTaxonomyField(ctx, fieldId, fieldName, fieldDisplayName, fieldGroupName, taxonomyGroupId, termSetId, taxonomyFieldMultiValue);
                }
            }
        }

        private void CreateTaxonomyField(ClientContext ctx, Guid taxonomyFieldId, string taxonomyFieldName, string taxonomyFieldDisplayName, string fieldGroupName, Guid taxonomyGroupId, Guid termSetId, bool multiValue)
        {
            if (!ctx.Web.FieldExistsById(taxonomyFieldId))
            {
                // Get access to the right term set
                TaxonomySession taxonomySession = TaxonomySession.GetTaxonomySession(ctx.Web.Context);
                TermStore termStore = taxonomySession.GetDefaultSiteCollectionTermStore();
                TermGroup termGroup = termStore.Groups.GetById(taxonomyGroupId);
                TermSet termSet = termGroup.TermSets.GetById(termSetId);
                ctx.Web.Context.Load(termStore);
                ctx.Web.Context.Load(termSet);
                ctx.Web.Context.ExecuteQueryRetry();

                TaxonomyFieldCreationInformation fieldCI = new TaxonomyFieldCreationInformation()
                {
                    Id = taxonomyFieldId,
                    InternalName = taxonomyFieldName,
                    DisplayName = taxonomyFieldDisplayName,
                    Group = fieldGroupName,
                    TaxonomyItem = termSet,
                    MultiValue = multiValue
                };
                ctx.Web.CreateTaxonomyField(fieldCI);
            }
        }

[christopher-walker]

Thanks for the info and fast response Dan,

We ended up taking a similar approach as you did, I guess as we are using a CT hub the note field should be attached and distributed along with the content type.

One thing I noticed when I was exporting the site columns was that it placed the note fields are the top of the Scheme and then the Taxonomy fields towards the bottom. Im not sure if this is intentional or not but may suggest some form of timing issue?

[PaoloPia]

Hi,
Is this one still an open issue? Otherwise, I will close it. Thanks.

[christopher-walker]

Hi Paolo,

I haven't had time to test the new provisioning tool yet have been working on some fully trusted solutions.

The issue is that Taxonomy fields provisioned with the "Provisioning" tool :), seem to work fine. It is only when you crawl your content and try to use the fields as refiners that you see that there is an issue with them. I haven't tested this online only with SharePoint 2013. So unless something has changed with the way the taxonomy fields are provisioned in code then there will still be an issue.

Thanks

Chris

[PaoloPia]

Ok, I self-assigned to me the issue and I will dig into it. Thanks for the update.

[bcage29]

I ran into this same issue where provisioned taxonomy fields were not being indexed the same as taxonomy fields created through the UI. The issue is that SharePoint wants your hidden note field to have "TaxHTField" in the name. If that is not there, it doesn't index it the same. Here is an updated sample for a taxonomy field that will get the correct indexed crawled/managed properties.

Also, the managed property that is automatically created (greyed out managed property) returns a different type of data than expected. It is an array of data with the label and unique identifier. If you need to query from this field, you need to query using the unique identifier. The data format is documented https://technet.microsoft.com/en-us/library/jj613136.aspx under 'Data format for Managed Metadata.' If you need only the label value (ie to use in a refiner), you can create a new managed property then map it to the ows_Project crawled property.

After performing a full crawl you should have the following:

Crawled Properties
ows_Project - stores the managed metatdata label
ows_taxId_Project - stores an array of data

Managed Property
owstaxIdProject - automatically created and greyed out

<Field Type="Note" DisplayName="ProjectTaxHTField" StaticName="ProjectTaxHTField" Name="ProjectTaxHTField" ID="{D7917FD4-5EEF-4307-9EBF-372245389DF9}" ShowInViewForms="FALSE" Required="FALSE" Hidden="TRUE" CanToggleHidden="TRUE" />
  <Field Type="TaxonomyFieldType" DisplayName="Project" Name="Project" StaticName="Project" ShowField="Term1033" Required="FALSE" EnforceUniqueValues="FALSE" Group="PnP" ID="{21515F04-1C08-4B94-A6ED-630436679ED3}">
    <Customization>
      <ArrayOfProperty>
        <Property>
          <Name>SspId</Name>
          <Value xmlns:q1="http://www.w3.org/2001/XMLSchema" p4:type="q1:string" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">{sitecollectiontermstoreid}</Value>
        </Property>
        <Property>
          <Name>TermSetId</Name>
          <Value xmlns:q2="http://www.w3.org/2001/XMLSchema" p4:type="q2:string" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">{termsetid:PnpTermSets:Projects}</Value>
        </Property>
        <Property>
          <Name>TextField</Name>
          <Value xmlns:q6="http://www.w3.org/2001/XMLSchema" p4:type="q6:string" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">D7917FD4-5EEF-4307-9EBF-372245389DF9</Value>
        </Property>
        <Property>
          <Name>IsPathRendered</Name>
          <Value xmlns:q7="http://www.w3.org/2001/XMLSchema" p4:type="q7:boolean" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">false</Value>
        </Property>
        <Property>
          <Name>IsKeyword</Name>
          <Value xmlns:q8="http://www.w3.org/2001/XMLSchema" p4:type="q8:boolean" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">false</Value>
        </Property>
      </ArrayOfProperty>
    </Customization>
  </Field>

[csharp01]

Hi bcage29, Thanks for the update!

I'm currently starting up a new O365 project for a customer and we need to do some content type provisioning. I'm going to try this solution with the "...TaxHTField" field name of the hidden Note field. I hope it will work with the xml provisioning. it would be great. :-)

[phillipharding]

I've tried doing this today and was having the same problems on Office365/SharePoint Online.

To get it working I changed my field ID's from uppercase to lowercase and also applied "TaxHTField" to the end of my Note fields.

After doing this, populating data, waiting for search to crawl etc the Search Schema showed the correct crawled properties and I was able to Search and do Term Driven/Managed navigation using the managed metadata fields.

A sample of the Field definitions that worked for me are;

<Field ID="{766e0ffd-9bf4-433d-8c13-3d3e4b852ead}" Type="Note" Name="polDocumentTagTaxHTField" StaticName="polDocumentTagTaxHTField" DisplayName="polDocumentTagTaxHTField" Group="ADoc Management" ShowInViewForms="FALSE" Required="FALSE" Hidden="TRUE" CanToggleHidden="TRUE" />
<Field ID="{990c32aa-fd36-4a03-a86c-9989c019e24b}" Type="TaxonomyFieldType" Name="polDocumentTag" StaticName="polDocumentTag" DisplayName="Document Tag" Group="ADoc Management" ShowField="Term1033" Required="FALSE" EnforceUniqueValues="FALSE" Mult="FALSE">
	<Default></Default>
	<Customization>
		<ArrayOfProperty>
			<Property>
				<Name>SspId</Name>
				<Value xmlns:q1="http://www.w3.org/2001/XMLSchema" p4:type="q1:string" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">{sitecollectiontermstoreid}</Value>
			</Property>
			<Property>
				<Name>TermSetId</Name>
				<Value xmlns:q2="http://www.w3.org/2001/XMLSchema" p4:type="q2:string" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">18601470-759c-4658-87c6-a1e489700673</Value>
			</Property>
			<Property>
				<Name>TextField</Name>
				<Value xmlns:q6="http://www.w3.org/2001/XMLSchema" p4:type="q6:string" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">{766e0ffd-9bf4-433d-8c13-3d3e4b852ead}</Value>
			</Property>
			<Property>
				<Name>AnchorId</Name>
				<Value xmlns:q3="http://www.w3.org/2001/XMLSchema" p4:type="q3:string" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">00000000-0000-0000-0000-000000000000</Value>
			</Property>
			<Property>
				<Name>IsPathRendered</Name>
				<Value xmlns:q7="http://www.w3.org/2001/XMLSchema" p4:type="q7:boolean" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">false</Value>
			</Property>
			<Property>
				<Name>IsKeyword</Name>
				<Value xmlns:q8="http://www.w3.org/2001/XMLSchema" p4:type="q8:boolean" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">false</Value>
			</Property>
		</ArrayOfProperty>
	</Customization>
</Field>

and for multi-value;

<Field ID="{50cc3df8-5c25-48ef-924e-3816c9830a58}" Type="Note" Name="polDocumentTaggingTaxHTField" StaticName="polDocumentTaggingTaxHTField" DisplayName="polDocumentTaggingTaxHTField" Group="ADoc Management" ShowInViewForms="FALSE" Required="FALSE" Hidden="TRUE" CanToggleHidden="TRUE" />
<Field ID="{9c2aa187-0834-4cea-aa4a-40e7e8e439d4}" Type="TaxonomyFieldTypeMulti" Name="polDocumentTagging" StaticName="polDocumentTagging" DisplayName="Document Tagging" Group="ADoc Management" ShowField="Term1033" Required="FALSE" EnforceUniqueValues="FALSE" Mult="TRUE">
	<Default></Default>
	<Customization>
	  <ArrayOfProperty>
	    <Property>
	      <Name>SspId</Name>
	      <Value xmlns:q1="http://www.w3.org/2001/XMLSchema" p4:type="q1:string" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">{sitecollectiontermstoreid}</Value>
	    </Property>
	    <Property>
	      <Name>TermSetId</Name>
	      <Value xmlns:q2="http://www.w3.org/2001/XMLSchema" p4:type="q2:string" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">18601470-759c-4658-87c6-a1e489700673</Value>
	    </Property>
	    <Property>
	      <Name>TextField</Name>
	      <Value xmlns:q6="http://www.w3.org/2001/XMLSchema" p4:type="q6:string" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">{50cc3df8-5c25-48ef-924e-3816c9830a58}</Value>
	    </Property>
			<Property>
				<Name>AnchorId</Name>
				<Value xmlns:q3="http://www.w3.org/2001/XMLSchema" p4:type="q3:string" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">00000000-0000-0000-0000-000000000000</Value>
			</Property>
	    <Property>
	      <Name>IsPathRendered</Name>
	      <Value xmlns:q7="http://www.w3.org/2001/XMLSchema" p4:type="q7:boolean" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">false</Value>
	    </Property>
	    <Property>
	      <Name>IsKeyword</Name>
	      <Value xmlns:q8="http://www.w3.org/2001/XMLSchema" p4:type="q8:boolean" xmlns:p4="http://www.w3.org/2001/XMLSchema-instance">false</Value>
	    </Property>
	  </ArrayOfProperty>
	</Customization>
</Field>

[csharp01]

Thanks Phil for your feedback on this issue.
I actually have not had time to try to add "TaxHTField" to the end of the name attributes. But if it works, it's good. I prefer to create taxonomy fields through CSOM/PnP (C#).
I do not remember the root cause of problems with ID, but I always creates new fields ID in lowercase.

• Add "TaxHTField" to the end of the field name properties.
• Use lowercase for the guid of new field ID property.

I think we can close this case now.