Question: can we filter metadata?

[FixRM]

Hello!

Thank you for a great tool! We use it to generate docs for Microsoft Common Data Service OData endpoint (formerly Dynamics CRM, formerly D365). This endpoint is generic and has quite big CSDL file (about 4 MB). Resulting JSON file is about 880 MB!!! Can you please suggest how we can cut some unused datasets and related stuff before generating OpenAPI file?

[xuzhg]

@FixRM Thanks for trying this tool.

1. there are some settings that you can use to do filter. For example: if set EnableNavigationPropertyPath = false, there will be no navigation property path.

2. IODataPathProvider has the "CanFilter" that is designed for your scenario, but so far, i don't have much requirement, so i haven't exposed it throw "ODataContext".

[FixRM]

@xuzhg, thanks for quick reply. Can you please give me a sample of how to use IODataPathProvider?

[xuzhg]

@FixRM My design is that you can derive from "ODataPathProvider" and override this virtual method. However, "ODataPathProvider" is used in the ODataContext, that's an internal class.

I'd like to expose an interface in the setting class so developer can create an instance of "his own ODataPathProvider", but it's not supported yet.

[FixRM]

I see. What other options do I have? I was thinking of filtering while loading XML nodes, but it seems tricky and my VS is dying when I try to load original file into it

[xuzhg]

1. use the Capabilities annotation, but this way will change your CSDL
2. I expose the Provider at the setting for you to set. Need some time to implement it.

[darrelmiller]

If we removed examples as proposed here #68 it would help to reduce the document size.

Also, yaml produces a much more compact format than JSON.

[darrelmiller]

@FixRM I think your best option would be to run the CSDL through an XSLT transform first to cut out the parts of the CSDL you don't want to process. We do some preprocessing of our CSDL this way using this approach https://github.com/microsoftgraph/msgraph-metadata/blob/master/transforms/csdl/preprocess_csdl.xsl

[FixRM]

@darrelmiller, thank you for your suggestion. Can you please help me with the transform?

My CSDL looks like this:

<edmx:Edmx Version="4.0">
    <edmx:Reference Uri="http://vocabularies.odata.org/OData.Community.Keys.V1.xml">
    <edmx:Include Namespace="OData.Community.Keys.V1" Alias="Keys"/>
    <edmx:IncludeAnnotations TermNamespace="OData.Community.Keys.V1"/>
</edmx:Reference>
<edmx:Reference Uri="http://vocabularies.odata.org/OData.Community.Display.V1.xml">
    <edmx:Include Namespace="OData.Community.Display.V1" Alias="Display"/>
    <edmx:IncludeAnnotations TermNamespace="OData.Community.Display.V1"/>
</edmx:Reference>
<edmx:DataServices>
    <Schema Namespace="Microsoft.Dynamics.CRM" Alias="mscrm">
        <EntityType Name="crmbaseentity" Abstract="true"/>
        <EntityType Name="accountleads" BaseType="mscrm.crmbaseentity">
                    ***

I started with this one, but it generates same document.

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  
  <xsl:output method="xml" indent="yes"/>
  <xsl:template match="@* | node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <xsl:template match="Schema/EntityType[not(@Name='account')]" />
  
</xsl:stylesheet>

What am I doing wrong?

[FixRM]

It was namespace problem. Correct transform:

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx"
                xmlns:edm="http://docs.oasis-open.org/odata/ns/edm"
                xmlns="http://docs.oasis-open.org/odata/ns/edm">
  <xsl:strip-space elements="*"/><!-- Remove empty space after deletions. -->
  <xsl:output method="xml" indent="yes"/>
  <xsl:template match="@* | node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <!--Remove unused entities-->
  <!--<xsl:template match="edm:Schema/edm:EntityType[not(@Name='account')]" />-->

  <!--Remove unused entity sets-->
  <xsl:template match="edm:Schema/edm:EntityContainer/edm:EntitySet[not(@Name='accounts')]" />

  <!--Remove unused actions-->
  <!--<xsl:template match="edm:Schema/edm:Action" />-->

  <!--Remove unused types-->
  <!--<xsl:template match="edm:Schema/edm:ComplexType" />-->

  <!--Remove unused functions-->
  <!--<xsl:template match="edm:Schema/edm:Function" />-->

  <!--Remove unused enums-->
  <!--<xsl:template match="edm:Schema/edm:EnumType" />-->  

</xsl:stylesheet>

[bkwdesign]

@FixRM My design is that you can derive from "ODataPathProvider" and override this virtual method. However, "ODataPathProvider" is used in the ODataContext, that's an internal class.

I'd like to expose an interface in the setting class so developer can create an instance of "his own ODataPathProvider", but it's not supported yet.

I also have an EDMX that produces over 6400 paths- and 1800+ schemas -my swagger result is only 17 MB, but I’m also interested in ways to pass in a schema-of-interest and have the rest ignored