Please provide examples of how to resolve conflicts in type names

[alexinsmail]

This is probably a repetition of other people’s topics. I think I’m not the first one to have encountered such problems.

Cargo.toml

[dependencies]
quick-xml = "0.42.0"
xsd-parser = "1.7.0"
xsd-parser-types = { version = "0.4.0", features = ["quick-xml"] }

[build-dependencies]
xsd-parser = "1.7.0"

Let’s consider an example:

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns="http://example.com/base/1" 
    xmlns:base="http://example.com/base/1" 
    xmlns:xs="http://www.w3.org/2001/XMLSchema" 
    targetNamespace="http://example.com/base/1" 
    elementFormDefault="qualified" 
    attributeFormDefault="unqualified">

	<xs:simpleType name="boNumberType">
		<xs:restriction base="xs:string">
			<xs:minLength value="1"/>
			<xs:maxLength value="19"/>
		</xs:restriction>
	</xs:simpleType>
	<xs:simpleType name="BONumber">
		<xs:restriction base="xs:string">
			<xs:minLength value="1"/>

			<xs:maxLength value="19"/>
		</xs:restriction>
	</xs:simpleType>
</xs:schema>

ERROR xsd_parser: error: Generator error: Type name BoNumberType is generated for both {example.com/base/1}Type(BONumber) and {example.com/base/1}Type(boNumberType)! Use different module flags or a custom naming convention to tell them apart.

It’s not clear what needs to be done.
In theory, you can do it very simply:

config.generator.type_postfix.type_ = "".into();

1. "BONumber" => struct BONumber(String)
2. "boNumberType" => struct BONumberType(String)

By default, the configuration looks like this:

config.generator.type_postfix.type_ = Type".into();

It seems he solves it like this:

1. "BONumber" => struct BONumberType(String)
2. "boNumberType" => struct BONumberType(String)

The solver should have recognized the conflict and made a decision along these lines:

1. "BONumber" => struct BONumberType(String)
2. "boNumberType" => struct BONumberTypeType(String)

The proposal to use trait Naming looks interesting, but it’s not logical.
Sample from xsd-parser/examples/custom_naming.rs

let mut config = Config::default().with_naming(PreserveSeparatorsNaming);

#[derive(Debug, Clone, Default)]
struct PreserveSeparatorsNaming(Arc<AtomicUsize>);

impl Naming for PreserveSeparatorsNaming {
    ...

    fn unify(&self, s: &str) -> String {
        s.chars()
            .map(|ch| if ch.is_alphanumeric() { ch } else { '_' })
            .collect()
    }

    ...
}

The presented example does not solve the problem of recognizing which namespace the mutable type name belongs to.
We’ll stumble over this a bit later.

For a narrow set of types, you can define rules for generating type names.
For most types, a basic implementation based on the config.generator.type_postfix parameter is sufficient.
We don’t have the option to refer to the basic implementation of trait Naming, which is described in the config.generator.type_postfix parameters.

Rust is not an OOP language. We don’t have a super method call. It’s useful to have a good solution.

Example of the same name in different namespaces.

BaseTypes.xsd

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns="http://example.com/base/1" 
    xmlns:base="http://example.com/base/1" 
    xmlns:xs="http://www.w3.org/2001/XMLSchema" 
    targetNamespace="http://example.com/base/1" 
    elementFormDefault="qualified" 
    attributeFormDefault="unqualified">

	<xs:simpleType name="participantType">
		<xs:restriction base="xs:string">
			<xs:enumeration value="L"/>
			<xs:enumeration value="F"/>
		</xs:restriction>
	</xs:simpleType>

</xs:schema>

CommonType.xsd

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns="http://example.com/common/1" 
    xmlns:base="http://example.com/common/1" 
    xmlns:xs="http://www.w3.org/2001/XMLSchema" 
    targetNamespace="http://example.com/common/1" 
    elementFormDefault="qualified" 
    attributeFormDefault="unqualified">

    <xs:import namespace="http://example.com/base/1" schemaLocation="BaseTypes.xsd"/>

	<xs:complexType name="participantType">
		<xs:choice>
			<xs:element name="legalEntityInfo">...</xs:element>
			<xs:element name="legalEntityForeignStateInfo">...</xs:element>
		</xs:choice>
    </xs:complexType>
</xs:schema>

ERROR xsd_parser: error: Generator error: Type name ParticipantType is generated for both {http://example.com/base/1}Type(participantType) and {http://example.com/common/1}Type(participantType)! Use different module flags or a custom naming to tell them apart.

As we can see, the “unify” method doesn’t help here. Let’s combine the first example with the last example. The task turns out to be non‑trivial.
“Use different module flags.” Please provide an example of such a solution.

[kasparovabi]

Both names unify to BoNumberType. BONumber becomes BoNumber plus the Type postfix, and boNumberType already ends in Type, so it stays BoNumberType. They are in the same namespace, so GeneratorFlags::USE_MODULES can't separate them. The way out the README points to is giving one of the two types an explicit name, as in the custom_names example.

That needs the step-by-step pipeline instead of generate(), because the name is set on the MetaTypes between the interpreter and the optimizer. With your schema and xsd-parser 1.7.0 this works (it fits in build.rs the same way):

use anyhow::{anyhow, Error};
use quote::ToTokens;
use xsd_parser::{
    config::{Config, IdentQuadruple, Schema},
    exec_generator, exec_interpreter, exec_optimizer, exec_parser, exec_render,
    models::IdentType,
};

fn main() -> Result<(), Error> {
    let mut config = Config::default();
    config.parser.schemas = vec![Schema::File("schema.xsd".into())];

    let schemas = exec_parser(config.parser)?;
    let mut types = exec_interpreter(config.interpreter, &schemas)?;

    // Give one of the two colliding types its own Rust name.
    let ident = IdentQuadruple::from((IdentType::Type, "base:BONumber")).resolve(&schemas)?;
    types
        .items
        .get_mut(&ident)
        .ok_or_else(|| anyhow!("type base:BONumber not found"))?
        .display_name = Some("BoNumber".into());

    let types = exec_optimizer(config.optimizer, types)?;
    let data_types = exec_generator(config.generator, &schemas, &types)?;
    let module = exec_render(config.renderer, &data_types)?;
    println!("{}", module.to_token_stream());
    Ok(())
}

With plain generate(config) I get the same error you posted. With the rename the output ends with

pub type BoNumber = String ; pub type BoNumberType = String ;

The base: prefix comes from the xmlns:base declaration in your schema. You can also pass the namespace URI with NamespaceIdent::namespace(b"http://example.com/base/1"), which the example shows too.

Your type_postfix.type_ = "".into() idea also gets past the error (BoNumber and BoNumberType), but it drops the postfix everywhere. The XSD built-ins then come out as UnsignedLong, Int and so on instead of UnsignedLongType, IntType. If you only have a few collisions, renaming those types one by one keeps the rest of the generated names unchanged.

[alexinsmail]

Oh, I didn’t expect that.

   size      file name
-----------+---------------------   
 4_684_934   IntegrationTypes.xsd

 7_480_366   mod.rs
61_026_149   quick_xml_deserialize.rs
18_257_938   quick_xml_serialize.rs

Java Maven plugin creates JAXB code, and it is translated faster than the cargo build process.
My machine is overloaded and doesn’t finish translating all the modules.

The project contains 18 xsd files with a total size of more than 10 MByte.
The total size of the source files before formatting is 140 MByte.
After formatting, it’s 175 MByte.

I think that if the module consisted not of 3 files, but of several folders with nested files, I would wait for the code to be translated before restarting Linux.

[Bergmann89]

Hey @alexinsmail,

@kasparovabi already covered the first case (same namespace), so here's the rest.

Same name in different namespaces (participantType)

This is what the module flags in the error message mean. With USE_MODULES every namespace gets its own Rust module, so the two types end up as base::ParticipantType and common::ParticipantType and don't collide anymore:

use xsd_parser::config::GeneratorFlags;

let config = config.with_generator_flags(GeneratorFlags::USE_MODULES);

The module names come from the namespace prefixes. The duplicate check compares the full Rust path, so the error goes away as soon as the types sit in different modules. If you need one module per schema file instead of per namespace, there's also USE_SCHEMA_MODULES.

The two cases combine fine: USE_MODULES for the cross-namespace clashes, plus the display_name rename from the answer above for the clashes inside one namespace.

Reusing the default naming ("no super call")

You don't need inheritance for this. The default implementation is a public type, xsd_parser::models::Naming, so you can wrap it and only override what you need:

use xsd_parser::{models::Naming as DefaultNaming, traits::Naming};

#[derive(Debug, Clone, Default)]
struct MyNaming(DefaultNaming);

impl Naming for MyNaming {
    fn make_type_name(&self, postfixes: &[String], ty: &MetaType, ident: &TypeIdent) -> Name {
        // special rules for a few types, e.g. based on `ident` (it carries the namespace) ...
        self.0.make_type_name(postfixes, ty, ident) // ... and the default for the rest
    }

    // forward the remaining methods to `self.0` the same way
}

The ident you get in make_type_name contains the namespace, so namespace-specific rules can live there too. That said, for a handful of collisions, renaming the types directly is usually less work than a custom naming.

On auto-resolving like BONumberTypeType: we intentionally don't do that. Which type gets the suffix would depend on the order the types are processed in, so a small schema change could silently rename types in your API. An error you fix once with an explicit name is easier to keep stable.

Another small remark on Naming::unify: This use used only to unify the names for internal use before it is processed in the pipeline. As the doc comment says, the default implementation uses pascal case here. This is just used to make the actual naming generation (like make_type_name) easier.

Size and build time

This is a known trade-off: xsd-parser generates the schema checks directly into the code instead of interpreting the schema at runtime. Every type gets its own specialized serializer and deserializer, which makes reading and writing XML really fast, but it also makes the generated code big. For very large schema sets like yours this approach may simply not be the best fit.

If you want to keep going with it, some things that help:

• Only generate what you actually use: config.with_generate([(IdentType::Element, "tns:YourRootElement")]) drops every type that isn't reachable from the listed root elements. For big schema sets this is usually the biggest win.
• If you only read or only write XML, use with_quick_xml_deserialize() or with_quick_xml_serialize() instead of with_quick_xml().
• With USE_MODULES you can write the code to one file per module with module.write_to_files(dir) instead of a single file. That doesn't make rustc faster on its own, but it lets you move the generated code into a separate crate (or several crates) that cargo builds once and caches, instead of rebuilding it with your application.
• Formatting the generated code isn't needed to compile it, so skip rustfmt if you don't read the files. But there is also a PR (feat(generator): add type modules and optional formatting #318) that addresses this problem using prettyplease.

[alexinsmail]

@Bergmann89 Thank you for the detailed response.

Right now, I'm using this set of parameters

let mut config = Config::default().with_quick_xml();

config.parser.flags = ParserFlags::all();
config.parser.schemas = vec![ 
    get_schema_file("schema_a.xsd"),
    get_schema_file("schema_b.xsd"),
    ... 
    ];


config.generator.generate = Generate::Named;
config.generator.type_postfix.type_ = "".into();
config.generator.flags |= GeneratorFlags::FLATTEN_CONTENT
   | GeneratorFlags::USE_MODULES
   | GeneratorFlags::USE_SCHEMA_MODULES;

config.renderer.flags |= RendererFlags::RENDER_DOCS;
...
let modules = generate_modules(config)?;
modules
   .write_to_files(&target_dir)
   .context("Error while writing generated code")?;

I provided an example of processing one XSD file in the list.
Currently, the Vs Code and Zed editors freeze in “cargo check” mode.

I tried splitting large files using r2factor. This is the pierreaubert/r2factor project on GitHub.
The size of the camel_case files exceeded the file length limit in Linux.

XSD schemas contain named types and nested anonymous types.
In Java JAXB, a class object is created for each named type. Unnamed types are created as nested classes.
In Rust, there are no nested types. This behavior can be replaced with a single module. The module (the *.rs file) takes its name from the named type.

One line in the generated file xs/mod.rs is still killing me.

pub type String = String;

It seems this comes from here

pub const NS_XS : Namespace = Namespace :: new_const (b"http://www.w3.org/2001/XMLSchema") ;

[Bergmann89]

The pub type String = String; line comes from type_postfix.type_ = "": xs:string is now named String, which shadows the Rust String inside the xs module, so the alias points to itself and rustc fails with E0391: cycle detected. The same happens for every generated type whose name matches a Rust type the generated code uses.

Enable BUILD_IN_ABSOLUTE_PATHS, then the generator refers to built-in types by their full path and the alias becomes pub type String = ::std::string::String;:

config.generator.flags |= GeneratorFlags::BUILD_IN_ABSOLUTE_PATHS;

I've added a check in #327, that now throws an error if this happens and a suitable error message that guide the use what to do.

About the size: Generate::Named generates every named type of all 18 schemas, including everything you never use. If you only need a few root elements, config.with_generate([(IdentType::Element, "tns:YourRootElement")]) instead only generates the types reachable from them, which is usually by far the biggest reduction.

About one file per type: master now has GeneratorFlags::USE_TYPE_MODULES, which puts every type into its own module, and with write_to_files into its own file. It is not released yet and will be part of the next version.

[alexinsmail]

It would be much more interesting to specify a list of rootElement, or xpath, for which a set of objects needs to be generated.
Throw away everything unnecessary.
That would be a real fairy tale.

Honestly. There are a couple of xsds that describe elements which are information exchange documents.
The first one describes a set of incoming documents. The second one describes a set of outgoing documents.
All other xsds describe document structures.

For a specific task, a small set of documents is selected from the overall set of documents.

Build a dependency tree for the specified set of starting elements. Discard all unnecessary types.

Is it possible to implement such an idea in reality?

[Bergmann89]

That's exactly what with_generate already does. You pass the list of root elements, and the generator walks the dependency tree from there: every type that is reachable from these elements is generated, everything else is dropped.

In your config, config.generator.generate = Generate::Named; is what generates everything. Replace it with the list of documents you need:

let config = Config::default()
    .with_quick_xml()
    .with_generate([
        (IdentType::Element, "in:OrderRequest"),
        (IdentType::Element, "in:InvoiceRequest"),
        (IdentType::Element, "out:OrderResponse"),
    ]);

The prefixes (in, out) are the ones declared with xmlns:in="..." in your schemas. If the prefixes differ between your files, you can use the namespace URI instead:

use xsd_parser::config::{IdentQuadruple, NamespaceIdent};

let root = IdentQuadruple::from((IdentType::Element, "OrderRequest"))
    .with_ns(NamespaceIdent::namespace(b"http://example.com/incoming"));

You still pass all 18 schema files to the parser, so all references can be resolved, but only the types your selected documents actually use end up in the generated code. For a small set of documents out of a large schema set, this usually shrinks the output dramatically.

[alexinsmail]

Thanks for the help.

If anything, I’m still using

[dependencies]
xsd-parser = "1.7.0"
xsd-parser-types = { version = "0.4.0", features = ["quick-xml"] }

    let mut config = Config::default()
        .with_quick_xml()
        .with_generate([(IdentType::Element, "ext:confirmation")]);

    config.parser.flags = ParserFlags::all();
    config.parser.schemas = vec![
        get_schema_file(&cargo_dir, "schema/extegration.xsd")?,
    ];

Perhaps we should switch to building from GitHub.

The original xsd doesn’t define a unique name.
I added a namespace to the xsd:

<xs:schema xmlns="http://example.com/incoming/1" 
    ...
    xmlns:ext="{http://example.com/incoming/1}"
    targetNamespace="http://example.com/incoming/1" 
    elementFormDefault="qualified" attributeFormDefault="unqualified">

I’m trying to create it with and without config.with_generate.

config = config.with_generate([(
        IdentType::Element,
        "ext:confirmation",
    )]);

The "confirmation" component is a response of the type "success" or "error" + a list of error strings.
There are no dependencies on imported xsds. He described it within the current xsd file.

I’m comparing directories using the meld program on Linux. It shows that the files are the same.
Probably, I missed something important.

After the work, I see directories and *.rs files for all the listed schemas.
My expectations are only the specified object and nothing else.
Correct me if I’m wrong.

[alexinsmail]

I apologize. I figured out my code.

After with_generate, I reassigned the new config.generator.
I spent a long time cleaning up the example and writing short xsds.