Support for complexType with sequnce of any

[MudrakIvan]

Feature Request

Currently, if type have an complexType with sequence of any, empty class is generated. This leads data loss, as the content of this element is not converted. Unfortunately I was unable to make the conversion automatically via TypeConverters.

Q	A
New Feature	yes
RFC	no
BC Break	no

Summary

To replicate this behaviour, following wsdl can be used:

<?xml version="1.0" encoding="UTF-8"?>
<definitions xmlns="http://schemas.xmlsoap.org/wsdl/"
    xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
    xmlns:tns="http://example.com/customerdetails"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    targetNamespace="http://example.com/customerdetails"
    name="CustomerDetailsService">

    <!-- Data Types -->
    <types>
        <schema xmlns="http://www.w3.org/2001/XMLSchema"
            targetNamespace="http://example.com/customerdetails">
            <element name="GetCustomerDetailsRequest">
                <complexType>
                    <sequence>
                        <element name="customerId" type="xsd:string" />
                        <element name="countryCode" type="xsd:string" nillable="true" />
                    </sequence>
                </complexType>
            </element>
            <element name="GetCustomerDetailsResponse">
                <complexType>
                    <sequence>
                        <element name="customerName" type="xsd:string" />
                        <element name="customerEmail" type="xsd:string" />
                        <element name="customerData">
                            <complexType>
                                <sequence>
                                    <any processContents="lax" />
                                </sequence>
                            </complexType>
                        </element>
                    </sequence>
                </complexType>
            </element>
        </schema>
    </types>

    <!-- Message Definitions -->
    <message name="GetCustomerDetailsRequestMessage">
        <part name="parameters" element="tns:GetCustomerDetailsRequest" />
    </message>
    <message name="GetCustomerDetailsResponseMessage">
        <part name="parameters" element="tns:GetCustomerDetailsResponse" />
    </message>

    <!-- Port Type (Abstract Interface) -->
    <portType name="CustomerDetailsPortType">
        <operation name="GetCustomerDetails">
            <input message="tns:GetCustomerDetailsRequestMessage" />
            <output message="tns:GetCustomerDetailsResponseMessage" />
        </operation>
    </portType>

    <!-- Binding (Concrete Implementation) -->
    <binding name="CustomerDetailsBinding" type="tns:CustomerDetailsPortType">
        <soap:binding style="document" transport="http://schemas.xmlsoap.org/soap/http" />
        <operation name="GetCustomerDetails">
            <soap:operation soapAction="http://example.com/GetCustomerDetails" />
            <input>
                <soap:body use="literal" />
            </input>
            <output>
                <soap:body use="literal" />
            </output>
        </operation>
    </binding>

    <!-- Service Definition -->
    <service name="CustomerDetailsService">
        <documentation>This service provides customer details based on customer ID.</documentation>
        <port name="CustomerDetailsPort" binding="tns:CustomerDetailsBinding">
            <soap:address location="http://example.com/customerdetails/service" />
        </port>
    </service>
</definitions>

In this example, customerData will be generated as a class with no properties. Would it be possible to store those data like \DOMDocument or like a string, so the content is not lost?

[veewee]

Hello @MudrakIvan,

Not sure what to do with this at the moment:

• any means it can be pretty much anything.
• It could contain multiple nodes given maxOccurs
• it can be combined with other properties

This is perfectly valid XSD and XML wise, but it just does not correspond to PHP:

<xs:element name="person">
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"> 
<xs:element name="person">
<xs:complexType>
    <xs:sequence>
      <xs:element name="firstname" type="xs:string"/>
      <xs:element name="lastname" type="xs:string"/>
      <xs:any minOccurs="0" maxOccurs="100" processContents="lax" />
    </xs:sequence>
  </xs:complexType>
</xs:element> 
</xs:schema>

<person>
  <firstname>Jos</firstname>
  <lastname>Bos</lastname>
  <hello />
  <world />
</person>

One could create a class Person with dynamic props for this, but there is not really a way to trustworthy fill up this object given the data is not known by the schema. So it feels a bit beyond the scope of this package : generating code based on the known parts of the schema.

In this specific situation, I'dd create a complexTypeEncoder for the http://example.com/customerdetails:customerData type (or the wrapping GetCustomerDetailsResponse type). This way you have full control on how you would like to parse and validate the data. That way you are in full control on how the data is being parsed, since you as implementator of the soap service know best what is going on in your implementation.

Parsing it could be as easy as using something like https://github.com/veewee/xml/blob/3.x/docs/encoding.md

[MudrakIvan]

Hello @veewee,

I completely understand your points however when the any is used as you shown, it would be required to decode elements with known type (even nested elements) manualy. For this specific case I like the approach SVCutil.exe (C#), where if any can occur multiple times, then array of XmlElement is generated and when the any is only type of complexType only one XmlElement is generated. This way the soap response/request can be decoded and encoded validly without the need of making custom complexTypeEncoder.

[veewee]

@MudrakIvan

Don't get me wrong : I like the idea of having it in here. It's just a lot of work and hard to implement:

• goetas/xsd-reader does not support any elements: https://github.com/goetas-webservices/xsd-reader/blob/51558f69e61e75caa32a196eeeb23345ebcdea23/src/SchemaReader.php#L459-L503
• Once it does have that, we need to find a way to specify it on our internal metadata types as well. The easiest will probably adding some variadic type property or something in that lines so that it can also deal with min/max occurs etc which allows to have previous exactly defines properties as well.
• Once our metadata knows about this any type, we need to find a way to encode / decode the data. In your example it's pretty straight forward to do - but in my example the xml could become very weird. For example: if firstname and lastname are known elements, the any elements can still contain one or more firstname and lastname tags. So the system needs to be smart enought to deal with this.

<person>
  <firstname>Jos</firstname>
  <lastname>Bos</lastname>
  
  <!-- Start of any -->
  <hello />
  <world />
  <firstname>Jos</firstname>
  <lastname>Bos</lastname>
  <firstname>Jos</firstname>
  <lastname>Bos</lastname>
</person>

• Next up is code generation : we need to find a way to generate code for this. Given my example, this will probably result in some internal data storage prop

class Person {
    private string $firstname;
    private string $lastName;
   
    use DynamicDataStorageTrait;
}

trait DynamicDataStorageTrait {
    private array $__dynamic_any_storage__veryuniquehashtomakesureitdoesnotexistinwsdl;

   // Some easier to use accessors:
    public function storeDynamicData(array $data): self  {/*...*/}
    public function fetchDynamicData(): array  {/*...*/}
}

As you can tell, it won't be easy to do so.
Any help here is highly appreciated!

---

Now given your initial problem : It's pretty easy to parse this data by using a custom complexType encoder:

use Soap\Encoding\Xml\Node\Element;
use function VeeWee\Xml\Encoding\xml_decode;
use function VeeWee\Xml\Encoding\document_encode;


$registry->addComplexTypeConverter(
    'http://example.com/customerdetails',
    'customerData',
    new class implements XmlEncoder {
        public function iso(Context $context): VeeWee\Reflecta\Iso\Iso
        {
            $typeName = $context->type->getName();

            return new Iso(
                to: static fn(array $data): string => document_encode([$typeName => $data])->stringifyDocumentElement(),
                from: static fn(Element|string $xml) => xml_decode(
                    ($xml instanceof Element ? $xml : Element::fromString($xml))->value()
                )[$typeName],
            );
        }
    }
);

<x:GetCustomerDetailsResponse xmlns:x="http://example.com/customerdetails">
    <customerName>John Doe</customerName>
    <customerEmail>john@doe.com</customerEmail>
    <customerData>
        <foo />
        <bar />
        <hello>world</hello>
    </customerData>
</x:GetCustomerDetailsResponse>

Result in:

^ {#1761
  +"customerName": "John Doe"
  +"customerEmail": "john@doe.com"
  +"customerData": array:3 [
    "foo" => ""
    "bar" => ""
    "hello" => "world"
  ]
}

For your specific case, it's rather easy to get the data parsed manually than going through all the steps above.
That's why I mentioned it might be something to considered to be a manual action.

[MudrakIvan]

Thanks for you effort @veewee. I understand your points and I have to admit it's quite an unusual case.

Before I close this issue, I wanted to ask, if you now some way to to decode xml with xsd definition to a php class. To be exact this entire problem is that I'm dealing with service that returns multiple different schemas (which are send as element with complexType any) and have to parse them based on another element values.

[veewee]

@MudrakIvan

No need to close this issue. Lets keep it open so that we can resolve the core of the issue some day eventually :)

There are 2 approaches:

1. Manual

You can add a XSD schema through the encoding component through a loader to validate the content of the XML.
On top of that, you can tell the decoder on how to parse the data inside the XML by using PSL types:

https://github.com/veewee/xml/blob/3.x/docs/encoding.md#typed

Example:

use function Psl\Type\int;
use function Psl\Type\shape;
use function Psl\Type\string;
use function Psl\Type\vector;
use function VeeWee\Xml\Dom\Configurator\validator;
use function VeeWee\Xml\Dom\Validator\xsd_validator;
use function VeeWee\Xml\Encoding\typed;

$data = typed(
    <<<EOXML
        <root>
           <item>
               <id>1</id>
               <name>X</name>
               <category>A</category>
               <category>B</category>
               <category>C</category>
           </item>     
        </root>
    EOXML,
    shape([
        'root' => shape([
            'item' => shape([
                'id' => int(),
                'name' => string(),
                'category' => vector(string()),
            ])
        ])
    ]),
    validator(xsd_validator('some-schema.xsd'))
);

Besides regular shapes, you can map it to classes directly by using the converted type:
https://github.com/azjezz/psl/tree/next/src/Psl/Type#converted

2. Automatic mapping by using the encoding package

Since the https://github.com/php-soap/encoding is already using the XSD(s) as the source of truth for encoding and decoding, you could do something like this: (❗ untested)

use GoetasWebservices\XML\XSDReader\SchemaReader;
use Soap\Encoding\Encoder\Context;
use Soap\Encoding\EncoderRegistry;
use Soap\Engine\Metadata\Collection\MethodCollection;
use Soap\Engine\Metadata\InMemoryMetadata;
use Soap\WsdlReader\Metadata\Converter\SchemaToTypesConverter;
use Soap\WsdlReader\Metadata\Converter\Types\TypesConverterContext;
use Soap\WsdlReader\Parser\Definitions\NamespacesParser;
use VeeWee\Xml\Dom\Document;

$yourXsd = Document::fromXmlFile('your.xsd');
$namespaces = NamespacesParser::tryParse($yourXsd);
$metadata = new InMemoryMetadata(
    $types = (new SchemaToTypesConverter())(
        (new SchemaReader())->readNode($yourXsd->locateDocumentElement()),
        TypesConverterContext::default($namespaces)
    ),
    new MethodCollection()
);
$registry = EncoderRegistry::default()
    ->addClassMap('http://somenamespace', 'someObject', YourClass::class);


$context = new Context($types->fetchFirstByName('someObject')->getXsdType(), $metadata, $registry, $namespaces);
$encoder = $registry->detectEncoderForContext($context);
$data = $encoder->iso($context)->from($theRawXmlString);

You could theoretically even use this similar approach to let this package generate the PHP type code for you.

[MudrakIvan]

The automatic mapping with the new encoding package works like a charm. Even the PHP class generator works after only some minor modifications.

Thanks again @veewee.

[veewee]

FYI:

Provided a PR for the first part of the process: Grabbing the information from the XSD: goetas-webservices/xsd-reader#86.

Next up is using this information to enhance the metadata and find a way to generate code from it.