Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WCF Connected Service - Base class properties are not deserialized #3073

Closed
sameera opened this issue Jul 31, 2018 · 10 comments
Closed

WCF Connected Service - Base class properties are not deserialized #3073

sameera opened this issue Jul 31, 2018 · 10 comments
Assignees
Labels
tooling
Milestone

Comments

@sameera
Copy link

@sameera sameera commented Jul 31, 2018

We've reported a number of issues working with NetSuite WebServices in .NET Core. We have uncovered another one (which may apply to other SOAP Webservices as well).

WSDL: https://webservices.netsuite.com/wsdl/v2017_2_0/netsuite.wsdl
See for generated Reference.cs file: Reference.zip

We are making a call such as the following to retrieve a TransactionBodyCustomField in the reponse:

var customFieldResponse = await client.getListAsync(new[] { new CustomizationRef {
                internalId = "57",
                type = RecordType.transactionBodyCustomField,
                typeSpecified = true
            } });

(The above code uses our wrapper library and I've used that for brevity. The request side is fine anyway, as it makes a valid WS call and NetSuite sends back a valid response).
NetSuite WS sends back a valid response which has scriptId and fieldType values set. See:
Webservice.response.xml.txt

But the deserialized value assigned to customFieldResponse has both these fields unassigned.
From the References.cs, we see that TransactionBodyCustomField is derived from CustomFieldType and both the above fields are defined in CustomFieldType class. We see that CustomFieldType has the required XmlInclude attributes to ensure that these fields deserialize.

We can also see that if we just serialize and deserialize a TransactionBodyCustomField as in the following code, we get desired results:

            var tbcf = new TransactionBodyCustomField {
                scriptId = "the_script_id",
                fieldType = CustomizationFieldType._date,
                fieldTypeSpecified = true
            };
            var sb = new StringBuilder();
            using (var writer = new StringWriter(sb))
            {
                new XmlSerializer(typeof(TransactionBodyCustomField)).Serialize(writer, tbcf);
            }

            using (var reader = new StringReader(sb.ToString()))
            {
                var result = new XmlSerializer(typeof(TransactionBodyCustomField)).Deserialize(reader);
            }

So, this seems to be an issue only on the WCF desrialization and not in the proxy classes themselves. Is there a way around this? A production customer reported these issues and we need to implement a quick fix soon (even if that's manually modifying the References.cs).

@sameera
Copy link
Author

@sameera sameera commented Jul 31, 2018

@mlacouture @hongdai If you could help?

@mlacouture
Copy link
Member

@mlacouture mlacouture commented Jul 31, 2018

Hi @sameera!
Thank you for reporting this issue. It is hard to tell what's going wrong from the provided information, this requires some investigation. Unfortunately, due to our current priorities, I doubt we will be able to look into this issue in this sprint. @mconnew, does this ring any bells?

@mlacouture mlacouture self-assigned this Jul 31, 2018
@mlacouture mlacouture added this to the S140 milestone Jul 31, 2018
@mconnew
Copy link
Member

@mconnew mconnew commented Jul 31, 2018

Your code snippet is serializing and deserializing with XmlSerializer so it will be consistent. If you output the contents of the string builder you will see that scriptId and fieldType are serialized first. XmlSerializer is doing the correct thing according to the schema in your WSDL. Here's a stackoverflow post where someone is asking a similar thing. Base type data comes first. Basically your web service isn't returning data which matches the schema in your WSDL and the response would actually fail schema validation against your WSDL.

@sameera
Copy link
Author

@sameera sameera commented Aug 1, 2018

@mconnew I think what you are saying is that since this webservice isn't returning the result properties in the order it defines in the WSDL, this is failing. And the reason why this works in XMLSerializer serialized-deserialized form is because the XmlSerializer respects the ordering when it serializes.

If the same WS is imported via Service Reference in a .net-full project (back when we were on .net-full), it works without an issue. It also used to work with classes generated by dotnet-svcutil v0.5
The difference is perhaps that the newer version of dotnet-svcutil is adding the Order property of the XmlElementAttribute. Is this property absolutely necessary?
If I remove the Order property manually, I get an error saying it's mandatory.

While I understand that this probably a mistake on the side, this is a third party webserivce that has existed for years; working well with .net and .net core (up to 1.1 afaik). Couldn't the Order requirement be made optional so that we can continue to work with these "legacy" webservices?

@mconnew
Copy link
Member

@mconnew mconnew commented Aug 1, 2018

The order property is added because in the WSDL it's defining a <Sequence> in the schema. This explicitly says items must appear in the order declared. I can't remember it off the top of my head but there's a different way to list the child elements where you aren't saying the order is defined. I can't speak to what a previous version was doing as we copied that code over as a basis. Is it possible the WSDL was different then? The order property is there because the WSDL says it should be. Adding options to svcutil to ignore certain aspects of the WSDL is a slippery slope which could result in svcutil having 100 switches to modify the output.
Are you aware that you don't have to use the output of svcutil without modification? In fact, you don't need to use svcutil at all if you choose not to, you can provide any valid service contract to WCF, we don't require it to be tool generated. You can edit the generated contract to remove the Order property and is likely the correct thing to do in this case as the web service and the WSDL don't 100% match. Another option might be to modify the ServiceContract on the ChannelFactory endpoint description before opening the channel factory, but unless you are updating the service contract regularly, that's the harder way to do things.

@sameera
Copy link
Author

@sameera sameera commented Aug 2, 2018

@mconnew

I can't speak to what a previous version was doing as we copied that code over as a basis. Is it possible the WSDL was different then?

Here's a screenshot of the diff of files generated by dotnet-svcutil 0.5 and 1.0.
image

Are you aware that you don't have to use the output of svcutil without modification?

Yes, we already "abuse" it to achieve different goals :) We are fine modifying the file and perform the required workaround.

The order property is added because in the WSDL it's defining a in the schema. This explicitly says items must appear in the order declared.

Yes, I totally understand that svcutil isn't at fault here and that it is the webservice. My rationale is that there are these well-known, widely used, 3rd party webservices that don't conform to these standards for whatever reason. And they have existed and been used, because the tooling had not been strict on something like this up to now. Does the webservice consumer gains any advantage in introducing the strict rules now?

@mconnew
Copy link
Member

@mconnew mconnew commented Aug 2, 2018

You are comparing an earlier version of the .Net Core svcutil with the latest version. I believe it was a bug not putting the Order property in the generated source which was later fixed as that is what the WSDL indicates should happen. You got lucky that incorrect behavior caused by a bug happened to make things work with the service which produces output which doesn't conform to the WSDL. The "correct" thing would be to get the service owner to either correct their WSDL to match their service or modify their service to match their WSDL, but realistically I understand that's out of your control.
I would be very surprised if add service reference or the full .Net Framework version of svcutil didn't add the Order property as there as some scenarios where excluding that could break communication (but that would be with DataContractSerializer, not XmlSerializer). @mlacouture, can you verify what the behavior is using svcutil from the full .Net Framework sdk and the older Add Service Reference as we should make sure we do the same as them.

@mlacouture
Copy link
Member

@mlacouture mlacouture commented Aug 2, 2018

I run the provided wsdl on the 0.5.0.0 version of the tool and got the Order attribute applied. I would guess the service metadata changed since then.

[System.CodeDom.Compiler.GeneratedCodeAttribute("dotnet-svcutil", "0.5.0.0")]
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.Xml.Serialization.XmlTypeAttribute(Namespace="urn:faults_2017_2.platform.webservices.netsuite.com")]
public partial class SoapFault
{
    private FaultCodeType codeField;
    private string messageField;
    
    /// <remarks/>
    [System.Xml.Serialization.XmlElementAttribute(Order=0)]
    public FaultCodeType code

@mlacouture mlacouture modified the milestones: S140, S139 Aug 2, 2018
@sameera
Copy link
Author

@sameera sameera commented Aug 3, 2018

@mlacouture That's quite intriguing. Thank you both for looking in to this. While we do have a workaround, I'd still like to understand this issue.

svcutil had upgraded to v1 by the time 2017.2 WSDL was released. So we didn't get to run it with that version. The diff was for 2016.2 version which we had built with v0.5. Since the WSDL could have changed and we don't have a cached copy of 2016.2, I went looking for a WSDL version that we do have cached. And that was 2015.1 (https://webservices.netsuite.com/wsdl/v2015_1_0/netsuite.wsdl)

The stub we have for 2015.1 was generated by adding a "Web Reference" back in VS 2013 days (IIRC). Again no ordering.
The actual types that are causing us a problem, are the extensions of customFieldType. These include TransactionBodyCustomField, EntityCustomField etc.
These are defined in the customizations.xsd (e.g. https://webservices.netsuite.com/xsd/setup/v2017_2_0/customization.xsd)

Comparing the cached XSD of 2015.1 (attached) to the live 2016.1 and 2017.2, I only see one difference (the presence of mixed=true). And I see that the live version of this file, no longer has this attribute. And sure enough, if I run svcutil on it now, it generates with ordering.

Is this assessment correct? Is the removal of mixed=true on the WS-end causing this?

2017-1 and 2015-1.zip

@Lxiamail Lxiamail added the tooling label Aug 3, 2018
@sameera
Copy link
Author

@sameera sameera commented Aug 11, 2018

Closing as I think I've found a reasonable explanation to the problem.

@sameera sameera closed this as completed Aug 11, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tooling
Projects
None yet
Development

No branches or pull requests

4 participants