Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import System.Runtime.Serialization from referencesource. #1736

Merged
merged 45 commits into from Apr 24, 2015

Conversation

atsushieno
Copy link
Member

This time it is really ready to be imported.

It builds on net_4_5 and monodroid so far.

We still need to eliminate dynamic code generation to make it work on iOS.
We don't use neither of expression trees nor SRE anyways, but it's easier
to reduce noises in nunit test failures to review the rest.
…ion stubs.

The new code is to implement reflection (non-emit) based serialization
so that it works on iOS.

Actually I _totally_ switched to this new implementation because dynamic
method based implementation also doesn't work fine, even on desktop
(due to invocation failure on internal methods, it seems that our dynamic
method needs some bugfixes).

The new methods with NotImplementedException in new classes need to be
implemented.
Though it's still buggy and shows several nunit test failures yet.
It was inconsistent with what referencesource codegen does.
…ary.

Collection serialization was mostly failing, when it was non-dictionary.

Now it should be closer to referencesource dynamic code behavior.
Array elements were not deserialized correctly and causing null (default)
items put into the array.
… interpreter.

Basically the primary cause of the problem was that invoked primitive array
reader methods via reflection had 'out' parameter, which needed special care.
…ce codegen.

The original code is complicated enough that I misinterpreted the logic...

The reulting code should look like this:

	DateTime? expr_94 = timeSubmitted;
	DateTime dateTime;
	bool arg_B5_0;
	if (XmlObjectSerializerWriteContext.GetHasValue<DateTime> (expr_94)) {
		dateTime = XmlObjectSerializerWriteContext.GetNullableValue<DateTime> (expr_94);
		arg_B5_0 = false;
	}
	else {
		dateTime = XmlObjectSerializerWriteContext.GetDefaultValue<DateTime> ();
		arg_B5_0 = true;
	}

The fixed code should be like this now.
The removed code caused unexpected omission of xsi:type by giving "false"
Type information that is from the actual value (when the expected type is
different, xsi:type should be written).
…havior.

It is likely that the second contract conflicts with the existing one and
WCF renames it to avoid that.
WriteStartDocument() can be used with document and document does not accept
more than one element, which basically contradicts MTOM writer usage.

For EOL changes, MTOM writer should not alter content EOL chars.
They don't pass all the tests in System.ServiceModel.Web yet.
…ep it open.

Otherwise MemoryStream.Position complains that it is already disposed.
Unlike our own XsdDataContractExporter, referencesource one generates
an extra schema that targets xs:* (XmlSchema.Namespace).
That should not be generated as part of WSDL, so exclude it.
…xmlns-es.

The namespace declarations existed when the entire Message is read but not
in the partial body contents. They caused regressions when we use
serialization stack from referencesource.

It is possible that more namespaces may be requied, but we will be importing
System.ServiceModel later on and then the issue will go away.
So far we don't want regression as long as they show up in our NUnit tests.
…re]d.

.NET WCF JSON serializer has been known to be buggy...
…ET IN JST.

Those tests were added without considering TimeZone difference. They cause
several kind of errors (SerializationException etc.) in JST (+09:00) or any
timezone that is earlier than UTC.

We likely had some "working" implementation, but was incompatible with .NET.
…sibility.

I wonder why, but when switched to referencesource, it turned out that
ResourceCollectionInfo calls SyndicationElementExtensionCollection.Add()
which targets a private overload, but (of course) failed and resolved to
another overload that takes object, and thus resulting in invalid code path.

With this fix the call to Add() resolves to the expected one and does not
regress with referencesource import.
JSON array is deserialized as Array, not List<T>.
Just like BinaryMessageFormatter needed to use another CreateBinaryWriter()
overload to NOT close the input stream, use CreateTextWriter() overload that
does not close the stream.
WS-Discovery never got working, skip cosmetic test to not block porting.
atsushieno added a commit that referenced this pull request Apr 24, 2015
Import System.Runtime.Serialization from referencesource.
@atsushieno atsushieno merged commit b839d14 into mono:master Apr 24, 2015
@akoeplinger
Copy link
Member

Hey @atsushieno, I just traced back a crash I'm seeing in the System.Runtime.Serialization testsuite to this PR. It crashes consistently with a SIGSEGV after showing the test results on Linux (amd64/Ubuntu 14.04):

...
***** MonoTests.System.Xml.XmlSimpleDictionaryWriterTest.XmlSpaceTestInvalidValue3
***** MonoTests.System.Xml.XmlSimpleDictionaryWriterTest.XmlSpaceTestInvalidValue4

Tests run: 413, Failures: 0, Not run: 2, Time: 5.216 seconds


Stacktrace:


Native stacktrace:

    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono() [0x4ec908]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono() [0x573980]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono() [0x419ed6]
    /lib/x86_64-linux-gnu/libpthread.so.0(+0xf0a0) [0x2b9c619cc0a0]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono(mono_metadata_generic_class_is_valuetype+0xf) [0x613291]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono(mono_type_is_reference+0x6a) [0x616791]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono() [0x5f8a66]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono() [0x74c9da]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono() [0x60ac3e]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono() [0x60ff6f]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono() [0x74cbdc]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono() [0x60fe4a]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono() [0x5e4e0b]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono() [0x5a9736]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono(mono_domain_free+0x3c9) [0x678be7]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono() [0x675ed2]
    /var/lib/jenkins/workspace/test-mono-mainline@2/label/debian-amd64/mono/mini/mono() [0x747212]
    /lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50) [0x2b9c619c3b50]
    /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x2b9c61cb495d]

Debug info from gdb:


=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

Aborted
make[10]: *** [run-test-lib] Error 1

Is this something you're seeing on your end as well?

@atsushieno
Copy link
Member Author

I see it as well. That's a runtime issue that the runtime team has to fix.

@kumpera
Copy link
Contributor

kumpera commented Jun 1, 2015

Can we revert this in the meanwhile?

@atsushieno
Copy link
Member Author

Do you really understand that you are going to revert EVERYTHING? You should consult @migueldeicaza on that.

@kumpera
Copy link
Contributor

kumpera commented Jun 1, 2015

There's will be plenty left if we revert this.
This is blocking master from been green and, given the author is not working on addressing it
I rather revert until the issue is resolved.

@atsushieno
Copy link
Member Author

Let me rephrase: you should consult @migueldeicaza. It is HUGE revert.

@kumpera
Copy link
Contributor

kumpera commented Jun 2, 2015

I fail to see the point of consulting him on such a simple technical decision as I'd be asking "Can Atsushi and I discuss and make a decision on System.Runtime.Serialization?". We're empowered to make this sort of decision, so let's do it!

What I'm trying to figure out is if it can be reverted until we fix the issue and then reapplied. That's all, I'm asking for your technical assessment on it. Since we have a merge commit, reverting/reapplying this as whole would, in theory, be easy.

Would a revert of this PR fix the crash? IOW, would it leave us with a mono as good as what
we had before it was merged?

If reverting turns to be complicated let's keep it in. But if it's super easy I think it's worth considering temporarily doing it.

@vargaz
Copy link
Contributor

vargaz commented Jun 2, 2015

I think the problem here is the crash, which will still be in the runtime even if we revert this, it just won't appear.

@marek-safar
Copy link
Member

@akoeplinger do you think you could get repro for the crash?

@akoeplinger
Copy link
Member

@marek-safar just run the testsuite on Linux, it is 100% consistent (and @atsushieno confirmed he sees it as well on his end).

@atsushieno
Copy link
Member Author

There are several reasons I find reverting this is stupid:

  • it is runtime's fault, not this class library's fault.
  • it does not show any regression in Xamarin product i.e. Linux environment specific issue.
  • the only recognized problem is the runtime crash that runs NUnit test runner, not System.Runtime.Serialization feature.

The negative impact of reverting this change is that there are not a few Xamarin customers who are looking for the referencesource merge and we kept telling them that it will be fixed in the next major release. If there is important reason that we have to revert it, then they would find it reasonable. The reason you want to revert this is just for nominal green build, which won't persuade Xamarin customers.

@vargaz
Copy link
Contributor

vargaz commented Jun 3, 2015

#1779 seems to fix this.

@migueldeicaza
Copy link
Contributor

Hey guys, I take it there was no need to revert this since #1779 was applied?

@kumpera
Copy link
Contributor

kumpera commented Jun 19, 2015

#1779 has not been merged, that PR was still under work last time I checked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants