Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency on specific Newtonsoft.Json package version = 9.0.1 #107

Closed
amit-kumar2 opened this issue Sep 5, 2017 · 86 comments
Closed

Dependency on specific Newtonsoft.Json package version = 9.0.1 #107

amit-kumar2 opened this issue Sep 5, 2017 · 86 comments

Comments

@amit-kumar2
Copy link

amit-kumar2 commented Sep 5, 2017

The latest version 1.0.2 of Microsoft.NET.Sdk.Functions has a specific Newtonsoft.Json package dependency = 9.0.1.
The alpha releases supported Newtonsoft.Json (>= 9.0.1).
This is causing a Json package version dependency mismatch in our solutions (between other referenced libraries and the functions project) and functions project fails to start since it cannot load 9.0.1 version.

As a workaround, we had to downgrade to 1.0.0-alpha6.
Are there any plans to support Newtonsoft.Json (>= 9.0.1) in the upcoming releases of Microsoft.NET.Sdk.Functions?

@kevbite
Copy link

kevbite commented Sep 6, 2017

I'm having the same problem, be nice to get this fixed.
See Stackoverflow Question for more details.

@pkanavos
Copy link

pkanavos commented Sep 6, 2017

This is caused by line 39 in csproj:

<PackageReference Include="Newtonsoft.Json" Version="[9.0.1]" />

Why require a specific Json.NET version? Why not :

<PackageReference Include="Newtonsoft.Json" Version="9.0.1" />

?

@BenWhite27
Copy link

Affected too, need to use Refit which has a dependency on v10.0.3

@brettsam
Copy link
Member

Hey @lindydonna / @ahmelsayed -- have heard that this can block some customers when building locally. They had to go back to the -alpha build in order to unblock. Is it possible to lift this restriction?

@ahmelsayed
Copy link
Contributor

/cc @fabiocav
I thought we added that restriction to fix an issue with people specifying a higher version then getting a runtime error that new methods are not available.

@fabiocav
Copy link
Member

That's correct. This, in the context of functions, prevents some runtime failures.

@brettsam
Copy link
Member

I think this is the only layer where we prevent this package from moving forward (i.e. -- the runtime lets you do it). In some cases, it seems to work. Since we don't restrict nugets anywhere else, should we relax it here as well?

Side note -- I agree that it's not a great solution either way. Just not sure if we shouldn't get out of people's way here.

@johlrich
Copy link

johlrich commented Oct 4, 2017

+1 for removing restriction since the other packages don't restrict. I personally haven't had any runtime issues with 10+ (to be fair it's been local development so far for this service).

For me this is an issue since Newtonsoft.Json 9 + F# 4.1 gives runtime exceptions when deserializing to FSharpList types.

@fabiocav
Copy link
Member

fabiocav commented Oct 4, 2017

Unfortunately, removing the restriction, in most cases, would just delay a problem that would now happen at runtime, and often in a way that is hard to identify.

@johlrich
Copy link

johlrich commented Oct 4, 2017

My only concern is since the other nuget packages don't have this restriction, it's still easy to get a hard to identify runtime scenario that wasn't really defined. Is it possible to share some details or links to the problems seen for those of us willing to brave on so we know what to look out for?

@kevbite
Copy link

kevbite commented Oct 5, 2017

https://github.com/dotnetsheff/dotnetsheff-api has this problem, it uses AlexaSkillsKit.NET/ which has a dependencies on

BouncyCastle (>= 1.8.1)
Microsoft.AspNet.WebApi.Client (>= 5.2.3)
Newtonsoft.Json (>= 7.0.1)

It also uses Refit which has dependencies on

Microsoft.AspNetCore.WebUtilities (>= 1.1.2)
NETStandard.Library (>= 1.6.1)
Newtonsoft.Json (>= 10.0.3)
System.Net.Http (>= 4.3.2)
System.Text.Encodings.Web (>= 4.4.0)

The only way around this we've found is to reference Refit@3.0.1 which has a dependency of Newtonsoft.Json (>= 9.0.1) then just copy the whole of AlexaSkillsKit.NET from github in to our project dotnetsheff.Api/AlexaSkill/AlexaSkillsKit.Lib

@BenWhite27
Copy link

Details on this runtime issue would be appreciated, I haven't had any issues so far.

@mcbridedm
Copy link

Possible to get an update on removing the hard dependency for version 9.0.1?

@mdsharpe
Copy link

I am struggling to set up a function that uses both Microsoft.NET.Sdk.Functions and Microsoft.Azure.WebJobs.ServiceBus because of this conflict on versions of Newtonsoft.Json.

@leftler
Copy link

leftler commented Oct 25, 2017

@mdsharpe I was running in to that issue too earlier today. I could not update Microsoft.Azure.WebJobs.ServiceBus to 2.1.0-beta4, however after removing the nuget package and re-adding it the update worked fine (be sure you have the pre-release box checked in the NuGet manager). Also make sure you remove all other packages that are now dependencies inside of Microsoft.NET.Sdk.Functions (Microsoft.Azure.WebJobs and friends)

@rposener
Copy link

+1 please fix!

@MovGP0
Copy link

MovGP0 commented Oct 27, 2017

i have the same issue here:

Warning	NU1608	Detected package version outside of dependency constraint: Microsoft.NET.Sdk.Functions 1.0.6 requires Newtonsoft.Json (= 9.0.1) but version Newtonsoft.Json 10.0.3 was resolved.	

@ali-kashanian
Copy link

Same here. Please fix this issue.
I closed issue #132 since it was a duplicate.

@Furynation
Copy link

Bumping as this is a major blocker.

@richhosek
Copy link

Was able to add the dependencies of the SDK.Function package separately and include the 10.0.3 version of NewtonSoft.Json, but reading about runtime issues has me worried. We have other Core packages that require Json >= 10.0.3. I too would like to hear more about the runtime issues and what needs to be avoided....

@tmgirvin
Copy link

+1
Been fighting this for a day trying to find various work-arounds. Please fix.

@fabiocav
Copy link
Member

This is locked on 9.0.1 by design to prevent failures at runtime that are often confusing and difficult to diagnose.

The Functions runtime currently binds to 9.0.1 and while some scenarios would work if there's a reference to a higher version, many of the binding scenarios would fail in ways that would cause confuse a large number of customers. Those scenarios are primarily where you'd need to bind to types from one of the assemblies we lock on (e.g. binding to JObject, CloudBlockBlob, etc.), and would typically lead to a type mismatch and failure to index the function.

@richhosek the issue above is the main thing to look for when referencing a higher version of Newtonsoft.Json. If the use of those types is primarily scoped to your function (doesn't involve passing or receiving object of those types to and from bindings), you should be OK.
We're investigating a couple of options to unblock users when this wouldn't cause an issue.

We do agree that although this is a behavior that protects some users against confusing runtime failures, it isn't optimal and does block valid scenarios. I had a quick chat with @ahmelsayed and we'll be exploring an approach to give us a good balance and unblock your scenarios.

We appreciate the patience and will keep this issue updated.

@richhosek
Copy link

richhosek commented Oct 31, 2017 via email

@rposener
Copy link

rposener commented Nov 1, 2017

@fabiocav thanks for the idea about how locking it in some perceptions is preventing confusing issues. Problem is that once you reference another project that requires >= 10.0.1 you get a build error and no way around that issue without doing exactly what you think you just prevented. The only work-around in that case is to explicitly add Newtonsoft.Json 10.0.1 or higher, which just forces this package to use it anyway. Locking to a specific lower level is not workable - specifically for a package like Newtonsoft.Json.

Does anyone have a list of what these supposed breaking changes from 9 -> 10? I'm not seeing them on the releases page. Perhaps @JamesNK can point this thread to that list?

@hoshauch
Copy link

hoshauch commented Nov 5, 2017

+1
Been fighting this for a day trying to find various work-arounds. Please fix.

@kevbite
Copy link

kevbite commented Nov 6, 2017

@hoshauch my workarounds were to just pull down any code that used Newtonsoft.Json and include it all in the project but target the same version of Newtonsoft.Json. Works but is a nasty hack.

@JamesNK
Copy link

JamesNK commented Nov 6, 2017

Does anyone have a list of what these supposed breaking changes from 9 -> 10? I'm not seeing them on the releases page. Perhaps @JamesNK can point this thread to that list?

All the items marked "change"

@rposener
Copy link

rposener commented Nov 6, 2017

Thanks @JamesNK! @fabiocav @hoshauch - can you tell us more about which of these specifically broke something for those of us forced to use 10.0.x versions? Maybe also what is broken?

Change - Removed .NET 4 portable class library target from NuGet package
Change - Removed obsolete JsonConverter.GetSchema method
Change - Removed obsolete constructor from DefaultContractResolver
Change - Removed obsolete async methods from JsonConvert
Change - Removed obsolete OnDeserialized, OnDeserializing, OnSerialized, OnSerializing, OnError from JsonContract
Change - Removed obsolete JsonDictionaryContract.PropertyNameResolver
Change - Removed obsolete ConstructorParameters, OverrideConstructor, ParametrizedConstructor from JsonObjectContract
Change - Obsoleted TypeNameAssemblyFormat properties on JsonSerializer and JsonSerializerSettings
Change - Obsoleted SerializationBinder
Change - Obsoleted Binder properties on JsonSerializer and JsonSerializerSettings
Change - Obsoleted FormatterAssemblyStyle in non-full .NET targets
Change - Obsoleted Newtonsoft.Json.Bson (moved to new NuGet package)
Change - Improved constructor parameter binding on .NET Core and portable builds
Change - Improved error when attempting to convert root JSON object property to an XML attribute
Change - Changed exception thrown when parsing invalid Unicode escape sequence to JsonReaderException
Change - Changed StringEnumConverter.AllowIntegerValues to also reject reading integer strings
Change - Changed StringEnumConverter.AllowIntegerValues to also reject writing enum values with no name

@mdsharpe
Copy link

mdsharpe commented Nov 6, 2017

I worked around this with a bindingRedirect. Still get warnings on build, but things seem to work.

@mcbridedm
Copy link

mcbridedm commented Dec 20, 2017 via email

@MovGP0
Copy link

MovGP0 commented Dec 20, 2017

@mcbridedm thank you! didn't know there is a template for Azure Functions / Core 2. So far I've tried a manual conversion of an existing project only. will check the template when I find some time.

@mcbridedm
Copy link

mcbridedm commented Dec 20, 2017 via email

@jansoren
Copy link

Could someone from the Azure community please look into/comment on the alpha-package suggestion? @ahmelsayed @vijayrkn

@mcbridedm
Copy link

mcbridedm commented Dec 20, 2017 via email

@jansoren
Copy link

jansoren commented Jan 3, 2018

bump

@NicolasRiou
Copy link

I downgraded my libs to NewtonSoft 9, so I can't reproduce the error right now but the error was a Constructor not found when we run this instruction :
_documentClient = new DocumentClient(settings.EndPoint, settings.AccessKey, GetJsonSerializerSettings(), GetConnectionPolicy(), GetConsistencyLevel())

The problem only appears when we use a custom serializer. Problem is we need this serializer to handle our data properly in cosmosDb.

@ultraviola
Copy link

@fabiocav

This is locked on 9.0.1 by design to prevent failures at runtime that are often confusing and difficult to diagnose.

Any update on fixing these "failures at runtime" with up to date versions of the package?

So many other things depend on newer versions of Newtonsoft.Json that remaining a major version behind to avoid potential runtime issues isn't viable. I'm excited about using Functions, but this could be a blocker for us.

@jansoren
Copy link

Could someone from the Azure community please look into/comment on the alpha-package suggestion? @ahmelsayed @vijayrkn

@fabiocav
Copy link
Member

@ultraviola I understand the concern. The issue is that, currently, just pushing the runtime up would be a breaking change for existing customers, and that's something that can't happen. The next version of Azure Functions will have options to remove those limitations.

Does the recommendation to add an explicit reference not work for your scenario?

@jboarman
Copy link

@fabiocav Can you point to an example of a breaking change that would result from upgrading the Newtonsoft.Json dependency? There's so much heat from this affecting us on this that something really has to be done.

I also tried cloning and building the libraries locally, with the intent of fixing this myself. However, everything compiled and ran successfully without any code changes from the current HEAD as far as I can tell. The only problem I faced was that all the nice VS Functions Tooling does not seem to work properly without having the NuGet reference.

@pseabury
Copy link

pseabury commented Feb 1, 2018

@fabiocav - So what you're saying is that if we don't use 9.0.1 we might end up with some weird runtime failures (maybe object serialization/deserialization issues with function bindings?).

But, If we use a dependency in our function that requires Newtonsoft.Json >=10.0.1 we can explicitly add that package to our functions project anyway at our own risk?

The obvious problem I see is that some of our dependencies may have legitimate requirements for >=10.0.1 and that we essentially have to either jettison Azure Functions, or our dependencies to make everyone happy. Kind of a rock & hard place issue for us.

BTW - We are using full .NetFramework, not core or standard.

@BowserKingKoopa
Copy link

Why is this such an issue for Functions for not for Webjobs? What’s the difference?

@kevbite
Copy link

kevbite commented Feb 1, 2018

@BowserKingKoopa I think it's the MSBuild Task that generates the function.json files.

@MV10
Copy link

MV10 commented Feb 2, 2018

It just got more annoying for me:

  1. New Functions app
  2. Manually upgrade to JSON 10
  3. Add Package X that needs JSON 10
  4. Add Package Y that refs WebJobs ... fail
  5. Remove Package X
  6. Remove JSON 10
  7. Add Package Y
  8. Re-add JSON 10
  9. Re-add Package X

Repeat several times per day until Package Y is fully developed.
So wish I could switch to netcore right now...

@shyamal890
Copy link

+1

@gurmitteotia
Copy link

Please fix it. We are also facing this issue in our project.

@mgroves
Copy link

mgroves commented Feb 7, 2018

I am also facing this issue, I think: https://stackoverflow.com/questions/48650419/could-not-load-file-or-assembly-the-system-cannot-find-the-file-specified. If I downgrade to 1.0.0-alpha6 with nuget, then the templates/tooling is no good.

@scottrudy
Copy link

I pulled down the 1.0.8 version of the library today and still see the issue.

@bobmclaren
Copy link

I get this error when I attempt to install the latest WindowsAzure.Storage package to my Azure Functions project. Additionally, if I attempt the rebinding trick, it will compile, but the function will not execute, providing only the obscure status 500 message, "An error has occurred. For more information, please check the logs for error ID..."

@scottrudy
Copy link

I spent several days slowly removing code from an Azure Function working with CosmosDB only to find that this is the underlying issue. I have used the binding redirect successfully before, but it did not work for this. All versions of the MicrosoftAzure.CosmosDB.Table library require Newtonsoft.Json >= 10.0.2. When trying the redirect the project compiles, but corrupted data is sent to CosmosDB, so only the PartitionKey, RowKey and TimeStamp show up in the table. I am going to cross post this in that library as well in the hopes that someone works to resolve this.

One suggestion might be to offer a version 2.0-alpha that eliminates this dependency.

@ahmelsayed
Copy link
Contributor

as mentioned in this issue, please add a direct dependency on which ever version of Newtonsoft.Json you want explicitly to your project and you should not have an issue. The error is only there for transit dependencies to warn you that you may get unexpected behavior if you rely on binding types from the different version of the assembly. Other than that, you should be able to use the versions you directly specify in your csproj.

I'll close this issue since I don't believe there is a plan for this build task to remove this dependency. Please open an issue on https://github.com/Azure/azure-functions-host to discuss design change in the runtime to not depend on these assembly versions. This repo is just for a build task that glues everything together, but the requirement doesn't come from the task itself.

/cc @fabiocav @paulbatum @davidebbo

@lorddev
Copy link

lorddev commented Feb 27, 2018

Has this been released to NuGet yet?

@ahmelsayed
Copy link
Contributor

What has? This issue was closed as 'by design'

@bottomup
Copy link

NewtonSoft released v11 recently, anyone got a chance to test the newest version?

@jboarman
Copy link

The error is only there for transit dependencies to warn you that you may get unexpected behavior if you rely on binding types from the different version of the assembly.
Other than that, you should be able to use the versions you directly specify in your csproj.

So, we might get unexpected results, if we use a newer assembly? Or, wait, are you saying to ignore this warning because we won't actually get unexpected results?

If we won't actually encounter unexpected results by using the latest version, then why can't the dependency requirements be opened up to allow latest versions of NewtonSoft?

This seems like a catch-22 that really should be resolved by now.

@Azure Azure locked as off-topic and limited conversation to collaborators Mar 1, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests