Skip to content

ip-intelligence/4.5.156

Choose a tag to compare

@Automation51D Automation51D released this 13 Nov 01:57
8b3220b

This release moves .NET toward one behavior across every 51Degrees language. Each element it changes now follows the published 51Degrees specifications, which in turn follow the RFCs they cite, so that the same input gives the same answer in .NET, Java, Node.js, Python, PHP, Go, Rust and C. .NET was the reference the other languages were aligned to, so many of its changes are small. The native libraries take the C library release 51degrees-cxx 4.5.105. Each change names the package family it is in.

This release adds .NET 10. Every package gains a net10.0 build and keeps the builds it had, so a .NET 10 application now loads code built for .NET 10, where it loaded the .NET 8 or .NET Standard 2.0 build before. One kind of .NET 10 web application has to raise two package versions before it will restore, which is set out under "Before you upgrade".

Before you upgrade

  1. The client-side ETag is a weak entity tag. The JavaScript and JSON endpoints send W/"...", where they sent a bare number, and compare If-None-Match weakly, as RFC 9110 section 8.8.3 and section 13.1.2 ask. Code that reads the ETag as a number needs to allow for W/ and the double quotes (#96, pipeline).
  2. Web integration options in a configuration file now take effect. On ASP.NET Core nothing bound PipelineWebIntegrationOptions to the configuration, so the defaults always applied. A file that sets ClientSideEvidenceEnabled or UseSetHeaderProperties to false now turns that feature off, so check both values (#115, pipeline).
  3. Usage Sharing accepts no Evidence. It reads the web request instead, so a Pipeline that holds it no longer accepts every header. A Flow Element of your own that read a header it did not declare must name the key in its EvidenceKeyFilter, or read the request from the new server.http-request Evidence value (#100, pipeline).
  4. An application that builds its own Evidence has to give Usage Sharing the web request. This applies to an application that adds a value to Evidence only when EvidenceKeyFilter on the Pipeline or the Flow Data includes its key. An application that uses the web integration has nothing to change. Up to pipeline 4.5.129 Usage Sharing accepted the headers it shares, so that check kept them. Usage Sharing now accepts none, so the same check drops every header that no other Flow Element reads, and those headers stop being shared. Add the request to the Flow Data under Constants.EVIDENCE_HTTPREQUEST_KEY in FiftyOne.Pipeline.Core, which is server.http-request. The value is an IReadOnlyDictionary<string, object> that holds each value of the request under the key it would have as Evidence, such as header.user-agent, cookie.51D_ScreenPixelsWidth, query.name and server.client-ip. Usage Sharing shares every Evidence value in the Flow Data, then each value of that dictionary whose key the Evidence does not hold. Its rules still apply, so the Cookie header, a blocked header and a cookie whose name does not start 51D_ are left out. The dictionary itself is never shared, is not sent to the cloud service and takes no part in a cache key. On ASP.NET Core, IWebRequestEvidenceService.AddEvidenceFromRequest in FiftyOne.Pipeline.Web adds the request for you (#100, pipeline).
  5. On .NET Framework, session Evidence is renamed. A session value is session.<name> and the session object is session.session, where they were <name> and session (#100, pipeline).
  6. IpiOnPremiseEngineBuilder defaults to the Balanced performance profile. With no profile set it loaded the whole Data File into memory. Call SetPerformanceProfile(PerformanceProfiles.MaxPerformance) to keep the old behavior. IpiOnPremisePipelineBuilder already used Balanced (#102, ip-intelligence).
  7. A client IP value is read only when all of it is one address. .NET read the first entry of a list, trimmed white space and dropped a zone index. A list such as 203.0.113.1, 198.51.100.7, a padded value, fe80::1%eth0 and 01.2.3.4 are now skipped, and the Engine tries the next client IP key. Pass one address in each value (#130, ip-intelligence).
  8. Property lists are read only. IFlowElement.Properties and IAspectEngine.Properties return a read only list, and code that changes the list gets a NotSupportedException. Copy the list first, for example new List<IElementPropertyMetaData>(element.Properties) (#99, pipeline).
  9. The JavaScript Builder needs a Sequence Element. Without one it renders sequence 1 and an empty session id, whatever the request carried. The web integration already adds one (#132, pipeline).
  10. Settings that were accepted now stop the Pipeline build. These are a JavaScript Builder protocol other than http or https (#59) and an object name that the script cannot declare, such as window, document or location (#123, pipeline).
  11. Evidence named resource or license no longer replaces the configured key. The Cloud Request Engine leaves out Evidence named resource, license, licence or values. Set the keys with SetResourceKey and SetLicenseKey (#124, pipeline).
  12. A .NET 10 web application may have to raise two package versions. This applies only to an application that targets net10.0, uses FiftyOne.Pipeline.Web directly or through another package, and names Microsoft.AspNetCore.Mvc.Razor.RuntimeCompilation or Microsoft.Extensions.FileProviders.Embedded in its own project file at a version below 10.0.14. The new net10.0 build of FiftyOne.Pipeline.Web needs both at 10.0.14 or later, where the build a .NET 10 application loaded before needed 6.0.15. NuGet then stops the restore with NU1605, "Detected package downgrade", which the .NET SDK treats as an error. Change both versions in the project file to 10.0.14 or later, or remove the two references so that FiftyOne.Pipeline.Web brings them in. An application that targets net8.0, or that names neither package, needs no change (#95, Pipeline).
  13. An Engine built with no Property names keeps the first Data File's Properties. A Property that only a later Data File holds reads as unavailable until the Engine is built again (#137, device-detection and ip-intelligence).
  14. A Property with no value gives a message. NoValueMessage, and the JSON nullreason, of an on-premise Property with no value hold "None of the results contain a value for the requested property.", where they were null (#116, device-detection and ip-intelligence).
  15. The agent signature element accepts only the eight Evidence keys its specification names. A signature that covers any other header is reported Unverified with the reason ComponentUnavailable (#106, pipeline).
  16. A 51Did with white space inside it, or with characters from both base64 alphabets, is refused. FodId.TryParse reports InvalidBase64, and the string constructor and FodId.FromBase64 throw FormatException (#85, pipeline).
  17. The Usage Sharing builder checks its back-off settings. SetFailuresWindowSeconds, SetInitialRecoveryDelaySeconds, SetMaxRecoveryDelaySeconds and SetRecoveryMultiplier reject a value the back-off cannot use. RecoverySeconds, UseExponentialBackoff and CreateRecoveryStrategy on ShareUsageBuilderBase are obsolete (#88, pipeline).
  18. _51DEGREES_CLOUD_ENDPOINT replaces FOD_CLOUD_API_URL, which still works in this release. When no endpoint is set in code or configuration, CloudRequestEngineBuilder and DidClient read _51DEGREES_CLOUD_ENDPOINT, and read FOD_CLOUD_API_URL only when the new name is unset or empty. The first use of the old name in a process is reported as deprecated, as a warning through the builder's logger, and by DidClient through System.Diagnostics.Trace. Constants.FOD_CLOUD_API_URL and DidClient.EndpointEnvironmentVariable still hold FOD_CLOUD_API_URL and compile with an obsolete warning, and Constants.CLOUD_ENDPOINT_ENV_VAR and the new DidClient.CloudEndpointEnvironmentVariable hold the new name. Set _51DEGREES_CLOUD_ENDPOINT to the same value as the old name (#131, #143, pipeline). The examples, tests and CI scripts also read every old name in the table under "Environment variables" as a deprecated name, when its new name is unset or empty (#144).

What changes

Web integration and client-side caching

  • A request whose If-None-Match matches gets 304 for GET and HEAD, and 412 for any other method, as RFC 9110 section 13.1.2 says. A response to a request whose processing failed has Cache-Control: private, no-cache and no ETag. The same Evidence gives the same ETag in every process, so a browser's cached response stays valid behind a load balancer (#96, pipeline).
  • PipelineWebIntegrationOptions gains JavaScriptPath, JsonPath, SequenceElementEnabled and SetHeadersElementEnabled (#115, pipeline).
  • In a web application, a DataFile or TempDirPath that starts ~/ is resolved against the application root (#118, pipeline).

JavaScript Builder and JSON Builder

  • The JavaScript Builder checks the callback URL before it writes it into the client script. The protocol must be http or https, and the host must be ASCII labels or an IPv6 address in brackets, with no user information, query, fragment or zone identifier, following RFC 3986. A URL from the request that fails is left out of the script and logged (#59, #112, pipeline).
  • An object name in the request that the script cannot declare is ignored in favor of the configured name, with a warning. The 30 names refused are the ones the specification lists (#123, pipeline).
  • The JSON Builder fails with the message "The JSON Builder needs a sequence number. Place a Sequence Element before the JSON Builder in the Pipeline." when the Evidence holds no sequence number. The JSON gains a sequence entry that holds session-id and sequence (#135, #115, pipeline).

Cloud

  • The Cloud Request Engine can authenticate with a License Key, alone or with a Resource Key. With a License Key alone the Properties wanted must be named with the new SetRequestedProperties (#129, pipeline).
  • The Cloud Request Engine sends every Evidence value apart from the reserved names. One name under two prefixes, in any letter case, is sent once, taken from query, then header, cookie, server, fiftyone and location (#124, #136, pipeline).
  • Only trouble with the cloud service counts toward the recovery period, meaning a request that did not complete, a 5xx or 429 answer, or an answer that cannot be read. A 4xx answer to one request no longer stops every other request (#120, pipeline).
  • A License Key appears in log entries and error messages only as its last 4 characters, with the rest as * (#119, pipeline).

Usage Sharing

  • The Cookie header is never shared. Cookies whose names start 51D_ are still shared. Identical Evidence is shared at most once in each repeat interval (#100, #87, pipeline).
  • Sending pauses after repeated failures, and disposing the element sends every queued entry, for at most 5 seconds (#88, #127, pipeline).

Device Detection

  • SetIndexedProperties("NativeModel", "TAC") on the Hash Engine builder makes the TAC and NativeModel Engines look a value up in the Hash Engine's own index, which is built again when the Data File is refreshed. With the index, a value matches only its exact spelling (#101, device-detection).
  • The robots.txt Engine writes each Terms Document Locator, the address of the terms a crawler works under, exactly as given, and drops a value that is not a URI under RFC 3986 section 3 (#109, device-detection).

IP Intelligence

  • A Property is read by the position of its name, fixed when the Engine is built, in place of a lookup by name on every read (#133, ip-intelligence).

C library

  • The C library moves from 51degrees-cxx 4.5.103 to 4.5.105. A thread waits for a free file handle in place of failing when more threads than the Concurrency setting read one Data File. Concurrency defaults to the processor count, and setting a performance profile no longer resets Concurrency. HighPerformance holds in memory only what the named Properties read (#116, #137, device-detection and ip-intelligence).

Packages and platforms

  • .NET 10. Every package gains a net10.0 build and keeps the builds it had. The packages that had only a .NET Standard 2.0 build now have two, and a .NET 8 application still loads the .NET Standard 2.0 build of those. FiftyOne.Pipeline.Web has net8.0, net10.0 and net462 builds, and its net10.0 build needs Microsoft.AspNetCore.Mvc.Razor.RuntimeCompilation and Microsoft.Extensions.FileProviders.Embedded at 10.0.14 or later, as "Before you upgrade" explains (#95, every family).
  • The descriptions of FiftyOne.DeviceDetection.Shared, FiftyOne.DeviceDetection.PropertyKeyed and FiftyOne.IpIntelligence.Shared say what each package does, and name the products each is an alternative to (#113, device-detection and ip-intelligence).
  • FiftyOne.Pipeline.Examples.Shared declares the EUPL-1.2 license, as every other package does. Versions 4.5.129 and earlier declared AGPL-3.0-or-later. The repository has one LICENSE file at its root, the EUPL 1.2 text (#138, pipeline).

Examples and documentation

  • The console examples and the getting started web examples run with no settings on the free Data Files, which they download the first time. A Property that only the Enterprise Data File holds is shown as needing that file, with a link to ask for a copy. The web and API examples that use both products still need the Enterprise Data Files (#74).

Environment variables

Each name below has a new name that starts _51DEGREES_, which every shell can set. CloudRequestEngineBuilder and DidClient read the cloud endpoint, and the examples, tests and CI scripts read the rest. Every old name in the table is still read in this release, but only when its new name is unset or empty. The first use of each old name in a process is reported as deprecated, through the builder's logger, through System.Diagnostics.Trace in DidClient, on the error stream in the examples and tests, and as a PowerShell warning in the CI scripts (#131, #143, #144).

Old name, deprecated New name
FOD_CLOUD_API_URL, 51DEGREES_CLOUD_ENDPOINT, 51D_CLOUD_ENDPOINT, FIFTYONE_CLOUD_ENDPOINT _51DEGREES_CLOUD_ENDPOINT
51DEGREES_RESOURCE_KEY, 51D_RESOURCE_KEY, RESOURCE_KEY, SUPER_RESOURCE_KEY _51DEGREES_RESOURCE_KEY
LICENSE_KEY _51DEGREES_LICENSE_KEY
51DEGREES_DD_PATH, DEVICEDETECTIONDATAFILE _51DEGREES_DD_PATH
51DEGREES_IPI_PATH, IPINTELLIGENCEDATAFILE _51DEGREES_IPI_PATH
FIFTYONE_TEST_FILTER _51DEGREES_TEST_FILTER

Fixed

  • With the results cache on, a cloud request whose Evidence matched an earlier request received the 51Did created for the earlier one. A response that holds a 51Did is no longer stored in or served from the cache (#125, pipeline).
  • Reading four collection settings of the IP Intelligence Engine's native configuration through reflection ended the process with heap corruption in FiftyOne.IpIntelligence.Engine.OnPremise 4.5.155 (#76, ip-intelligence).
  • A browser's revalidation of the client-side JavaScript or JSON never got a 304, because the ETag differed on every request and in every process (#96, pipeline).
  • Usage Sharing lost the whole message when Evidence held a key that starts with ., an emoji beside a control character, or U+FFFE or U+FFFF (#114). It lost one entry each time a message was full (#88), and all but the first message of the queue when the element was disposed (#127, pipeline).
  • A data update check whose timer fired before the service had stored the timer left no later check scheduled (#104). A manual CheckForUpdate that applied an update turned automatic updates off (#128, pipeline).
  • The Set Headers Element kept an empty configuration until the process restarted when a Cloud Engine had not loaded its Properties by the first request (#126, pipeline).
  • The generated robots.txt file gave NotCrawler, the profile every browser resolves to, a group of its own with User-Agent: N/A (#78), and a Terms Document Locator lost its percent escapes (#109, device-detection).
  • A missing Property whose accessor the Engine's data type inherits, such as HardwareVendor on the Lite Data File, gave the reason Unknown, where the reason is that the Data File needs upgrading (#97, pipeline).