v4.0.0

Examine v4 has been in beta for a very long time - the first beta went out in October 2023 - and it's finally done. The headline feature is native faceting with proper taxonomy index support, but there's a pile of other work in here too: nullable reference types everywhere, .NET 8/9/10 targets, a much more robust replicator, and a fair amount of API cleanup.

If you're on v3, most of your code will keep compiling. There are breaking changes and they're all listed below, but the ones that bite are mostly constructors and a couple of types that went internal.

Facets

This is the big one. Examine can now do faceted search natively, with or without a Lucene taxonomy index.

You declare facet fields the same way you declare any other field, using new entries in FieldDefinitionTypes:

• Numeric: FacetDouble, FacetFloat, FacetInteger, FacetLong
• Full text: FacetFullText, FacetFullTextSortable
• Date: FacetDateTime, FacetDateYear, FacetDateMonth, FacetDateDay, FacetDateHour, FacetDateMinute
• Taxonomy variants of all of the above, prefixed with FacetTaxonomy

Then you ask for facets on a query with WithFacets(...) and read them back off the results with GetFacet() / GetFacets(). Range faceting is supported through three new immutable structs - DoubleRange, FloatRange and Int64Range - and IFacetOperations gives you FacetString, FacetDoubleRange, FacetFloatRange and FacetLongRange.

There's also LuceneFacetSamplingQueryOptions for random sampling if you're faceting over something enormous and don't need exact counts.

Taxonomy indexes

Taxonomy support started life as a separate TaxonomyIndex type and ended up merged into LuceneIndex. You turn it on per index with LuceneIndexOptions.UseTaxonomyIndex, and the taxonomy data is stored in a taxonomy subdirectory alongside the main index.

It's optional, which matters more than it sounds - a lot of the beta cycle was spent making sure that indexes without a taxonomy directory behave correctly, particularly around replication. New surface for this includes ITaxonomyDirectoryFactory, ILuceneTaxonomySearcher, ITaxonomySearchContext and ITaxonomySearcherReference, plus LuceneIndex.IsTaxonomyEnabled and LuceneIndex.TaxonomySearcher.

Deep paging

SearchAfterOptions lets you page deep into a result set without Lucene having to materialize everything up to your offset. It works for faceted queries too. ShardIndex is now a non-nullable int.

Query additions

• Boost factors - IExamineValueBoosted and a WithBoost() extension so you can weight individual search values.
• Phrase queries - SearchExtensions.Phrase(), backed by a new Examineness.Phrase.
• Examineness.Default - for when you want the analyzer's default behavior rather than picking one explicitly.
• ExamineValue.Create() - static factory methods replace the old parameterless constructor.
• Mixed value sets - a ValueSet can now hold a mix of single objects and enumerables in the same dictionary (#340).

Replication is a lot more robust

ExamineReplicator took a beating during the beta and came out considerably better:

• Taxonomy indexes replicate properly, including when the taxonomy writer hasn't been initialized yet (#454).
• Replicating with a non-taxonomy directory factory no longer crashes (#452).
• SyncedFileSystemDirectoryFactory no longer ends up with a main index and taxonomy directory that disagree with each other (#465).
• Transient lock failures when creating index or taxonomy writers are retried instead of throwing (#467).
• Persistent commit failures are now surfaced rather than swallowed, via IsReplicationHealthy and MaxConsecutiveReplicationFailures (#461).

If you're running Examine on App Service or anywhere else with a network file system, this is the section worth reading twice.

Nullable reference types

The whole codebase is <Nullable>enable</Nullable> with TreatWarningsAsErrors, and every public API is annotated. If you're building with nullable enabled you'll probably get some new warnings on upgrade. They're real - the annotations are telling you something that was always true.

Target frameworks and packaging

	v3.10.0	v4.0.0
Targets	net6.0, net8.0	net8.0, net9.0, net10.0
.NET Standard	dropped in v3	not present
Lucene.NET	4.8.0-beta00018	4.8.0-beta00018
Microsoft.Extensions.*	8.x/9.x	10.0.11

.NET 6 support is gone. Package versions are now managed centrally through Directory.Packages.props, and every public API is tracked with the Microsoft.CodeAnalysis.PublicApiAnalyzers shipped/unshipped files - which is how the breaking change list below was generated rather than guessed at.

Breaking changes

266 public APIs were added and 33 were removed or changed. About a third of the removals are just nullable re-annotations of the same member, so the list that actually matters is shorter than the number suggests.

Types

• Examine.Lucene.Providers.LuceneSearcher is now internal. Use BaseLuceneSearcher if you were deriving from it, or resolve ISearcher from DI if you were consuming it.

Constructors replaced by the options pattern

BaseLuceneSearcher and MultiIndexSearcher now take IOptionsMonitor<LuceneSearcherOptions> and IOptionsMonitor<LuceneMultiSearcherOptions> respectively. The old Analyzer-taking constructors are gone.

SearchContext, LuceneSearchResults and ExamineReplicator all have new constructor signatures - the replicator now takes a source directory and an optional destination taxonomy directory.

SyncedFileSystemDirectoryFactory and TempEnvFileSystemDirectoryFactory constructors changed to accept taxonomy directory factories. FileSystemDirectoryFactory and LuceneQueryOptions had their old constructors reintroduced during the beta for compatibility (#573, #550).

Members

• LuceneIndex.IsCancellationRequested removed.
• MultiIndexSearcher.Searchers returns IEnumerable<BaseLuceneSearcher> instead of IEnumerable<LuceneSearcher>.
• SearchAfterOptions.ShardIndex is int, not int?.
• ExamineValue's parameterless constructor is gone - use ExamineValue.Create().
• BaseLuceneSearcher.Dispose() is abstract, so derived searchers have to implement it.
• FileSystemDirectoryFactory.CreateDirectory and GenericDirectoryFactory.CreateDirectory overrides changed with nullable annotations.
• ReaderStatus was deleted (#336).

DI registration

AddExamineLuceneIndex and AddExamineLuceneMultiSearcher gained options-based overloads. The old parameter-list overloads still exist with nullable annotations applied.

Other fixes worth knowing about

• FacetFullTextField.MaxCount defaults to int.MaxValue instead of 10, so you get all your facet values back by default (#435).
• Single values index as a SingleField rather than a DoubleField.
• Empty SearchableFields from a brand new index aren't cached any more, which fixes searches against an index that was queried before it was first populated.
• Null queries are guarded before boost factors are applied (#514).
• FacetsConfig.Build is always called for non-taxonomy indexes.
• All options properties are persisted in AddExamineLuceneIndex (#424).
• Unused facet APIs removed before they shipped as public surface (#432).
• v3 binary-compat shims added to keep Umbraco 17 working (#436).

Upgrading

1. Retarget to net8.0 or later.
2. Rebuild and see what the nullable annotations tell you.
3. If you constructed searchers, directory factories or the replicator by hand, check the constructor list above.
4. If you want facets, add the facet field types to your FieldDefinitionCollection and turn on UseTaxonomyIndex if you want taxonomy-backed ones.

Everything else should just work. If it doesn't, open an issue and I'll take a look.

Thanks

Faceting and taxonomy support are almost entirely the work of @nzdev and @nikcio, who kept at it across a very long beta. @kjac found and fixed the compatibility problems that would have bitten Umbraco users. Thanks also to @bjarnef, @callumbwhyte, @andrewmckaskill and @vivekBoii.

Full Changelog: v3.10.0...v4.0.0