Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Identify/Fix Bottlenecks #261

Closed
clambertus opened this issue Oct 10, 2019 · 2 comments · Fixed by #298 or #303
Closed

Identify/Fix Bottlenecks #261

clambertus opened this issue Oct 10, 2019 · 2 comments · Fixed by #298 or #303
Labels
Lucene.Net Core up-for-grabs This issue is open to be worked on by anyone
Milestone
4.8.0

Comments

@clambertus
Copy link

clambertus commented Oct 10, 2019 •
edited by NightOwl888

We need some help profiling Lucene.NET to identify and/or fix any potential bottlenecks. Please make us aware of anything you find by opening a new GitHub issue or by letting us know on the dev mailing list.

JIRA link - [LUCENENET-630] created by nightowl888
@clambertus clambertus added Lucene.Net Core up-for-grabs This issue is open to be worked on by anyone labels May 5, 2020
@NightOwl888 NightOwl888 added this to the Lucene.NET 4.8.0 milestone May 7, 2020
@NightOwl888 NightOwl888 mentioned this issue Jun 9, 2020
Closed
31 tasks
@NightOwl888 NightOwl888 mentioned this issue Jun 17, 2020
Closed
asfgit pushed a commit that referenced this issue Jun 17, 2020
@NightOwl888
PERFORMANCE: Boxing issue causing issue with some unit tests that wer…
a54545d 
…e using AreEqual with bool. Added overloads of AreEqual that accept bool to cover the project systemically without changing tests. (#261, #295)
NightOwl888 added a commit to bongohrtech/lucenenet that referenced this issue Jun 23, 2020
@NightOwl888
Lucene.Net.Tests.Suggest.Fst.FSTCompletionTest: Removed unnecessary c…
a17936a 
…alls to ElementAt() and Count() LINQ (Addresses apache#261)
NightOwl888 added a commit to bongohrtech/lucenenet that referenced this issue Jun 23, 2020
@NightOwl888
Lucene.Net.TestFramework: Added overloads for int to Assert.AreEqual …
4af199b 
…(addresses apache#295, apache#261)
NightOwl888 added a commit to bongohrtech/lucenenet that referenced this issue Jun 23, 2020
@NightOwl888
SWEEP: Lucene.Net.Util: Added using for Assert class to ensure our st…
acd6722 
…rongly-typed overloads are being called to reduce boxing. (addresses apache#295, apache#261)
NightOwl888 added a commit to bongohrtech/lucenenet that referenced this issue Jun 23, 2020
@NightOwl888
Lucene.Net.Tests.Util.TestOfflineSorter::AssertFilesIdentical(): Chan…
6eeece6 
…ged test to utilize the J2N's ArrayEqualityComparer to optimize the speed of the test (apache#295, apache#261)
NightOwl888 added a commit to bongohrtech/lucenenet that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.TestFramework: Optimized array asserts by first checking w…
2563735 
…ith J2N.Collections.ArrayEqualityComparer<T> before cascading only the failing calls to NUnit.CollectionAssert.AreEqual to generate the error message (apache#295, apache#261)
NightOwl888 added a commit to bongohrtech/lucenenet that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.Tests.Search + Lucene.Net.Tests.Support: Switched to using…
a5e37b9 
… the Assert class in the test framework instead of NUnit (apache#295, apache#261)
NightOwl888 added a commit to bongohrtech/lucenenet that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.Tests.Index: Switched to using the Assert class in the tes…
218b4e2 
…t framework instead of NUnit (apache#295, apache#261)
NightOwl888 added a commit to bongohrtech/lucenenet that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.Tests.Analysis + Lucene.Net.Tests.Codecs + Lucene.Net.Test…
b62684c 
…s.Document: Switched to using the Assert class from the test framework instead of NUnit (apache#295, apache#261)
NightOwl888 added a commit to bongohrtech/lucenenet that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.Tests.ICU: Switched to using the Assert class from the tes…
c5d6e92 
…t framework instead of NUnit (apache#295, apache#261).
NightOwl888 added a commit to bongohrtech/lucenenet that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.Tests.Analysis.Phonetic: Switched to using the Assert clas…
f6404d3 
…s from the test framework instead of NUnit (apache#295, apache#261)
NightOwl888 added a commit to bongohrtech/lucenenet that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.Tests.Facet: Switched to using the Assert class from the t…
50017c4 
…est framework instead of NUnit (apache#295, apache#261)
NightOwl888 added a commit to bongohrtech/lucenenet that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.TestFramework: Added assert overloads for float (apache#295,
353e423 
apache#261)
@NightOwl888 NightOwl888 linked a pull request Jun 24, 2020 that will close this issue
Fixes several slow tests (#295) #298
Merged
NightOwl888 added a commit that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.Tests.Suggest.Fst.FSTCompletionTest: Removed unnecessary c…
6df2371 
…alls to ElementAt() and Count() LINQ (Addresses #261)
NightOwl888 added a commit that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.TestFramework: Added overloads for int to Assert.AreEqual …
6f504d2 
…(addresses #295, #261)
NightOwl888 added a commit that referenced this issue Jun 24, 2020
@NightOwl888
SWEEP: Lucene.Net.Util: Added using for Assert class to ensure our st…
666c9bb 
…rongly-typed overloads are being called to reduce boxing. (addresses #295, #261)
NightOwl888 added a commit that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.Tests.Util.TestOfflineSorter::AssertFilesIdentical(): Chan…
b0d23b2 
…ged test to utilize the J2N's ArrayEqualityComparer to optimize the speed of the test (#295, #261)
NightOwl888 added a commit that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.TestFramework: Optimized array asserts by first checking w…
2e0f25d 
…ith J2N.Collections.ArrayEqualityComparer<T> before cascading only the failing calls to NUnit.CollectionAssert.AreEqual to generate the error message (#295, #261)
NightOwl888 added a commit that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.Tests.Search + Lucene.Net.Tests.Support: Switched to using…
b5f8a94 
… the Assert class in the test framework instead of NUnit (#295, #261)
NightOwl888 added a commit that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.Tests.Index: Switched to using the Assert class in the tes…
fd48c40 
…t framework instead of NUnit (#295, #261)
NightOwl888 added a commit that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.Tests.Analysis + Lucene.Net.Tests.Codecs + Lucene.Net.Test…
c08ebef 
…s.Document: Switched to using the Assert class from the test framework instead of NUnit (#295, #261)
NightOwl888 added a commit that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.Tests.ICU: Switched to using the Assert class from the tes…
aca9e47 
…t framework instead of NUnit (#295, #261).
NightOwl888 added a commit that referenced this issue Jun 24, 2020
@NightOwl888
Lucene.Net.Tests.Analysis.Phonetic: Switched to using the Assert clas…
5c25835 
…s from the test framework instead of NUnit (#295, #261)
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.Util.AttributeSource: Optimize creation of built-in attrib…
c2d23de 
…utes (#295, #261)
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.Util.AttributeSource: Optimized string building for conver…
2713663 
…ting Attribute interface name to Attribute class name (#295, #261)
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.TestFramework.Randomized.Generators.RandomInts::RandomInt3…
1f24d31 
…2Between(): Debug.Assert needs to be compiled out of the build in this case to avoid performance issues with its string formatting (#295, #261)
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.Util.Automaton: Fixed State class to initialize and trim m…
867834b 
…ore efficiently (#295, #261). Fixes the performance of Lucene.Net.Util.Automaton.TestBasicOperations::TestEmptyLanguageConcatenate().
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.Util.Automaton.DacuikMihovAutomatonBuilder: Reduce the num…
508389f 
…ber of zero length array allocations (#295, #261)
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.TestFramework.Search.RandomSimilarityProvider::ToString():…
c7f23fa 
… Use StringBuilder for better efficiency (#295, #261)
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.Analsis.TokenAttributes.CharTermAttribute::ResizeBuffer():…
98c0295 
… Use Array.Resize() rather than copy (#261)
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.TestFramework.Util.LuceneTestCase: Cache codecType and sim…
793fea9 
…ilarityName as strings so they don't have to be regenerated on each test (#261, #295)
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.Tests.Suggest: Removed calls to ElementAt() (#261)
eb99971
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.Suggest: Removed ElementAt() calls from Lists, also revers…
6f98edf 
…ing the list in place is faster than LINQ (#261)
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.QueryParser.ComplexPhrase.ComplexPhraseQueryParser: Remove…
29d186c 
… ElementAt() from List (#261)
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.Index.Term: Optimized equality checking (#295, #261)
3086148
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.Tests.Index.TestDocumentsWriterDeleteQueue: Updated compar…
7550f36 
…isons to reduce memory allocations (#261)
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.TestFramework: Changed ConcurrentMergeSchedulerFactories.V…
4a26905 
…alues to only return the TaskMergeScheduler rarely, since it is no longer a default setting and is slowing down tests (#295, #261)
NightOwl888 added a commit that referenced this issue Jun 30, 2020
@NightOwl888
Lucene.Net.Util (MathUtil + NumericUtils + SloppyMath + UnicodeUtil):…
40186fb 
… Added some aggressive inlining, made classes static (#261)
@NightOwl888 NightOwl888 linked a pull request Jun 30, 2020 that will close this issue
Fix for slow Lucene.Net.Util namespace tests (addresses #295) #303
Merged
NightOwl888 added a commit to NightOwl888/lucenenet that referenced this issue Jul 29, 2020
@NightOwl888
PERFORMANCE: Lucene.Net.TestFramework: Fixed FSTTester to delete whil…
cfb72dc 
…e iterating forward instead of using .ElementAt() to iterate in reverse, which takes about 3x longer (see apache#261)
NightOwl888 added a commit to NightOwl888/lucenenet that referenced this issue Jul 29, 2020
@NightOwl888
PERFORMANCE: Lucene.Net.Facet.Taxonomy.WriterCache.NameInt32CacheLRU:…
a0b53c6 
… Changed from Dictionary to ConcurrentDictionary so we can delete items from the cache while forward iterating through it. (see apache#261)
NightOwl888 added a commit to NightOwl888/lucenenet that referenced this issue Jul 29, 2020
@NightOwl888
PERFORMANCE: Lucene.Net.Index.FieldInfos: Changed Builder.FieldInfo()…
8a33faf 
… method to TryGetFieldInfo() to optimize check for value (see apache#261)
NightOwl888 added a commit to NightOwl888/lucenenet that referenced this issue Jul 29, 2020
@NightOwl888
PERFORMANCE: Use J2N's ICollection<T>.ToArray() extension method that…
8018cae 
… uses ICollection<T>.CopyTo(), which takes precedence over the LINQ IEnumerable<T>.ToArray() extension method. Benchmarks show about a 1/3 increase in performance. (see apache#261)
NightOwl888 added a commit to NightOwl888/lucenenet that referenced this issue Jul 29, 2020
@NightOwl888
PERFORMANCE: Eliminated several calls to FirstOrDefault(), LastOrDefa…
8c768c2 
…ult(), Skip(), First(), and Last() (see apache#261)
NightOwl888 added a commit that referenced this issue Jul 29, 2020
@NightOwl888
PERFORMANCE: Lucene.Net.TestFramework: Fixed FSTTester to delete whil…
b4c473e 
…e iterating forward instead of using .ElementAt() to iterate in reverse, which takes about 3x longer (see #261)
NightOwl888 added a commit that referenced this issue Jul 29, 2020
@NightOwl888
PERFORMANCE: Lucene.Net.Facet.Taxonomy.WriterCache.NameInt32CacheLRU:…
4bdb4e3 
… Changed from Dictionary to ConcurrentDictionary so we can delete items from the cache while forward iterating through it. (see #261)
NightOwl888 added a commit that referenced this issue Jul 29, 2020
@NightOwl888
PERFORMANCE: Lucene.Net.Index.FieldInfos: Changed Builder.FieldInfo()…
299e047 
… method to TryGetFieldInfo() to optimize check for value (see #261)
NightOwl888 added a commit that referenced this issue Jul 29, 2020
@NightOwl888
PERFORMANCE: Use J2N's ICollection<T>.ToArray() extension method that…
5f9bcb9 
… uses ICollection<T>.CopyTo(), which takes precedence over the LINQ IEnumerable<T>.ToArray() extension method. Benchmarks show about a 1/3 increase in performance. (see #261)
NightOwl888 added a commit that referenced this issue Jul 29, 2020
@NightOwl888
PERFORMANCE: Eliminated several calls to FirstOrDefault(), LastOrDefa…
61bd539 
…ult(), Skip(), First(), and Last() (see #261)
@NightOwl888 NightOwl888 mentioned this issue Aug 11, 2020
Merged
@theolivenbaum
Copy link
Contributor

Two changes from what I noticed while benchmarking Lucene: #370 and #372
Working now on a better replace for Debugging.Assert that does not depend a delegate here: #373

@NightOwl888
Copy link
Contributor

I am closing this now, as we are finding and reporting bottlenecks as specific issues, so we no longer need a general task for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Lucene.Net Core up-for-grabs This issue is open to be worked on by anyone
Projects
None yet
Milestone
4.8.0
3 participants
@NightOwl888 @clambertus @theolivenbaum