-
Notifications
You must be signed in to change notification settings - Fork 262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mjl index speedup #213
Mjl index speedup #213
Commits on Oct 17, 2019
-
Use elasticsearch's parallel_bulk for indexing, add ELASTICSEARCH_DSL…
…_PARALLEL default setting and parameters to management command. Use qs.iterator() for fetching data during reindex, as this is much more memory efficient and performant. Instead of finding out which methods to call to prepare fields, do that finagling once and cache it for subsequent model instance prepares. See issue django-es#154 for performance analysis and details.
Martin J. Laubach committedOct 17, 2019 Configuration menu - View commit details
-
Copy full SHA for b4920c9 - Browse repository at this point
Copy the full SHA b4920c9View commit details -
Move collection of prepare functions to __init__, where it's conceptu…
…ally cleaner. Also shaves off a test per object.
Martin J. Laubach committedOct 17, 2019 Configuration menu - View commit details
-
Copy full SHA for e5cd2df - Browse repository at this point
Copy the full SHA e5cd2dfView commit details -
Minor cleanup: Move prepare cache to Document object instead of Model…
…, as it's conceptually possible to have several indices on the same model. Also remove forced ordering that is a remnant of earlier code.
Martin J. Laubach committedOct 17, 2019 Configuration menu - View commit details
-
Copy full SHA for 861f704 - Browse repository at this point
Copy the full SHA 861f704View commit details
Commits on Oct 18, 2019
-
chunk_size parameter for queryset.iterator() appeared in Django 2
Martin J. Laubach committedOct 18, 2019 Configuration menu - View commit details
-
Copy full SHA for 827800d - Browse repository at this point
Copy the full SHA 827800dView commit details -
Do not crash in init_prepare when no fields have been defined
Martin J. Laubach committedOct 18, 2019 Configuration menu - View commit details
-
Copy full SHA for 1773ed6 - Browse repository at this point
Copy the full SHA 1773ed6View commit details -
Crank up diff size to see what is going on
Martin J. Laubach committedOct 18, 2019 Configuration menu - View commit details
-
Copy full SHA for acda7e7 - Browse repository at this point
Copy the full SHA acda7e7View commit details -
Adapt test to changed call pattern
Martin J. Laubach committedOct 18, 2019 Configuration menu - View commit details
-
Copy full SHA for 8279746 - Browse repository at this point
Copy the full SHA 8279746View commit details -
Adapt tests to changed call patterns
Martin J. Laubach committedOct 18, 2019 Configuration menu - View commit details
-
Copy full SHA for c4f230d - Browse repository at this point
Copy the full SHA c4f230dView commit details -
Mark pagination test as expected failure for now.
Martin J. Laubach committedOct 18, 2019 Configuration menu - View commit details
-
Copy full SHA for 1dcc013 - Browse repository at this point
Copy the full SHA 1dcc013View commit details -
Define _prepared_fields as attribute in class so to_dict() won't pick…
… it up as document field
Martin J. Laubach committedOct 18, 2019 Configuration menu - View commit details
-
Copy full SHA for 505406e - Browse repository at this point
Copy the full SHA 505406eView commit details
Commits on Oct 21, 2019
-
Martin J. Laubach committed
Oct 21, 2019 Configuration menu - View commit details
-
Copy full SHA for 93f7d7c - Browse repository at this point
Copy the full SHA 93f7d7cView commit details
Commits on Oct 24, 2019
-
Add parameter no to do a count(*) before indexing, as for complex que…
…rysets that might be expensive.
Martin J. Laubach committedOct 24, 2019 Configuration menu - View commit details
-
Copy full SHA for c4a8a24 - Browse repository at this point
Copy the full SHA c4a8a24View commit details
Commits on Oct 25, 2019
-
Configuration menu - View commit details
-
Copy full SHA for 2d2e996 - Browse repository at this point
Copy the full SHA 2d2e996View commit details -
Correctly clean up after test run (delete indices with the right name).
Martin J. Laubach committedOct 25, 2019 Configuration menu - View commit details
-
Copy full SHA for 3b69009 - Browse repository at this point
Copy the full SHA 3b69009View commit details -
Add tests for usage of init_prepare() and _prepared_fields. Add tests for correct calling of bulk/parallel_bulk.
Martin J. Laubach committedOct 25, 2019 Configuration menu - View commit details
-
Copy full SHA for f613926 - Browse repository at this point
Copy the full SHA f613926View commit details -
Make sure we compare w/ stable order
Martin J. Laubach committedOct 25, 2019 Configuration menu - View commit details
-
Copy full SHA for 51360c3 - Browse repository at this point
Copy the full SHA 51360c3View commit details -
Adjust for different types for methods/partials in py2
Martin J. Laubach committedOct 25, 2019 Configuration menu - View commit details
-
Copy full SHA for 475e8a8 - Browse repository at this point
Copy the full SHA 475e8a8View commit details -
Correct es dependency (was conflicting with requirements.txt)
Martin J. Laubach committedOct 25, 2019 Configuration menu - View commit details
-
Copy full SHA for 45f62e7 - Browse repository at this point
Copy the full SHA 45f62e7View commit details
Commits on Oct 28, 2019
-
Pass queryset_pagination as chunk_size into parallel_bulk too.
Martin J. Laubach committedOct 28, 2019 Configuration menu - View commit details
-
Copy full SHA for c933fb7 - Browse repository at this point
Copy the full SHA c933fb7View commit details -
Add explanation why we use deque()
Martin J. Laubach committedOct 28, 2019 Configuration menu - View commit details
-
Copy full SHA for 96c3883 - Browse repository at this point
Copy the full SHA 96c3883View commit details -
Correct typo in explanation of test
Martin J. Laubach committedOct 28, 2019 Configuration menu - View commit details
-
Copy full SHA for 9c138d6 - Browse repository at this point
Copy the full SHA 9c138d6View commit details -
Remove leftover instrumentation print
Martin J. Laubach committedOct 28, 2019 Configuration menu - View commit details
-
Copy full SHA for 9ec41a0 - Browse repository at this point
Copy the full SHA 9ec41a0View commit details -
Better formatting to avoid backslash-continuation line
Martin J. Laubach committedOct 28, 2019 Configuration menu - View commit details
-
Copy full SHA for 5086396 - Browse repository at this point
Copy the full SHA 5086396View commit details