Skip to content

Conversation

@rzezeski
Copy link
Contributor

@rzezeski rzezeski commented Oct 4, 2013

This makes the necessary requirements to upgrade to Solr 4.4.0. A simple benchmark shows same performance before/after. I'll post graphs tomorrow.

Solr 4.4.0 also enables us to try out the new "schemaless" mode but that should be it's own PR. Furthermore, 4.5.0 should be coming out any day now. That upgrade should be done for the 0.11.0 release. But I wanted to update to 4.4.0 first to make that transition easier.

Make sure to delete the solr dir and build dir generated at compile
time.
Upgrade to the custom Yokozuna build of Solr 4.4.0.

* Modify the `solr.xml` file to use the new format.  See http://wiki.apache.org/solr/Solr.xml%204.4%20and%20beyond

* Fix `grab-solr.sh` to not fail on mkdir.
* Use set -e so that script fails if patches fail to apply

* Add some additional checks for robustness
@rzezeski
Copy link
Contributor Author

rzezeski commented Oct 4, 2013

This should address #150 and #129.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐟

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dat set -e.

@rzezeski
Copy link
Contributor Author

rzezeski commented Oct 4, 2013

Here are the benchmark results. Numbers are very close.

The interesting thing to note is that when I first ran the benchmark query throughput had dropped to 670 ops/s or 44% of baseline. I had forgotten to update the solr dist search patch in solr-patches. This means that the dist search patch still can have a major effect on query throughput and latency. I need to ping the Solr jira issue again to see if they might accept my patch upstream.

https://issues.apache.org/jira/browse/SOLR-4509

Indexing

Measurement Solr 4.3.0 Solr 4.4.0
Mean Thruput 848 ops/s 845 ops/s
Median Latency 38.0ms 38.2ms
95th Latency 59.6ms 60.6ms
99th Latency 77.4ms 77.2ms

Query

Measurement Solr 4.3.0 Solr 4.4.0
Mean Thruput 1512 ops/s 1525 ops/s
Median Latency 22.0ms 21.6ms
95th Latency 29.8ms 29.4ms
99th Latency 35.3ms 34.6ms

Query Without Dist Search Patch

Measurement Solr 4.3.0 Solr 4.4.0 Solr 4.4.0 w/o Patch
Mean Thruput 1512 ops/s 1525 ops/s 670 ops/s (44%)
Median Latency 22.0ms 21.6ms 46.2ms (2.1x)
95th Latency 29.8ms 29.4ms 76.8ms (2.6x)
99th Latency 35.3ms 34.6ms 86.2ms (2.5x)

@hazen
Copy link

hazen commented Oct 4, 2013

Tests are consistent with current 2.0 testing, i.e., all pass but yz_rs_migration-bitcask:

yz_wm_extract_test-bitcask   : pass
yz_stat_test-bitcask         : pass
yz_solr_start_timeout-bitcask: pass
yz_siblings-bitcask          : pass
yz_schema_admin-bitcask      : pass
yz_rs_migration-bitcask      : fail
yz_pb-bitcask                : pass
yz_monitor_solr-bitcask      : pass
yz_mapreduce-bitcask         : pass
yz_languages-bitcask         : pass
yz_index_admin-bitcask       : pass
yz_flag_transitions-bitcask  : pass
yz_fallback-bitcask          : pass
yz_errors-bitcask            : pass
yokozuna_essential-bitcask   : pass
aae_test-bitcask             : pass
---------------------------------------------
1 Tests Failed
15 Tests Passed
That's 93.75% for those keeping score
/tmp/yz-verify
INFO: IF YOU ARE READING THIS THEN I THINK THINGS WORKED

rzezeski added a commit that referenced this pull request Oct 4, 2013
@rzezeski rzezeski merged commit 297d5d0 into develop Oct 4, 2013
@coderoshi
Copy link
Contributor

Same deal with me. yz_rs_migration was a no go.

yz_wm_extract_test-bitcask   : pass
yz_stat_test-bitcask         : pass
yz_solr_start_timeout-bitcask: pass
yz_siblings-bitcask          : pass
yz_schema_admin-bitcask      : pass
yz_rs_migration-bitcask      : fail
yz_pb-bitcask                : pass
yz_monitor_solr-bitcask      : pass
yz_mapreduce-bitcask         : pass
yz_languages-bitcask         : pass
yz_index_admin-bitcask       : pass
yz_flag_transitions-bitcask  : pass
yz_fallback-bitcask          : pass
yz_errors-bitcask            : pass
yokozuna_essential-bitcask   : pass
aae_test-bitcask             : pass
---------------------------------------------
1 Tests Failed
15 Tests Passed
That's 93.75% for those keeping score

@rzezeski
Copy link
Contributor Author

rzezeski commented Oct 4, 2013

Yep, that test is still broken. Needs this update plus other fixes.

@rzezeski rzezeski deleted the feature/rz/solr-4.4 branch November 21, 2013 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants