Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fresh solr 2.9.1 #765

Closed
wants to merge 57 commits into from
Closed

Conversation

jagguli
Copy link

@jagguli jagguli commented Mar 1, 2020

Pull request with yokozuna updated to solr7

llelf and others added 19 commits July 9, 2018 21:56
XML External Entity attack

1. if HTTP API is exposed:

  - read any file on the system — via /search/extract, the error message leaks
    file content;

  - send HTTP «GET /» request to any host — by PUT/POSTing text/xml document,
    or via /search/extract.  This is also likely riak DoS if the host is
    attacker-controlled.

2. if PB API is exposed

  - send HTTP «GET /» request to any host — by PUT/POST, see above.

Example request:

<?xml version="1.0"?>
<!DOCTYPE meow [
  <!ENTITY xxe2 SYSTEM "/etc/passwd">
  <!ENTITY xxe1 SYSTEM "http://host/ping-me">
]>
<meow>&xxe1;</meow>
It doesn't check user-provided path in any way.
This allows overriding any file on the system with riak permissions.
Security fixes

1. if HTTP API is exposed:

  - read any file on the system — via /search/extract, the error message leaks
    file content;

  - send HTTP «GET /» request to any host — by PUT/POSTing text/xml document,
    or via /search/extract.  This is also likely riak DoS if the host is
    attacker-controlled.

2. if PB API is exposed:

  - send HTTP «GET /» request to any host — by PUT/POST, see above.

3. if HTTP API is exposed:

  - override (with garbage) any file on the system with riak permissions.
@martinsumner
Copy link

Plans for this at the moment. I want to do run the riak_test tests this weekend, I suspect many won't pass (as already highlighted) - but if we can justify why, that's OK, and we can see what sort of test coverage we're left with.

I have an urgent need to cut riak 2.9.2 - so if there's anything I'm not sure about I'll hold off merging in until that release is completed. I intend to put out a release candidate with this in soon after - but I may try and juggle two streams for now.

ASAP, I want to get this, and the backlog of 2.9 changes, into the 3.0 release.

@martinsumner
Copy link

Running riak_test, the majority of yokozuna tests pass:

yz_aae_test-bitcask: pass
yz_core_properties_create_unload-bitcask: pass
yz_crdt-bitcask: pass
yz_dt_test-bitcask: pass
yz_ensemble-bitcask: pass
yz_errors-bitcask: pass
yz_faceted_search-bitcask: pass
yz_fallback-bitcask: pass
yz_handoff-bitcask: pass
yz_handoff_blocking-bitcask: pass
yz_languages-bitcask: pass
yz_mapreduce-bitcask: pass
yz_monitor_solr-bitcask: pass
yz_pb-bitcask: pass
yz_ring_resizing-bitcask: pass
yz_security-bitcask: pass
yz_siblings-bitcask: pass
yz_solrq_test-bitcask: pass
yz_stat_test-bitcask: pass
yz_wm_extract_test-bitcask: pass

Some test fail, but are tests of upgrades which we therefore wouldn't expect to pass:

yz_default_bucket_type_upgrade-bitcask: fail -  upgrade test
yz_extractors-bitcask: fail - upgrade test
yz_fuse_upgrade-bitcask: fail - upgrade test
yz_solr_upgrade_downgrade-bitcask: fail - upgrade test

There are some other test failures though. These might be explained by simple changes (e.g. to the solr rest API), so potentially could be easily solvable:

yz_entropy_data-bitcask: fail - url Issue?
yz_index_admin-bitcask: fail - url issue?
yz_schema_admin-bitcask: fail - cannot ping node after schema correction
yz_schema_change_reset-bitcask: fail - may also be ping related after schema change
yz_search_http-bitcask: fail - 500 error (not 415) on non-applicable content-type
yz_solr_start_timeout-bitcask: fail - looks for "alloted time" when actual phrase is "allowed time"
yz_startup_shutdown-bitcask: fail - query processes fail could be url issue?

@martinsumner
Copy link

martinsumner commented Apr 20, 2020

A bigger set of problems after merging into 3.0:

yokozuna_essential-bitcask: fail
yz_aae_test-bitcask: pass
yz_core_properties_create_unload-bitcask: pass
yz_crdt-bitcask: fail
yz_dt_test-bitcask: pass
yz_ensemble-bitcask: pass
yz_errors-bitcask: pass
yz_entropy_data-bitcask: fail
yz_faceted_search-bitcask: pass
yz_fallback-bitcask: pass
yz_handoff-bitcask: pass
yz_handoff_blocking-bitcask: pass
yz_index_admin-bitcask: fail
yz_languages-bitcask: pass
yz_mapreduce-bitcask: fail
yz_monitor_solr-bitcask: pass
yz_pb-bitcask: fail
yz_ring_resizing-bitcask: pass
yz_schema_admin-bitcask: fail
yz_schema_change_reset-bitcask: fail
yz_search_http-bitcask: fail
yz_security-bitcask: pass
yz_siblings-bitcask: fail
yz_solr_start_timeout-bitcask: fail
yz_solrq_test-bitcask: pass
yz_startup_shutdown-bitcask: fail
yz_stat_test-bitcask: pass
yz_wm_extract_test-bitcask: pass

@martinsumner
Copy link

The failure of yokozuna_essential and yz_pb both appear to be related to different handling of unicode within OTP20+. Both tests pass multiple sub-tests until hitting one with unicode data/keys.

@martinsumner
Copy link

The failure of yz_mapreduce is related to the deprecation of JS-map/reduce. The test needs to be changed to use inbuilt erlang map/reduce functions.

@martinsumner
Copy link

The failure of yz_crdt appears to be an in issue with an intercept function

@jagguli
Copy link
Author

jagguli commented Apr 22, 2020

deprecated by #766

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants