Improve Solr integration with VariantStorage when querying

[j-coll]

Currently, variant queries can be executed on the Variant Storage (mongodb or hbase), or on Solr.

We have seen that complex queries using multiple indexes work really good over Solr, and not so good on MongoDB or HBase.

In the query can not be 100% solved by Solr (because it requests or filters by fields not stored in solr) we can decide (using some heuristics) to query first solr to get the ids, and then to the real variant storage.

We expect to see a big speed-up on complex queries (more than 5 filters).

• Add embedded Solr for JUnit tests
• Join Solr results with results from Mongo or Hadoop
  • Implement multi variant iterator
  • Be aware of limit-skip. If base query is not null, will need client side skip
  • Use hbase scan instead of phoenix query for queries fully resolved in solr? Not now.
    Maybe in a future, with Table#get(List<Get>) or MultiRowRangeFilter
• Implement approximated count
• summary parameter should be an alias for a set of included fields

New configuration parameters

intersect.active : true          # Allow intersect queries with the SearchEngine (Solr) 
intersect.always : false         # Force intersect queries 
intersect.params.threshold : 3   # Minimum number of QueryParams in the query to intersect