Upgrade to Lucene 4.8

[larsga]

From lar...@gmail.com on December 31, 2012 14:10:16

Lucene 4.0 is out now, and we should try to use it.

Original issue: http://code.google.com/p/duke/issues/detail?id=84

[larsga]

From lar...@gmail.com on January 14, 2013 05:24:52

Compiling causes about 10 compilation errors, and they're non-trivial to fix, because there's no documentation of how the API has changed. So basically, every single error requires figuring out what the changes are, and some them seem fairly deep.

Labels: Component-Core

[larsga]

From yann.bar...@gmail.com on January 14, 2013 05:33:00

Hi, http://lucene.apache.org/core/4_0_0/MIGRATE.html Won't this help ?

[larsga]

From lar...@gmail.com on January 14, 2013 06:04:34

Yes, that helps a lot. Thank you! :)

[larsga]

From lar...@gmail.com on January 24, 2013 23:08:17

While we waited, 4.1 came out, so let's upgrade to that instead.

[larsga]

From lar...@gmail.com on January 25, 2013 01:55:45

Yes, there are a number of new features I want to try, but haven't had time yet. Feel free to add issues for any specific ones not already in the issue list.

Also, if you want to commit patches for the 4.x migration (or anything else), please feel encouraged to do it.

[larsga]

From lar...@gmail.com on February 01, 2013 10:38:02

Have started porting now, but tests fail. Will continue working on this tomorrow. http://code.google.com/p/duke/source/detail?r=cf75f271bed34b9b0e888684902551b93abdff0d&name=lucene4

[larsga]

From lar...@gmail.com on February 01, 2013 10:38:16

Status: Started

[larsga]

From lar...@gmail.com on February 01, 2013 10:39:59

Blocking: duke:26

[larsga]

From lar...@gmail.com on February 02, 2013 01:19:05

Well, with the latest commit we have everything working on Lucene 4.1. However, it seems to run at roughly half the speed of Lucene 3.6. That's not very good. Need to consider what to do.

[larsga]

From yann.bar...@gmail.com on February 12, 2013 00:46:39

Sorry not to help more. At the moment, I'm focused on Elasticsearch tests...

[larsga]

From lar...@gmail.com on February 24, 2013 03:44:02

It turns out Lucene 4.0 is faster than both 4.1 and 3.6, so we'll go for 4.0 for the moment, while we try to figure out how to improve performance on 4.1.

This question looks like it might contain a few clues: http://stackoverflow.com/questions/14851780/solve-terrible-performance-after-upgrading-from-lucene-4-0-to-4-1

[larsga]

From lar...@gmail.com on February 24, 2013 07:58:28

Tried adding this code:

import org.apache.lucene.codecs.FilterCodec;
import org.apache.lucene.codecs.StoredFieldsFormat;
import org.apache.lucene.codecs.lucene41.Lucene41Codec;
import org.apache.lucene.codecs.lucene40.Lucene40StoredFieldsFormat;

    cfg.setCodec(new CustomCodec());

public final class CustomCodec extends FilterCodec {
public CustomCodec() {
super("CustomCodec", new Lucene41Codec());
}

public StoredFieldsFormat storedFieldsFormat() {
  return new Lucene40StoredFieldsFormat();
}

}

It didn't work, though, because the CustomCodec needs to be registered in some kind of SPI framework so that Lucene can read the index. Will look at that later.

[larsga]

From lar...@gmail.com on March 16, 2013 04:40:01

Blocking: -duke:26

[larsga]

This is becoming more urgent as new Lucene versions are released. Need to take a serious new look at this.