Deprecate Token class and remove in trunk [LUCENE-3844]

[asfimport]

After issues like #4916, introducing new attributes, we should remove Token class in trunk, as it leads to code that ignores those new attributes (like PositionLengthAttribute, ScriptAttribute, KeywordAttribute,...). If you want a holder for all Attributes a TokenStream, use TS.cloneAttributes().

---

Migrated from LUCENE-3844 by Uwe Schindler (@uschindler), updated May 09 2016
Linked issues:

• implement PositionLengthAttribute for all tokenstreams where its appropriate [LUCENE-3843] #4916

[asfimport]

Uwe Schindler (@uschindler) (migrated from JIRA)

And finally: die, Token, die :-), see @UweSays: https://twitter.com/#!/UweSays/status/176082735792787456

[asfimport]

Robert Muir (@rmuir) (migrated from JIRA)

How much work is this? I think even removing would be difficult.

Maybe we should push deprecating this out to 4.0, giving us a lot of time to clean things up
before its totally removed.

[asfimport]

Chris M. Hostetter (@hossman) (migrated from JIRA)

Bulk changing fixVersion 3.6 to 4.0 for any open issues that are unassigned and have not been updated since March 19.

Email spam suppressed for this bulk edit; search for hoss20120323nofix36 to identify all issues edited

[asfimport]

Steven Rowe (@sarowe) (migrated from JIRA)

Bulk move 4.4 issues to 4.5 and 5.0

[asfimport]

Uwe Schindler (@uschindler) (migrated from JIRA)

Move issue to Lucene 4.9.