StringUtil.tokenize with delimiter "__" issue

Original issue 1246 created by codenameone on 2014-12-29T05:26:23.000Z:

I was tokenizing Strings with the delimeter "__" and realized the tokens happen with _ too

    String entry = "AdsStatData__ad_clicks__724__2014_12_29_03_49_04";
    java.util.List<String> list = StringUtil.tokenize(entry, "__");
    for (String list1 : list) {
        System.out.println(list1);
    }

This produces a list with size ten instead of size 4.
My observation is whenever there is any trace of _ in the delimiter same issue happens so

    String entry = "AdsStatDatan_ad_clicksn_724n_2014_12_29_03_49_04";
    java.util.List<String> list = StringUtil.tokenize(entry, "n_");
    for (String list1 : list) {
        System.out.println(list1);
    }


    String entry = "AdsStatData_nad_clicks_n724_n2014_12_29_03_49_04";
    java.util.List<String> list = StringUtil.tokenize(entry, "_n");
    for (String list1 : list) {
        System.out.println(list1);
    }

I initially thought it was more than one character issue but realized after several test that is has to do with _ only

This works perfectly and gets the size 4 list
String entry = "AdsStatDatannad_clicksnn724nn2014_12_29_03_49_04";
java.util.List<String> list = StringUtil.tokenize(entry, "nn");
for (String list1 : list) {
System.out.println(list1);
}

Comment #1 originally posted by codenameone on 2015-01-05T06:47:56.000Z:

That's how StringTokenizer works on JavaSE.