Mysterious stemmed token '_yyyyy_' in newer quanteda versions

[cschwem2er]

Hi,

I was experiencing strange stemming behavior for quite some time, but was not quite sure what's going on: On some of my PC's stemming with quanteda produced a very strange token '_yyyyy_' and I had no idea on which stem it was based upon.

Today I updated quanteda on my server (where I never experienced this problem before) and after the update I had the exact same problem. I think the last time I updated the server was about two months ago, so somewhere in between was a change related to this.

Do you have an idea where this is coming from? All my documents are encoded in utf-8. I'm also pretty sure that '_yyyyy_' is being created from a very common pattern (maybe even whitespace?), as it occurs very frequently.

Edit: I just double checked and can confirm that '_yyyyy_' appears in every single document:

print(Dfm)

Document-feature matrix of: 932,854 documents, 422,224 features.

topfeatures(Dfm)

    _yyyyy_         mal       schon          ja        mehr deutschland     deutsch         uns 
     932854      102544      100035       96696       88126       84847       73906       62030 
      polit        land 
      59199       59041 

Cheers,
Carsten

[kbenoit]

Well spotted!