You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Full disclosure, I have only spent a handful of hours with the tool so if there is an easy fix for this my apologies.
I started with a data set where it's very common to see two ints following each other separated by a space (could be a single space or could be multiple spaces). When I go to the labeling UI, I noticed that the two ints are together as one token. They are even tokenized as one token when they are separated by a comma. Screenshots for full repro below.
Any thoughts in how I can get these to be separate tokens? Hoping there are some simple settings I can change.
The text was updated successfully, but these errors were encountered:
Full disclosure, I have only spent a handful of hours with the tool so if there is an easy fix for this my apologies.
I started with a data set where it's very common to see two ints following each other separated by a space (could be a single space or could be multiple spaces). When I go to the labeling UI, I noticed that the two ints are together as one token. They are even tokenized as one token when they are separated by a comma. Screenshots for full repro below.
Any thoughts in how I can get these to be separate tokens? Hoping there are some simple settings I can change.
The text was updated successfully, but these errors were encountered: