Skip to content
This repository has been archived by the owner on May 14, 2023. It is now read-only.

Commit

Permalink
add ":" to the list of default prefixes
Browse files Browse the repository at this point in the history
  • Loading branch information
nicolas.sassi committed Dec 26, 2020
1 parent 18dbb86 commit 0431e34
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion tokenize.go
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ var sanitizer = strings.NewReplacer(
"’", "'")
var contractions = []string{"'ll", "'s", "'re", "'m", "n't"}
var suffixes = []string{",", ")", `"`, "]", "!", ";", ".", "?", ":", "'"}
var prefixes = []string{"$", "(", `"`, "["}
var prefixes = []string{"$", "(", `"`, "[", ":"}
var emoticons = map[string]int{
"(-8": 1,
"(-;": 1,
Expand Down

0 comments on commit 0431e34

Please sign in to comment.