Add Words and Chars primitives

[Seth-Rothschild]

Add two text primitives, NumWords and NumCharacter which count the number of words and the number of characters when the variable type is Text.

[codecov-io]

Codecov Report

Merging #51 into master will increase coverage by 0.04%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #51      +/-   ##
==========================================
+ Coverage   87.14%   87.19%   +0.04%     
==========================================
  Files          74       74              
  Lines        6946     6973      +27     
==========================================
+ Hits         6053     6080      +27     
  Misses        893      893

Impacted Files	Coverage Δ
primitives/transform_primitive.py	97.46% <0%> (+0.13%)	⬆️
.../feature_function_tests/test_transform_features.py	85.45% <0%> (+0.37%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4d30e76...687e1b0. Read the comment docs.

[kmax12]

Good to merge after these changes

[kmax12]

Should we name this NumCharacters to be consistent with NumWords below?

[Seth-Rothschild]

Changed

[kmax12]

I think you can use more pandas built syntax for this since the array variable is going to be a pandas series (please double check this though)

In [1]: array = pd.Series(["1","12 2","1212 3"])

In [2]: array.str.len()
Out[2]: 
0    1
1    4
2    6
dtype: int64

so, it would just be

def get_function(self):
    return lambda array: array.str.len()

[Seth-Rothschild]

Array is a numpy.ndarray

[kmax12]

thanks for double checking. I don't have strong preference, but perhaps do this to avoid the list comprehension

pd.Series(array).str.len()

[kmax12]

similar to above, you can do

def get_function(self):
    return lambda array: array.str.split(" ").str.len()

[kmax12]

actually, this is probably better.

def get_function(self):
    return lambda array: array.str.count(" ") + 1

easier to read and might be up to 25% faster

[PaulHobbs]

Those aren't quite equivalent - what if there's some leading or trailing whitespace, or if there's more than one space between characters?

[kmax12]

I think both pieces of code handle white space and more than one space the same way because they look for just " ". To do the multiple spaces, the only way I can think of is a regex and I rather keep it simple for now. However, for the trailing or leading white space we could do

def get_function(self):
    return lambda array: array.str.strip().str.count(" ") + 1

[Seth-Rothschild]

Multiple spaces could be taken care of in the str.split(" ") case since they'll show up as empty strings. Could just remove with

new_list = [x for x in split_list if x != '']

For now, it has been changed to str.count(" ") + 1