[FEA] Implement extract_all_re function

[andygrove]

Is your feature request related to a problem? Please describe.
I would like to be able to implement a GPU version of Spark's regexp_extract_all function. Here is an example usage from the documentation.

SELECT regexp_extract_all('100-200, 300-400', '(\\d+)-(\\d+)', 1);
 ["100","300"]

Describe the solution you'd like
I would like a cuDF function extract_all_re which is similar to extract_re but returns columns of type List<String> containing all matches for each group.

Additionally, it would be nice if I could optionally pass cuDF a group index so that we don't always have to extract for all of the groups in the pattern. There is a related issue #9855 asking for this capability for extract_re.

Describe alternatives you've considered
None

Additional context
None

[davidwendt]

Looking a bit closer, this does appear to be similar to extract_record but the all means continue match/extracting after the first match is found? Also, I'm guessing the example in the description includes an index feature that is the subject of a separate issue #9855 and therefore is a bit confusing here. Ignoring indexing, this is what I'm looking at implementing so far:

s = [ "100-200 300-400", "500-600", "700-800", "900" ]
r =  extract_all( s, "'(\d+)-(\d+)" )
r is a LIST column of strings that looks like this:

[ [ "100", "200", "300", "400" ], // 2 sets of matches
  [ "500", "600" ], // 1 match
  [ "700", "800" ], // 1 match
  NULL              // no match
]

The 1st string "100-200 300-400" had 2 matches to (\d+)-(\d+) and so both sets of groups are returned.
The 2nd and 3rd strings each had one match and included those in the output rows.
The 4th string "900" does not match the pattern and so the output for this row is NULL.

[andygrove]

Yes, the all here means continue matching. Your example looks correct to me, ignoring indexing. For reference, here is the equivalent example in Spark:

scala> val df = Seq("100-200 300-400", "500-600", "700-800", "900").toDF("str")
scala> df.createOrReplaceTempView("strings")

scala> spark.sql("SELECT regexp_extract_all(str, '(\\\\d+)-(\\\\d+)', 0) FROM strings").show
+---------------------------------------+
|regexp_extract_all(str, (\d+)-(\d+), 0)|
+---------------------------------------+
|                     [100-200, 300-400]|
|                              [500-600]|
|                              [700-800]|
|                                     []|
+---------------------------------------+

scala> spark.sql("SELECT regexp_extract_all(str, '(\\\\d+)-(\\\\d+)', 1) FROM strings").show
+---------------------------------------+
|regexp_extract_all(str, (\d+)-(\d+), 1)|
+---------------------------------------+
|                             [100, 300]|
|                                  [500]|
|                                  [700]|
|                                     []|
+---------------------------------------+

scala> spark.sql("SELECT regexp_extract_all(str, '(\\\\d+)-(\\\\d+)', 2) FROM strings").show
+---------------------------------------+
|regexp_extract_all(str, (\d+)-(\d+), 2)|
+---------------------------------------+
|                             [200, 400]|
|                                  [600]|
|                                  [800]|
|                                     []|
+---------------------------------------+

[davidwendt]

Please do not include indexing examples here.

I don't understand the first example.

+---------------------------------------+
|regexp_extract_all(str, (\d+)-(\d+), 0)|
+---------------------------------------+
|                     [100-200, 300-400]|
|                              [500-600]|
|                              [700-800]|
|                                     []|
+---------------------------------------+

This does not look like extract to me.
Extract will return the values found inside the groups ( ). Like [ "100", "200" ] in my previous comment post.
Your example now appears to return the entire match result instead. Like [ "100-200" ]

[andygrove]

The Spark function is confusing. A zero index is a special case and returns all of the strings that match the entire regexp pattern. I will see if the existing cuDF functions can already support this case and will file a new issue if not.

For the "normal" indexing case I think I can implement based on the proposed results that you previously posted.

[davidwendt]

I think the closest behavior we have is findall
But I suspect you will need _record version of this API to return a lists column instead of a table. This would match the result in this comment.

Do you still need an extract_all as I've described in my comment?
Or perhaps an extract_record that would be similar to this but would only return the first matched group set.

[andygrove]

Do you still need an extract_all as I've described in my comment?

Yes, I believe that this would give me what I need.