Refactor XmlSeeker to support more types of return values #55

jimhester · 2015-11-05T15:34:08Z

This refactors the XmlSeeker class to return RObjects rather than strictly XPtrNode or Lists. It provides a general search() method that directly returns the result RObject, and an exported xpath_search() function which provides a superset of the behavior of the previous node_find_all(), node_find_one() functions. It also allows one to return a user defined subset of the results (1:max found), rather than just 1 or all.

The existing behavior and user facing API is preserved, however you can also use xpath_search() to return booleans, numbers, characters, e.g.

x <- read_xml("<blah></blah>")
xpath_search(x$node, x$doc, "1+1", nsMap = character(), num_results = Inf)
#> [1] 2
xpath_search(x$node, x$doc, "1=1", nsMap = character(), num_results = Inf)
#> [1] TRUE
xpath_search(x$node, x$doc, "2=1", nsMap = character(), num_results = Inf)
#> [1] FALSE
xpath_search(x$node, x$doc, "string(5)", nsMap = character(), num_results = Inf)
#> [1] "5"

I figured we should discuss the R API for this, so I did not flesh that out at all or add tests, but it is a functional first pass.

jimhester · 2015-11-07T14:53:45Z

Character returns also seem to be working fine, I edited the above comment respectively.

jimhester · 2015-11-07T14:58:15Z

Some libxml2 tests for return types

hadley · 2015-11-09T16:13:11Z

src/xml2_xpath.cpp

+          }
+          return out;
+        }
+      case XPATH_NUMBER: { return Rcpp::NumericVector(1, result_->floatval); }


You can use Rcpp::NumericVector::create(result_->floatval)

hadley · 2015-11-09T16:16:06Z

src/xml2_xpath.cpp

+        }
+      case XPATH_NUMBER: { return Rcpp::NumericVector(1, result_->floatval); }
+      case XPATH_BOOLEAN: { return Rcpp::LogicalVector(1, result_->boolval); }
+      case XPATH_STRING: { return Rcpp::CharacterVector(1, reinterpret_cast<const char*>(result_->stringval)); }


Need to make sure this gets correctly encoded

hadley · 2015-11-09T16:18:19Z

I prefer type-stable functions, so maybe the R interface should be: xml_find_num(), xml_find_chr(), and xml_find_lgl(). They'd throw an error if the input wasn't the correct type.


        Prototype of XmlSeeker refactoring


        Tests passing for current behavior


        Modify logic slightly to allow for alternate types


        Dispatch nodes_duplicated based on calss of input nodes.


        Support character outputs from xpath_search


        Handle missing nodes.

Return an empty list on missing and add default methods for all xml_ methods.


        Re-oxygenate


        Use proper function signature


        Return an explicit missing object rather than an empty list


        Remove duplicated nodes in xml_nodeset()


        Remove qualified names from Rcpp namespace


        Return xml_missing objects for xml_parent and xml_find_one


        Export the xml_missing methods


        Use create methods


        Properly encode XmlChar result


        xml_find_num, xml_find_chr, xml_find_lgl generic and methods


        Add xml_find_num, xml_find_chr and xml_find_lgl.


        Add tests


        Use call. = FALSE to avoid printing stack trace

jimhester · 2015-11-10T15:27:44Z

@hadley I think this is ready for review, I added the type specific functions as well as tests for them. The only thing remaining would be tests for the missing functionality if you think we need them.

I was getting segfaults when I tried to use the Xml2String() class to convert the XmlChar (see jimhester@142c95e). I do not get them with the reinterpret_cast. Maybe Xml2String is not doing a deep copy of the data and the character array is being freed when the XmlSeeker destructor is called? What is the best way to force a deep copy with Rcpp? Also Xml2String::asRString() seems to return a CHARXSP rather than a CharacterVector, is using Rcpp::as<CharacterVector> the best way to convert that?

hadley · 2015-11-10T20:15:20Z

I think it's because that object should actually be a const xmlChar* - Xml2String assumes if you pass it in xmlChar* it needs to free it.

A slightly better fix is CharacterVector::create(Rf_mkCharCE((char*) result_->stringval, CE_UTF8)), which ensures that the string is always flagged as UTF-8 in R. I'll make the change

hadley · 2015-11-10T20:22:41Z

Hmmm, I think there's something wrong in the logic for nodes_duplicated:

x <- read_xml("<body>
  <p>Some <b>text</b>.</p>
  <p>Some <b>other</b> <b>text</b>.</p>
  <p>No bold text</p>
</body>")
para <- xml_find_all(x, ".//p")

xml_find_one(para, ".//b")
# Error: expecting an external pointer

I'm not sure about the logic in there - should it be checking for xml_missing? What does as<XPtrNode>(List(nodes[i])["node"]) do?

jimhester · 2015-11-11T19:35:34Z

Re: nodes_duplicated(), you are correct, it needs to check for xml_missing as well. I wrote that code before merging the two pull requests and did not update it properly. I will add a test case for this.


        Use suggested cast to force UTF-8 encoding.


        Add tests for xml_missing for all xml_ methods

jimhester · 2015-11-11T21:44:03Z

0a12a80 adds tests for all the methods on xml_missing objects and contains the fix for the xml_find_one error you mentioned. Also added support for xml_missing to nodes_duplicated.

hadley · 2015-11-11T22:43:49Z

Ok, this looks great :)

Two last things:

Can you please add a couple of bullets to NEWS
Can you please update the xml_find_all() example to the code below:

#' # Find all vs find one -----------------------------------------------------
#' x <- read_xml("<body>
#'   <p>Some <b>text</b>.</p>
#'   <p>Some <b>other</b> <b>text</b>.</p>
#'   <p>No bold here!</p>
#' </body>")
#' para <- xml_find_all(x, ".//p")
#'
#' # If you apply xml_find_all to a nodeset, it finds all matches,
#' # de-duplicates them, and returns as a single list. This means you
#' # never know how many results you'll get
#' xml_find_all(para, ".//b")
#'
#' # xml_find_one only returns one match per input node. If there are 0
#' # matches it will return a missing node; if there are more than one it picks
#' # the first with a warning
#' xml_find_one(para, ".//b")
#' xml_text(xml_find_one(para, ".//b"))


        Add note to NEWS


        Update xml_find_one() examples to include missing node


        Add missing test file

jimhester · 2015-11-11T23:14:15Z

I missing committing the new test file (whoops) in the previous round, but it is now added, as well as the documentation and notes to NEWS.


        Merge pull request #55 from jimhester/numeric

Refactor XmlSeeker to support more types of return values

hadley · 2015-11-12T12:34:30Z

Thanks Jim!

jimhester changed the title ~~Refactor XmlSeeker to support More types of return values~~ Refactor XmlSeeker to support more types of return values Nov 6, 2015

hadley reviewed Nov 9, 2015
View changes

src/xml2_xpath.cpp Outdated

}

return out;

}

case XPATH_NUMBER: { return Rcpp::NumericVector(1, result_->floatval); }

This comment has been minimized.

Sign in to view

Copy link

hadley Nov 9, 2015
Member

You can use Rcpp::NumericVector::create(result_->floatval)

hadley reviewed Nov 9, 2015
View changes

jimhester added 19 commits Nov 5, 2015

Prototype of XmlSeeker refactoring

186e0b4

Tests passing for current behavior

6c01b7b

Modify logic slightly to allow for alternate types

e1f90b6

Dispatch nodes_duplicated based on calss of input nodes.

683fe86

Support character outputs from xpath_search

25b4042

Handle missing nodes.

4b5340e

Return an empty list on missing and add default methods for all xml_ methods.

Re-oxygenate

a2e95a6

Use proper function signature

03ac616

Return an explicit missing object rather than an empty list

d396dfe

Remove duplicated nodes in xml_nodeset()

324f0d7

Remove qualified names from Rcpp namespace

32f7bc2

Return xml_missing objects for xml_parent and xml_find_one

18c09e6

Export the xml_missing methods

964a91e

Use create methods

4a00ff9

Properly encode XmlChar result

142c95e

xml_find_num, xml_find_chr, xml_find_lgl generic and methods

5aa9229

Add xml_find_num, xml_find_chr and xml_find_lgl.

37dd4ba

Add tests

e1353ad

Use call. = FALSE to avoid printing stack trace

Loading status checks…

811bf67

jimhester force-pushed the jimhester:numeric branch from 1e72e64 to 811bf67 Nov 10, 2015

jimhester mentioned this pull request Nov 10, 2015

Handle missing nodes #57

Closed

jimhester added 2 commits Nov 11, 2015

Use suggested cast to force UTF-8 encoding.

052607b

Add tests for xml_missing for all xml_ methods

Loading status checks…

0a12a80

jimhester added 3 commits Nov 11, 2015

Add note to NEWS

f1e91ec

Update xml_find_one() examples to include missing node

d1c4ae0

Add missing test file

Loading status checks…

1143aad

hadley added a commit that referenced this pull request Nov 12, 2015

Merge pull request #55 from jimhester/numeric

Loading status checks…

59c2c2f

Refactor XmlSeeker to support more types of return values

hadley merged commit 59c2c2f into r-lib:master Nov 12, 2015
1 check passed

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

r-lib/xml2

Refactor XmlSeeker to support more types of return values #55

Refactor XmlSeeker to support more types of return values #55

jimhester commented Nov 5, 2015

This comment has been minimized.

jimhester commented Nov 7, 2015

This comment has been minimized.

jimhester commented Nov 7, 2015

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

hadley commented Nov 9, 2015

This comment has been minimized.

jimhester commented Nov 10, 2015

This comment has been minimized.

hadley commented Nov 10, 2015

This comment has been minimized.

hadley commented Nov 10, 2015

This comment has been minimized.

jimhester commented Nov 11, 2015

This comment has been minimized.

jimhester commented Nov 11, 2015

This comment has been minimized.

hadley commented Nov 11, 2015

This comment has been minimized.

jimhester commented Nov 11, 2015

This comment has been minimized.

hadley commented Nov 12, 2015

r-lib/xml2

Join GitHub today

Refactor XmlSeeker to support more types of return values #55

Refactor XmlSeeker to support more types of return values #55

Conversation

jimhester commented Nov 5, 2015

This comment has been minimized.

jimhester commented Nov 7, 2015

This comment has been minimized.

jimhester commented Nov 7, 2015

This comment has been minimized.

hadley Nov 9, 2015

This comment has been minimized.

hadley Nov 9, 2015

This comment has been minimized.

hadley commented Nov 9, 2015

This comment has been minimized.

jimhester commented Nov 10, 2015

This comment has been minimized.

hadley commented Nov 10, 2015

This comment has been minimized.

hadley commented Nov 10, 2015

This comment has been minimized.

jimhester commented Nov 11, 2015

This comment has been minimized.

jimhester commented Nov 11, 2015

This comment has been minimized.

hadley commented Nov 11, 2015

This comment has been minimized.

jimhester commented Nov 11, 2015

This comment has been minimized.

hadley commented Nov 12, 2015