Refactor XmlSeeker to support more types of return values #55

jimhester · 2015-11-05T15:34:08Z

This refactors the XmlSeeker class to return RObjects rather than strictly XPtrNode or Lists. It provides a general search() method that directly returns the result RObject, and an exported xpath_search() function which provides a superset of the behavior of the previous node_find_all(), node_find_one() functions. It also allows one to return a user defined subset of the results (1:max found), rather than just 1 or all.

The existing behavior and user facing API is preserved, however you can also use xpath_search() to return booleans, numbers, characters, e.g.

x <- read_xml("<blah></blah>")
xpath_search(x$node, x$doc, "1+1", nsMap = character(), num_results = Inf)
#> [1] 2
xpath_search(x$node, x$doc, "1=1", nsMap = character(), num_results = Inf)
#> [1] TRUE
xpath_search(x$node, x$doc, "2=1", nsMap = character(), num_results = Inf)
#> [1] FALSE
xpath_search(x$node, x$doc, "string(5)", nsMap = character(), num_results = Inf)
#> [1] "5"

I figured we should discuss the R API for this, so I did not flesh that out at all or add tests, but it is a functional first pass.

jimhester · 2015-11-07T14:53:45Z

Character returns also seem to be working fine, I edited the above comment respectively.

jimhester · 2015-11-07T14:58:15Z

Some libxml2 tests for return types

hadley · 2015-11-09T16:13:11Z

src/xml2_xpath.cpp

+          }
+          return out;
+        }
+      case XPATH_NUMBER: { return Rcpp::NumericVector(1, result_->floatval); }


You can use Rcpp::NumericVector::create(result_->floatval)

hadley · 2015-11-09T16:18:19Z

I prefer type-stable functions, so maybe the R interface should be: xml_find_num(), xml_find_chr(), and xml_find_lgl(). They'd throw an error if the input wasn't the correct type.

Return an empty list on missing and add default methods for all xml_ methods.

jimhester · 2015-11-10T15:27:44Z

@hadley I think this is ready for review, I added the type specific functions as well as tests for them. The only thing remaining would be tests for the missing functionality if you think we need them.

I was getting segfaults when I tried to use the Xml2String() class to convert the XmlChar (see jimhester@142c95e). I do not get them with the reinterpret_cast. Maybe Xml2String is not doing a deep copy of the data and the character array is being freed when the XmlSeeker destructor is called? What is the best way to force a deep copy with Rcpp? Also Xml2String::asRString() seems to return a CHARXSP rather than a CharacterVector, is using Rcpp::as<CharacterVector> the best way to convert that?

hadley · 2015-11-10T20:15:20Z

I think it's because that object should actually be a const xmlChar* - Xml2String assumes if you pass it in xmlChar* it needs to free it.

A slightly better fix is CharacterVector::create(Rf_mkCharCE((char*) result_->stringval, CE_UTF8)), which ensures that the string is always flagged as UTF-8 in R. I'll make the change

hadley · 2015-11-10T20:22:41Z

Hmmm, I think there's something wrong in the logic for nodes_duplicated:

x <- read_xml("<body>
  <p>Some <b>text</b>.</p>
  <p>Some <b>other</b> <b>text</b>.</p>
  <p>No bold text</p>
</body>")
para <- xml_find_all(x, ".//p")

xml_find_one(para, ".//b")
# Error: expecting an external pointer

I'm not sure about the logic in there - should it be checking for xml_missing? What does as<XPtrNode>(List(nodes[i])["node"]) do?

jimhester · 2015-11-11T19:35:34Z

Re: nodes_duplicated(), you are correct, it needs to check for xml_missing as well. I wrote that code before merging the two pull requests and did not update it properly. I will add a test case for this.

jimhester · 2015-11-11T21:44:03Z

0a12a80 adds tests for all the methods on xml_missing objects and contains the fix for the xml_find_one error you mentioned. Also added support for xml_missing to nodes_duplicated.

hadley · 2015-11-11T22:43:49Z

Ok, this looks great :)

Two last things:

Can you please add a couple of bullets to NEWS
Can you please update the xml_find_all() example to the code below:

#' # Find all vs find one -----------------------------------------------------
#' x <- read_xml("<body>
#'   <p>Some <b>text</b>.</p>
#'   <p>Some <b>other</b> <b>text</b>.</p>
#'   <p>No bold here!</p>
#' </body>")
#' para <- xml_find_all(x, ".//p")
#'
#' # If you apply xml_find_all to a nodeset, it finds all matches,
#' # de-duplicates them, and returns as a single list. This means you
#' # never know how many results you'll get
#' xml_find_all(para, ".//b")
#'
#' # xml_find_one only returns one match per input node. If there are 0
#' # matches it will return a missing node; if there are more than one it picks
#' # the first with a warning
#' xml_find_one(para, ".//b")
#' xml_text(xml_find_one(para, ".//b"))

jimhester · 2015-11-11T23:14:15Z

I missing committing the new test file (whoops) in the previous round, but it is now added, as well as the documentation and notes to NEWS.

Refactor XmlSeeker to support more types of return values

hadley · 2015-11-12T12:34:30Z

Thanks Jim!

jimhester changed the title ~~Refactor XmlSeeker to support More types of return values~~ Refactor XmlSeeker to support more types of return values Nov 6, 2015

hadley reviewed Nov 9, 2015
View changes

jimhester added 19 commits Nov 10, 2015

Prototype of XmlSeeker refactoring

186e0b4

Tests passing for current behavior

6c01b7b

Modify logic slightly to allow for alternate types

e1f90b6

Dispatch nodes_duplicated based on calss of input nodes.

683fe86

Support character outputs from xpath_search

25b4042

Handle missing nodes.

4b5340e

Return an empty list on missing and add default methods for all xml_ methods.

Re-oxygenate

a2e95a6

Use proper function signature

03ac616

Return an explicit missing object rather than an empty list

d396dfe

Remove duplicated nodes in xml_nodeset()

324f0d7

Remove qualified names from Rcpp namespace

32f7bc2

Return xml_missing objects for xml_parent and xml_find_one

18c09e6

Export the xml_missing methods

964a91e

Use create methods

4a00ff9

Properly encode XmlChar result

142c95e

xml_find_num, xml_find_chr, xml_find_lgl generic and methods

5aa9229

Add xml_find_num, xml_find_chr and xml_find_lgl.

37dd4ba

Add tests

e1353ad

Use call. = FALSE to avoid printing stack trace

811bf67

jimhester force-pushed the numeric branch from 1e72e64 to 811bf67 Compare Nov 10, 2015

jimhester mentioned this pull request Nov 10, 2015

Handle missing nodes #57

Closed

jimhester added 2 commits Nov 11, 2015

Use suggested cast to force UTF-8 encoding.

052607b

Add tests for xml_missing for all xml_ methods

0a12a80

jimhester added 3 commits Nov 11, 2015

Add note to NEWS

f1e91ec

Update xml_find_one() examples to include missing node

d1c4ae0

Add missing test file

1143aad

hadley added a commit that referenced this issue Nov 12, 2015

Merge pull request #55 from jimhester/numeric

59c2c2f

Refactor XmlSeeker to support more types of return values

hadley merged commit 59c2c2f into r-lib:master Nov 12, 2015
1 check passed

Refactor XmlSeeker to support more types of return values #55

Refactor XmlSeeker to support more types of return values #55

jimhester commented Nov 5, 2015

jimhester commented Nov 7, 2015

jimhester commented Nov 7, 2015

hadley Nov 9, 2015

hadley commented Nov 9, 2015

jimhester commented Nov 10, 2015

hadley commented Nov 10, 2015

hadley commented Nov 10, 2015

jimhester commented Nov 11, 2015

jimhester commented Nov 11, 2015

hadley commented Nov 11, 2015

jimhester commented Nov 11, 2015

hadley commented Nov 12, 2015

Refactor XmlSeeker to support more types of return values #55

Refactor XmlSeeker to support more types of return values #55

Conversation

jimhester commented Nov 5, 2015

jimhester commented Nov 7, 2015

jimhester commented Nov 7, 2015

hadley Nov 9, 2015

Choose a reason for hiding this comment

hadley commented Nov 9, 2015

jimhester commented Nov 10, 2015

hadley commented Nov 10, 2015

hadley commented Nov 10, 2015

jimhester commented Nov 11, 2015

jimhester commented Nov 11, 2015

hadley commented Nov 11, 2015

jimhester commented Nov 11, 2015

hadley commented Nov 12, 2015