adding a toJSON method for custom class

[erikriverson]

Hello, I would like to implement a toJSON method for a custom class.

In "The jsonlite Package: A Practical and Consistent Mapping Between JSON Data and R Objects", I see the following:

"For all of the common classes in R, the jsonlite package implements toJSON methods as described in this document. Users in R can extend this system by implementing additional methods for other classes. This also means that classes that do not have the toJSON method defined are not supported."

However, inspecting the implementation of toJSON, it seems that the above description might actually apply to the "asJSON" function. Is this accurate?

Based on my reasoning above, I then tried the following for the class 'foo', and a trivial asJSON function:

setMethod("asJSON", "foo", function(x, collapse = TRUE, na = NULL, oldna = NULL,
  is_df = FALSE, auto_unbox = FALSE, ...) {
    x
})

Error in setMethod("asJSON", "foo", function(x, collapse = TRUE, na = NULL,  (from unknown@4053-wR#1) : no existing definition for function ‘asJSON’

I'm guessing this is because asJSON is not exported from the jsonlite package. The above command does work when jsonlite is loaded into R via the load_all function in devtools (asJSON is then visible on the search path.)

Thanks!

[jeroen]

Hmm. The asJSON methods now has so many internal arguments that I am not sure anymore if it is still a good idea to add methods for specific classes. The problem is also that you will get different JSON output when trying to convert a particular object, depending on whether the package with the extended methods is loaded.

Can you explain how you are trying to use this feature?

Note that you can always define a new method which wraps around toJSON, something like this:

myJSON <- function(x, ...){
  UseMethod("myJSON")
}

myJSON.default <- jsonlite::toJSON;

myJSON.foo <- function(x, ...){
  return('["this is the foo"]')
}

test <- 1:3
class(test) <- "foo"

myJSON(test)
myJSON(1:3)

[flying-sheep]

as a hack, i’m doing this:

asJSON <- jsonlite:::asJSON
setMethod('asJSON', 'repr', function(x, ...) jsonlite:::asJSON(unclass(x), ...))

because it’s enough for my use case. it would be nice to have some hook for own classes though.

[yihui]

For the record, #90 was supposed to solve this kind of problems. The approach @jeroenooms mentioned here has a problem, that is it does not work on recursive data structures such as lists, e.g. list(x = 1, y = structure(2, class = 'foo')); it only works on the top-level class of an object.

I understand the point of consistency, but there is no absolute consistency in the universe. This is a general issue regarding reproducibility: I will not be surprised that toJSON() produces different results if users call it differently. What software authors should guarantee is that the software should give identical results when two environments are identical. I don't think you would think the difference between toJSON(..., digits = 5) and toJSON(..., digits = 16) is a problem of jsonlite, and forbid the the possibility of customizing the digits argument accordingly. Similarly, users getting different output by defining different methods for a certain class should not be a problem.

[flying-sheep]

of course it shouldn’t. after all, they define those methods exactly for the purpose of making toJSON work the way they want

[jeroen]

@yihui the problem with custom asJSON methods is that the output will not depend only on parameters specified by the user (as in your digits example) but it depends on which 3rd party packages happen to be loaded at that point.

For example if a package foo implements a special list class and an asJSON.foo method. Then when the user runs toJSON(foo) it gets one output. But if the user saves the object to disk, and later tries to run toJSON(foo) again, he will get different results if he didn't explicitly load the foo package first.

Also now that asJSON methods have to support R based indentation, implementing custom methods has become even more difficult and error prone. I think it is much safer to let users and package authors write a simple wrapper function that converts your special class to a basic data types (list, dataframe, vector) which will map naturally to JSON and back.

[yihui]

1. If the user forgot to load an important package, I think that is completely the user's fault. The user must be responsible for his/her own R session, and I don't it is anything you or the author of asJSON.foo has to worry about -- you load it, you get it; you don't, you fail. That sounds pretty fair to me.
2. 1 is not the end of the world even if it occurs, because by default it will lead to an error if the method for foo is not defined. Users will not get different results -- actually they will not get results at all. It should be fairly straightforward to discover such errors.
3. If you don't think digits or other arguments of toJSON() is a problem, then Make toJSON() extensible on other classes #90 should not be a problem, either, since both digits and force are arguments (are all arguments created equal, or some are more equal than others?...).
4. As I said, your myJSON.foo only works for the top-level class, and it will be very awkward if one has to define all methods of jonslite:::asJSON, e.g. myJSON.list = function(x, ...) asJSON(x, ...), myJSON.matrix = function(...), myJSON.data.frame = function(...), myJSON.foo = function(...), ... When there is an "obvious" way to extend asJSON, you probably don't feel very good if you have to extend it by providing wrapper methods for all existing methods.
5. If you do not like users to extend asJSON, Make toJSON() extensible on other classes #90 is a quick-n-dirty workaround, although extending asJSON should still be the canonical way. Depending how many users want to extend how many classes, Make toJSON() extensible on other classes #90 may "just work", or we still have to count on you to open the door of asJSON to us.
6. I'd say the indentation problem is solvable if you expose the indent argument to the asJSON method. The worst case is one uses his/her R code (the paste() family) or C code to add the indentation. I imagine in most cases he/she does not have to do so, if he/she is able to coerce his data object to base R classes (lists, vectors, etc), and just call the next asJSON method.

To sum up, I think there are two choices in front of you when users ask for extensibility: 1) No, you are out of luck when your class is unknown to me, or you have to do hard and boring work to reinvent the S4 mechanism. 2) Yes, here I grant you the power, but remember your responsibility at the same time, since you will no longer be blessed by the Godfather. You seem to vote for 1, and I will vote for 2.

BTW, I do not really desire this feature at the moment. I have got all I wanted from you (many thanks), but I'm just writing my thoughts here since I guess there will certainly be more users asking this question.

[flying-sheep]

i observed that when i load a .rda/.RData file that sometimes R attaches a package (e.g. "loading required package flowCore")

am i correct that this means that for S4 classes, packages are automatically loaded?

if so, asJSON will automatically do the right thing as long as the necessary asJSON method is defined in the same package as the class (and therefore always loaded when handling an instance of that class)

[mmuurr]

I'm curious if over the years there's been a coalescence towards a 'standard' (or 'recommended') way to handle this type of scenario with {jsonlite}. I love working with {jsonlite}, except for the frequent want for both:

1. custom asJSON() (or toJSON()) implementations and
2. support for those custom methods with the base recursive functionality used by, e.g., asJSON.list().

It seems (just from reading the various GitHub Issues threads) that writing one's own asJSON() methods is frowned-upon, but I'm not sure how else to deal with the recursive aspect.

I routinely do resort to registering custom asJSON() implementations for my classes, like so:

setMethod(getGeneric("asJSON", package = "jsonlite"), "mycls", function(x, ...) to_json.mycls(x, ...))

... where to_json.mycls() would be my implementation, usually itself calling jsonlite::toJSON() with some specific parameter settings.

This approach works particularly well when mycls is an R6 class, and instead of to_json.mycls(), the class has an asJSON() R6 method defined, so the registration looks more like:

setMethod(getGeneric("asJSON", package = "jsonlite"), "mycls", function(x, ...) x$asJSON(...))

(Note that this can cause some headaches when R6 inheritance is used (and we'd like to inherit the $asJSON() method), however, since the jsonlite::asJSON.ANY() dispatcher modifies reference objects in-place.)

I realize that having methods available (or not) via namespace-loading can then become a source of inconsistency, but I think I agree with @yihui that the authors and users of packages need to agree to good writing and reading of documentation, else all bets are off.

I also see the point that {jsonlite} was originally intended for 1:1 serialization:deserialization of data within the R ecosystem, but I imagine a lot of usage nowadays is for two-way communication with external APIs, where the JSON requirements are outside of the R developer's control. (Nested lists of specially-encoded objects are the most common case of this problem for me -- I want to use {jsonlite}'s default list recursion but need to create special structures that may themselves have lists of other special structures.)

In any case, I do think {jsonlite}'s been a huge help to the R community, so this isn't a complaint -- just musings and curiousness about others' patterns. I'd be more than happy to collect such patterns, discuss the pros & cons, and write up a vignette on this issue, too, if that'd be helpful.