Explicit proto import search path

[siddharthab]

This tries to address #39.

The new function is more consistent with the behavior of protoc command line tool, and is easy enough for end users to get right. It expects that all provided paths are relative to an explicit search path.

With the previous function, the working dir was always added to the search path, and the files argument was always taken as an absolute path. I tried to change the existing function but it was difficult to be backward compatible in a clean way, hence the new function.

Open to suggestions on naming the new function.

[eddelbuettel]

That doesn't look unreasonable, and well localized. @murraystokely any views?

[murraystokely]

I'd like a self contained example of the problem we are solving before introducing a '2' function. I had a very clean way to do this previously, but I don't remember if I had local employer-specific wrappers to do this. I do feel there is a slightly simpler solution. Can the submitter add a small self-contained unit test that requires this function (and has two .proto files in different subduers of unitTests). If I can't find a better/cleaner way to get it to work, then yes we should just add this. - Murray

…

On Thu, Jan 18, 2018 at 5:19 AM, Dirk Eddelbuettel ***@***.*** > wrote: That doesn't look unreasonable, and well localized. @murraystokely <https://github.com/murraystokely> any views? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#40 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEwu7ZnK-R4YhSco_xCQG1QUBFapo0ZMks5tL0TKgaJpZM4RiiaY> .

[eddelbuettel]

Unit tests are always good. In favor.

[siddharthab]

Thank you. I just wanted some buy-in before I invested more into unit tests. I will send in an update soon.

[siddharthab]

I tried to write unit tests but it is impossible to get a clean environment in the unit test setup. Every call to readProtoFiles adds directories to the search path that are never removed.

rprotobuf/src/DescriptorPoolLookup.cpp

Line 80 in 8219122

// source_tree.removeDirectories( dirs ) ;

As one use case of where the existing function gets tricky, I added a new proto file to a subdirectory in the unit tests in grailbio-external@d780a38 and wrote unit tests using the new function. While there could be ways to make it work with the old function (mostly by changing working dir), I suspect they are not clean and intuitive. As of now, even the existing unit tests are failing because of the addition of this new import.

A rare use case that is impossible to make work with the existing function is when two directories have cyclical dependencies but are independent components in the search path. See grailbio-external@0f5e650 for an example. This case can be easily handled with protoc and the new function.

Another advantage of the new function is that you will no longer require to retain the search path, and will be able to clear it after every call. Retaining the search path is also bug-prone when multiple files have same basename because the user will not realize that the importer is using a file from a previously added directory and not the file that he intends.

If you want, I can write up some more detailed test scripts so that we get a clean environment at the start of each script. Short of that, we will have to introduce utility methods to clear state, etc.

[siddharthab]

As another example of how things are non-intuitive with existing function, this statement does not work when a user might expect it to work out of the box.

RProtoBuf::readProtoFiles(dir=system.file("unitTests", "data", package="RProtoBuf"))

[siddharthab]

Ping.

Would you like me to simplify the counter examples I have provided?

[rangi513]

I found this PR very useful for a project I'm working on! Is this going to be merged in to the CRAN version sometime soon? In the meantime I'm using siddharthab's branch.

[eddelbuettel]

@murraystokely Any views? On the margin. it probably won't hurt. Just a new entry point...

[eddelbuettel]

I would have liked to hear @murraystokely on this, but looks like I won't.

As the patch cleanly extends the API and makes no changes to existing code, I see no harm. You should examples of existing code not allowing you to do what you wanted, so on balance this is a net benefit.

Thanks for the PR. I'll merge it now,

[siddharthab]

Thank you!