Working Travis Build #29

Paxanator · 2019-05-19T23:06:56Z

Making this mostly for commenting and help rather then testing the build. I'm in the process of testing the travis build on my forked repository, but it continues to fail R CMD CHECK, but passes on osx. It has to do with dynamic loading of the library

Error: package or namespace load failed for ‘fRanz’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/home/pboueri/fRanz.Rcheck/fRanz/libs/fRanz.so':
  librdkafka++.so.1: cannot open shared object file: No such file or directory```

This has lead me down the path of correctly building multi-platform R packages and have been overwhelmed with the choice/can't get one to work. I have . opened #28 as a result

Paxanator · 2019-05-19T23:07:32Z

.travis.yml


 language: r
 r: 
  - release
 cache: packages
-
-os:


osx doesn't work w/ docker compose

Paxanator · 2019-05-19T23:07:59Z

fRanz/DESCRIPTION

@@ -1,6 +1,6 @@
 Package: fRanz
 Type: Package
-Title: An R Kafka Client
+Title: An R Kafka Client.


There's a funny R CMD CHECK note that says this should be a full sentence

Paxanator · 2019-05-19T23:08:19Z

fRanz/R/KafkaConsumer.R

@@ -25,7 +28,8 @@ KafkaConsumer <- R6::R6Class(
        }

        , consume = function(topic, numResults=100) {
-            Filter(function(msg) !is.null(msg), KafkaConsume(private$consumerPtr, numResults))
+            results <- Filter(function(msg) !is.null(msg), KafkaConsume(private$consumerPtr, numResults))


Shoot my other PR of serializers bled into this

mjermann · 2019-05-26T14:33:51Z

.travis.yml

@@ -2,14 +2,14 @@ services:
  - docker

 before_install:
+  - mkdir dest
+  - make librdkafka LIBRDKAFKA_CONF=--prefix=dest
+  - export LD_LIBRARY_PATH=LD_LIBRARY_PATH:dest/lib


I pulled this locally and am testing. what is the purpose of the dest directory? I'm finding it empty after I run make librdkafka

i got this to populate by changing it to --prefix=../dest, will see if that makes any difference

… new system

…p measure

Paxanator · 2019-05-27T16:30:55Z

fRanz/R/RcppExports.R

 #' @description Consume a fixed number of results from whatever topic(s)
 #'              the provided consumer is subscribed to.
+=======


ugh i screwed up the merge

lesson learned. Making changes in VS Code w/o saving is bad for biz

…uced up Contributing and README. Removed stray Dockerfile

…CK's ** testing if installed package can be loaded from temporary location

…ll process

Paxanator · 2019-05-27T21:52:19Z

For posterity i think the reason it installs on mac and not linux is because

gcc  -shared -dynamiclib -Wl,-install_name,/Users/pboueri/Repos/oss/fRanz/fRanz/inst/librdkafka/lib/librdkafka++.1.dylib RdKafka.o ConfImpl.o HandleImpl.o ConsumerImpl.o ProducerImpl.o KafkaConsumerImpl.o TopicImpl.o TopicPartitionImpl.o MessageImpl.o HeadersImpl.o QueueImpl.o MetadataImpl.o -o librdkafka++.1.dylib -L../src -lrdkafka -lstdc++

of the librdkafka compilation
-install_name I think tells fRanz.so where librdfkaka++/.dylib is so when fRanz.so moves as part of the installation process , it still can reference it

Current options are in my mind:

Figure out how to compile fRanz.so with librdkafka.a as part of it by overriding SHLIB like this: https://stackoverflow.com/a/31754107
Put compiled librdkafka stuff in /inst and then figure out how to make a relative link against it in fRanz.so. thing to note here is that inst/ stuff is moved one level up when being installed
Figure out if there’s a way to override install.packages.R to move both fRanz.so and librdfkafka.a/dylib/so to the installed libs dir and link locally
Do the system level library linking thing

Paxanator commented May 19, 2019

View reviewed changes

.travis.yml

language: r

r:

- release

cache: packages

os:

Copy link

Collaborator Author

Paxanator May 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

osx doesn't work w/ docker compose

Paxanator commented May 19, 2019

View reviewed changes

Paxanator force-pushed the master branch from f86d12e to 87f5482 Compare May 20, 2019 03:13

mjermann reviewed May 26, 2019

View reviewed changes

mjermann mentioned this pull request May 26, 2019

WIP: fix(travis): configure full path for librdkafka; #33

Open

Paxanator added 6 commits May 27, 2019 11:23

fix(install): Added missing inst directory to install librdkafka on a…

86247d8

… new system

fix(build): Fixing install process to rely on system library as a tem…

ac1b58b

…p measure

ci: getting travis to work

965637d

ci: getting travis to work

8570217

ci: getting travis to work

05689fe

fix: travis build and R CMD Check

97f69a2

Paxanator force-pushed the master branch from 5666752 to 97f69a2 Compare May 27, 2019 16:25

Paxanator commented May 27, 2019

View reviewed changes

Paxanator added 4 commits May 27, 2019 11:48

fix(ci): Some stray merge conflicts made it through. Additionally spr…

fd5c1c2

…uced up Contributing and README. Removed stray Dockerfile

fix(ci): Removing fq path in Makevars to attempt to rectify R CMD CHE…

d4c2985

…CK's ** testing if installed package can be loaded from temporary location

fix(check): Fixed all NOTES/WARNINGS on MAC for R CMD CHECK

c55435a

fix(check): Trying to ship librdkafka binaries with fRanz.so in insta…

d7b40fb

…ll process

chore(install): Added devel/docker to help with installation checking

e060e76

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Working Travis Build #29

Working Travis Build #29

Paxanator commented May 19, 2019

Paxanator May 19, 2019

Paxanator May 19, 2019

Paxanator May 19, 2019

mjermann May 26, 2019

mjermann May 26, 2019

Paxanator May 27, 2019

Paxanator May 27, 2019

Paxanator commented May 27, 2019 •

edited

Working Travis Build #29

Are you sure you want to change the base?

Working Travis Build #29

Conversation

Paxanator commented May 19, 2019

Paxanator May 19, 2019

Choose a reason for hiding this comment

Paxanator May 19, 2019

Choose a reason for hiding this comment

Paxanator May 19, 2019

Choose a reason for hiding this comment

mjermann May 26, 2019

Choose a reason for hiding this comment

mjermann May 26, 2019

Choose a reason for hiding this comment

Paxanator May 27, 2019

Choose a reason for hiding this comment

Paxanator May 27, 2019

Choose a reason for hiding this comment

Paxanator commented May 27, 2019 • edited

Paxanator commented May 27, 2019 •

edited