Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[r] Extend iterated reader interface with platform_config #556

Merged

Conversation

eddelbuettel
Copy link
Contributor

This PR extend the 'iterating reader' to accept an optional argument config: a (standard R) named character vector for key=value pairs used to seed the platform_config map. This now permits to set configuration values. Here is an example of turning logging one by passing config=c("config.logging_level" = "5"):

$ ./quickCheck.R 
[2022-11-18 17:24:38.034] [Process: 2773728] [info] [1668813877360419015-Context: 2] [Query: 1] [DenseReader: 1] using cache
[2022-11-18 17:24:38.274] [Process: 2773728] [info] [1668813877360419015-Context: 2] [Query: 1] [DenseReader: 1] using cache
[2022-11-18 17:24:38.355] [Process: 2773728] [info] [1668813877360419015-Context: 2] [Query: 1] [DenseReader: 1] using cache
[2022-11-18 17:24:38.543] [Process: 2773728] [info] [1668813877360419015-Context: 2] [Query: 1] [DenseReader: 1] using cache
[2022-11-18 17:24:38.543] [Process: 2773728] [info] [1668813877360419015-Context: 2] [Query: 1] [DenseReader: 1] using cache
[2022-11-18 17:24:38.765] [Process: 2773728] [trace] [1668813878010788925-Global] malloc_trim did unmap memory
$ 

We can combine this with the existing logging support from R which, when set to debug, also shows the value being set (see second line):

$ ./quickCheck.R 
[2022-11-18 17:49:39.119] [default] [Process: 2837302] [info] [sr_setup] Setting up ~/git/tiledb-soma/test/soco/pbmc3k_processed/ms/mRNA/X/data/
[2022-11-18 17:49:39.119] [default] [Process: 2837302] [debug] [sr_setup] config map adding 'config.logging_level' = '5'
[2022-11-18 17:49:39.119] [default] [Process: 2837302] [debug] [sr_setup] creating ctx object with supplied config
[2022-11-18 17:49:39.122] [default] [Process: 2837302] [info] [soma_reader] Dimension soma_dim_0 type INT64 domain [0,2637] extent 2048
[2022-11-18 17:49:39.122] [default] [Process: 2837302] [info] [soma_reader] Dimension soma_dim_1 type INT64 domain [0,1837] extent 1838
[2022-11-18 17:49:39.137] [Process: 2837302] [info] [1668815378482964946-Context: 2] [Query: 1] [DenseReader: 1] using cache
[2022-11-18 17:49:39.186] [default] [Process: 2837302] [info] [sr_complete] Complete test is false
[2022-11-18 17:49:39.186] [default] [Process: 2837302] [info] [sr_next] Read 1213080 rows and 3 cols
[2022-11-18 17:49:39.354] [default] [Process: 2837302] [info] [sr_complete] Complete test is false
[2022-11-18 17:49:39.365] [Process: 2837302] [info] [1668815378482964946-Context: 2] [Query: 1] [DenseReader: 1] using cache
[2022-11-18 17:49:39.403] [default] [Process: 2837302] [info] [sr_next] Read 1211242 rows and 3 cols
[2022-11-18 17:49:39.436] [default] [Process: 2837302] [info] [sr_complete] Complete test is false
[2022-11-18 17:49:39.449] [Process: 2837302] [info] [1668815378482964946-Context: 2] [Query: 1] [DenseReader: 1] using cache
[2022-11-18 17:49:39.493] [default] [Process: 2837302] [info] [sr_next] Read 1213080 rows and 3 cols
[2022-11-18 17:49:39.618] [default] [Process: 2837302] [info] [sr_complete] Complete test is false
[2022-11-18 17:49:39.633] [Process: 2837302] [info] [1668815378482964946-Context: 2] [Query: 1] [DenseReader: 1] using cache
[2022-11-18 17:49:39.633] [Process: 2837302] [info] [1668815378482964946-Context: 2] [Query: 1] [DenseReader: 1] using cache
[2022-11-18 17:49:39.681] [default] [Process: 2837302] [info] [sr_next] Read 1211242 rows and 3 cols
[2022-11-18 17:49:39.835] [default] [Process: 2837302] [info] [sr_complete] Complete test is true
[2022-11-18 17:49:39.838] [Process: 2837302] [trace] [1668815379114681381-Global] malloc_trim did unmap memory
$ 

No other code change. No test added as there is currently no way to return a config so hard to check that values are set. We can address this later, as we can extending the other (simpler) SOMAReader interface.

I had accidentally pushed this same commit to main a few minutes ago, but have undo that as main should only change after reviews of pull requests.

@shortcut-integration
Copy link

This pull request has been linked to Shortcut Story #23591: Expose platform config for iterated reader.

@eddelbuettel eddelbuettel force-pushed the de/sc-23591/add_platform_config_to_iterated_reader branch from 5d466be to 112a598 Compare November 21, 2022 23:06
@eddelbuettel eddelbuettel force-pushed the de/sc-23591/add_platform_config_to_iterated_reader branch from 112a598 to 2e7af36 Compare November 22, 2022 16:53
@eddelbuettel eddelbuettel merged commit e8606cc into main Nov 22, 2022
@eddelbuettel eddelbuettel deleted the de/sc-23591/add_platform_config_to_iterated_reader branch November 22, 2022 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants