Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[R][C++] Calling bucket$ls on GCS bucket without recursive = TRUE doesn't list full contents #35993

Open
thisisnic opened this issue Jun 8, 2023 · 4 comments
Open

[R][C++] Calling bucket$ls on GCS bucket without recursive = TRUE doesn't list full contents #35993

thisisnic opened this issue Jun 8, 2023 · 4 comments
Labels
Component: C++ Component: R Type: enhancement

Comments

@thisisnic
Copy link
Member

Describe the bug, including details regarding any error messages, version, and platform.

library(arrow)
bucket <- gs_bucket("voltrondata-labs-datasets", anonymous = TRUE)
# implying only "pums" exists
bucket$ls()
#> [1] "pums"
# nope, there's loads in here!
bucket$ls(recursive = TRUE)
#>   [1] "diamonds/cut=Fair/part-0.parquet"               
#>   [2] "diamonds/cut=Good/part-0.parquet"               
#>   [3] "diamonds/cut=Ideal/part-0.parquet"              
#>   [4] "diamonds/cut=Premium/part-0.parquet"            
#>   [5] "diamonds/cut=Very Good/part-0.parquet"          
#>   [6] "nyc-taxi-tiny/year=2009/month=1/part-0.parquet" 
#>   [7] "nyc-taxi-tiny/year=2009/month=10/part-0.parquet"
#>   [8] "nyc-taxi-tiny/year=2009/month=11/part-0.parquet"
#>   [9] "nyc-taxi-tiny/year=2009/month=12/part-0.parquet"
#>  [10] "nyc-taxi-tiny/year=2009/month=2/part-0.parquet" 
#>  [11] "nyc-taxi-tiny/year=2009/month=3/part-0.parquet" 
#>  [12] "nyc-taxi-tiny/year=2009/month=4/part-0.parquet" 
#>  [13] "nyc-taxi-tiny/year=2009/month=5/part-0.parquet" 
#>  [14] "nyc-taxi-tiny/year=2009/month=6/part-0.parquet" 
#>  [15] "nyc-taxi-tiny/year=2009/month=7/part-0.parquet" 
#>  [16] "nyc-taxi-tiny/year=2009/month=8/part-0.parquet" 
#>  [17] "nyc-taxi-tiny/year=2009/month=9/part-0.parquet" 
#>  [18] "nyc-taxi-tiny/year=2010/month=1/part-0.parquet" 
#>  [19] "nyc-taxi-tiny/year=2010/month=10/part-0.parquet"
#>  [20] "nyc-taxi-tiny/year=2010/month=11/part-0.parquet"
#>  [21] "nyc-taxi-tiny/year=2010/month=12/part-0.parquet"
#>  [22] "nyc-taxi-tiny/year=2010/month=2/part-0.parquet" 
#>  [23] "nyc-taxi-tiny/year=2010/month=3/part-0.parquet" 
#>  [24] "nyc-taxi-tiny/year=2010/month=4/part-0.parquet" 
#>  [25] "nyc-taxi-tiny/year=2010/month=5/part-0.parquet" 
#>  [26] "nyc-taxi-tiny/year=2010/month=6/part-0.parquet" 
#>  [27] "nyc-taxi-tiny/year=2010/month=7/part-0.parquet" 
#>  [28] "nyc-taxi-tiny/year=2010/month=8/part-0.parquet" 
#>  [29] "nyc-taxi-tiny/year=2010/month=9/part-0.parquet" 
#>  [30] "nyc-taxi-tiny/year=2011/month=1/part-0.parquet" 
#>  [31] "nyc-taxi-tiny/year=2011/month=10/part-0.parquet"
#>  [32] "nyc-taxi-tiny/year=2011/month=11/part-0.parquet"
#>  [33] "nyc-taxi-tiny/year=2011/month=12/part-0.parquet"
#>  [34] "nyc-taxi-tiny/year=2011/month=2/part-0.parquet" 
#>  [35] "nyc-taxi-tiny/year=2011/month=3/part-0.parquet" 
#>  [36] "nyc-taxi-tiny/year=2011/month=4/part-0.parquet" 
#>  [37] "nyc-taxi-tiny/year=2011/month=5/part-0.parquet" 
#>  [38] "nyc-taxi-tiny/year=2011/month=6/part-0.parquet" 
#>  [39] "nyc-taxi-tiny/year=2011/month=7/part-0.parquet" 
#>  [40] "nyc-taxi-tiny/year=2011/month=8/part-0.parquet" 
#>  [41] "nyc-taxi-tiny/year=2011/month=9/part-0.parquet" 
#>  [42] "nyc-taxi-tiny/year=2012/month=1/part-0.parquet" 
#>  [43] "nyc-taxi-tiny/year=2012/month=10/part-0.parquet"
#>  [44] "nyc-taxi-tiny/year=2012/month=11/part-0.parquet"
#>  [45] "nyc-taxi-tiny/year=2012/month=12/part-0.parquet"
#>  [46] "nyc-taxi-tiny/year=2012/month=2/part-0.parquet" 
#>  [47] "nyc-taxi-tiny/year=2012/month=3/part-0.parquet" 
#>  [48] "nyc-taxi-tiny/year=2012/month=4/part-0.parquet" 
#>  [49] "nyc-taxi-tiny/year=2012/month=5/part-0.parquet" 
#>  [50] "nyc-taxi-tiny/year=2012/month=6/part-0.parquet" 
#>  [51] "nyc-taxi-tiny/year=2012/month=7/part-0.parquet" 
#>  [52] "nyc-taxi-tiny/year=2012/month=8/part-0.parquet" 
#>  [53] "nyc-taxi-tiny/year=2012/month=9/part-0.parquet" 
#>  [54] "nyc-taxi-tiny/year=2013/month=1/part-0.parquet" 
#>  [55] "nyc-taxi-tiny/year=2013/month=10/part-0.parquet"
#>  [56] "nyc-taxi-tiny/year=2013/month=11/part-0.parquet"
#>  [57] "nyc-taxi-tiny/year=2013/month=12/part-0.parquet"
#>  [58] "nyc-taxi-tiny/year=2013/month=2/part-0.parquet" 
#>  [59] "nyc-taxi-tiny/year=2013/month=3/part-0.parquet" 
#>  [60] "nyc-taxi-tiny/year=2013/month=4/part-0.parquet" 
#>  [61] "nyc-taxi-tiny/year=2013/month=5/part-0.parquet" 
#>  [62] "nyc-taxi-tiny/year=2013/month=6/part-0.parquet" 
#>  [63] "nyc-taxi-tiny/year=2013/month=7/part-0.parquet" 
#>  [64] "nyc-taxi-tiny/year=2013/month=8/part-0.parquet" 
#>  [65] "nyc-taxi-tiny/year=2013/month=9/part-0.parquet" 
#>  [66] "nyc-taxi-tiny/year=2014/month=1/part-0.parquet" 
#>  [67] "nyc-taxi-tiny/year=2014/month=10/part-0.parquet"
#>  [68] "nyc-taxi-tiny/year=2014/month=11/part-0.parquet"
#>  [69] "nyc-taxi-tiny/year=2014/month=12/part-0.parquet"
#>  [70] "nyc-taxi-tiny/year=2014/month=2/part-0.parquet" 
#>  [71] "nyc-taxi-tiny/year=2014/month=3/part-0.parquet" 
#>  [72] "nyc-taxi-tiny/year=2014/month=4/part-0.parquet" 
#>  [73] "nyc-taxi-tiny/year=2014/month=5/part-0.parquet" 
#>  [74] "nyc-taxi-tiny/year=2014/month=6/part-0.parquet" 
#>  [75] "nyc-taxi-tiny/year=2014/month=7/part-0.parquet" 
#>  [76] "nyc-taxi-tiny/year=2014/month=8/part-0.parquet" 
#>  [77] "nyc-taxi-tiny/year=2014/month=9/part-0.parquet" 
#>  [78] "nyc-taxi-tiny/year=2015/month=1/part-0.parquet" 
#>  [79] "nyc-taxi-tiny/year=2015/month=10/part-0.parquet"
#>  [80] "nyc-taxi-tiny/year=2015/month=11/part-0.parquet"
#>  [81] "nyc-taxi-tiny/year=2015/month=12/part-0.parquet"
#>  [82] "nyc-taxi-tiny/year=2015/month=2/part-0.parquet" 
#>  [83] "nyc-taxi-tiny/year=2015/month=3/part-0.parquet" 
#>  [84] "nyc-taxi-tiny/year=2015/month=4/part-0.parquet" 
#>  [85] "nyc-taxi-tiny/year=2015/month=5/part-0.parquet" 
#>  [86] "nyc-taxi-tiny/year=2015/month=6/part-0.parquet" 
#>  [87] "nyc-taxi-tiny/year=2015/month=7/part-0.parquet" 
#>  [88] "nyc-taxi-tiny/year=2015/month=8/part-0.parquet" 
#>  [89] "nyc-taxi-tiny/year=2015/month=9/part-0.parquet" 
#>  [90] "nyc-taxi-tiny/year=2016/month=1/part-0.parquet" 
#>  [91] "nyc-taxi-tiny/year=2016/month=10/part-0.parquet"
#>  [92] "nyc-taxi-tiny/year=2016/month=11/part-0.parquet"
#>  [93] "nyc-taxi-tiny/year=2016/month=12/part-0.parquet"
#>  [94] "nyc-taxi-tiny/year=2016/month=2/part-0.parquet" 
#>  [95] "nyc-taxi-tiny/year=2016/month=3/part-0.parquet" 
#>  [96] "nyc-taxi-tiny/year=2016/month=4/part-0.parquet" 
#>  [97] "nyc-taxi-tiny/year=2016/month=5/part-0.parquet" 
#>  [98] "nyc-taxi-tiny/year=2016/month=6/part-0.parquet" 
#>  [99] "nyc-taxi-tiny/year=2016/month=7/part-0.parquet" 
#> [100] "nyc-taxi-tiny/year=2016/month=8/part-0.parquet" 
#> [101] "nyc-taxi-tiny/year=2016/month=9/part-0.parquet" 
#> [102] "nyc-taxi-tiny/year=2017/month=1/part-0.parquet" 
#> [103] "nyc-taxi-tiny/year=2017/month=10/part-0.parquet"
#> [104] "nyc-taxi-tiny/year=2017/month=11/part-0.parquet"
#> [105] "nyc-taxi-tiny/year=2017/month=12/part-0.parquet"
#> [106] "nyc-taxi-tiny/year=2017/month=2/part-0.parquet" 
#> [107] "nyc-taxi-tiny/year=2017/month=3/part-0.parquet" 
#> [108] "nyc-taxi-tiny/year=2017/month=4/part-0.parquet" 
#> [109] "nyc-taxi-tiny/year=2017/month=5/part-0.parquet" 
#> [110] "nyc-taxi-tiny/year=2017/month=6/part-0.parquet" 
#> [111] "nyc-taxi-tiny/year=2017/month=7/part-0.parquet" 
#> [112] "nyc-taxi-tiny/year=2017/month=8/part-0.parquet" 
#> [113] "nyc-taxi-tiny/year=2017/month=9/part-0.parquet" 
#> [114] "nyc-taxi-tiny/year=2018/month=1/part-0.parquet" 
#> [115] "nyc-taxi-tiny/year=2018/month=10/part-0.parquet"
#> [116] "nyc-taxi-tiny/year=2018/month=11/part-0.parquet"
#> [117] "nyc-taxi-tiny/year=2018/month=12/part-0.parquet"
#> [118] "nyc-taxi-tiny/year=2018/month=2/part-0.parquet" 
#> [119] "nyc-taxi-tiny/year=2018/month=3/part-0.parquet" 
#> [120] "nyc-taxi-tiny/year=2018/month=4/part-0.parquet" 
#> [121] "nyc-taxi-tiny/year=2018/month=5/part-0.parquet" 
#> [122] "nyc-taxi-tiny/year=2018/month=6/part-0.parquet" 
#> [123] "nyc-taxi-tiny/year=2018/month=7/part-0.parquet" 
#> [124] "nyc-taxi-tiny/year=2018/month=8/part-0.parquet" 
#> [125] "nyc-taxi-tiny/year=2018/month=9/part-0.parquet" 
#> [126] "nyc-taxi-tiny/year=2019/month=1/part-0.parquet" 
#> [127] "nyc-taxi-tiny/year=2019/month=10/part-0.parquet"
#> [128] "nyc-taxi-tiny/year=2019/month=11/part-0.parquet"
#> [129] "nyc-taxi-tiny/year=2019/month=12/part-0.parquet"
#> [130] "nyc-taxi-tiny/year=2019/month=2/part-0.parquet" 
#> [131] "nyc-taxi-tiny/year=2019/month=3/part-0.parquet" 
#> [132] "nyc-taxi-tiny/year=2019/month=4/part-0.parquet" 
#> [133] "nyc-taxi-tiny/year=2019/month=5/part-0.parquet" 
#> [134] "nyc-taxi-tiny/year=2019/month=6/part-0.parquet" 
#> [135] "nyc-taxi-tiny/year=2019/month=7/part-0.parquet" 
#> [136] "nyc-taxi-tiny/year=2019/month=8/part-0.parquet" 
#> [137] "nyc-taxi-tiny/year=2019/month=9/part-0.parquet" 
#> [138] "nyc-taxi-tiny/year=2020/month=1/part-0.parquet" 
#> [139] "nyc-taxi-tiny/year=2020/month=10/part-0.parquet"
#> [140] "nyc-taxi-tiny/year=2020/month=11/part-0.parquet"
#> [141] "nyc-taxi-tiny/year=2020/month=12/part-0.parquet"
#> [142] "nyc-taxi-tiny/year=2020/month=2/part-0.parquet" 
#> [143] "nyc-taxi-tiny/year=2020/month=3/part-0.parquet" 
#> [144] "nyc-taxi-tiny/year=2020/month=4/part-0.parquet" 
#> [145] "nyc-taxi-tiny/year=2020/month=5/part-0.parquet" 
#> [146] "nyc-taxi-tiny/year=2020/month=6/part-0.parquet" 
#> [147] "nyc-taxi-tiny/year=2020/month=7/part-0.parquet" 
#> [148] "nyc-taxi-tiny/year=2020/month=8/part-0.parquet" 
#> [149] "nyc-taxi-tiny/year=2020/month=9/part-0.parquet" 
#> [150] "nyc-taxi-tiny/year=2021/month=1/part-0.parquet" 
#> [151] "nyc-taxi-tiny/year=2021/month=10/part-0.parquet"
#> [152] "nyc-taxi-tiny/year=2021/month=11/part-0.parquet"
#> [153] "nyc-taxi-tiny/year=2021/month=12/part-0.parquet"
#> [154] "nyc-taxi-tiny/year=2021/month=2/part-0.parquet" 
#> [155] "nyc-taxi-tiny/year=2021/month=3/part-0.parquet" 
#> [156] "nyc-taxi-tiny/year=2021/month=4/part-0.parquet" 
#> [157] "nyc-taxi-tiny/year=2021/month=5/part-0.parquet" 
#> [158] "nyc-taxi-tiny/year=2021/month=6/part-0.parquet" 
#> [159] "nyc-taxi-tiny/year=2021/month=7/part-0.parquet" 
#> [160] "nyc-taxi-tiny/year=2021/month=8/part-0.parquet" 
#> [161] "nyc-taxi-tiny/year=2021/month=9/part-0.parquet" 
#> [162] "nyc-taxi-tiny/year=2022/month=1/part-0.parquet" 
#> [163] "nyc-taxi-tiny/year=2022/month=2/part-0.parquet" 
#> [164] "nyc-taxi/year=2009/month=1/part-0.parquet"      
#> [165] "nyc-taxi/year=2009/month=10/part-0.parquet"     
#> [166] "nyc-taxi/year=2009/month=11/part-0.parquet"     
#> [167] "nyc-taxi/year=2009/month=12/part-0.parquet"     
#> [168] "nyc-taxi/year=2009/month=2/part-0.parquet"      
#> [169] "nyc-taxi/year=2009/month=3/part-0.parquet"      
#> [170] "nyc-taxi/year=2009/month=4/part-0.parquet"      
#> [171] "nyc-taxi/year=2009/month=5/part-0.parquet"      
#> [172] "nyc-taxi/year=2009/month=6/part-0.parquet"      
#> [173] "nyc-taxi/year=2009/month=7/part-0.parquet"      
#> [174] "nyc-taxi/year=2009/month=8/part-0.parquet"      
#> [175] "nyc-taxi/year=2009/month=9/part-0.parquet"      
#> [176] "nyc-taxi/year=2010/month=1/part-0.parquet"      
#> [177] "nyc-taxi/year=2010/month=10/part-0.parquet"     
#> [178] "nyc-taxi/year=2010/month=11/part-0.parquet"     
#> [179] "nyc-taxi/year=2010/month=12/part-0.parquet"     
#> [180] "nyc-taxi/year=2010/month=2/part-0.parquet"      
#> [181] "nyc-taxi/year=2010/month=3/part-0.parquet"      
#> [182] "nyc-taxi/year=2010/month=4/part-0.parquet"      
#> [183] "nyc-taxi/year=2010/month=5/part-0.parquet"      
#> [184] "nyc-taxi/year=2010/month=6/part-0.parquet"      
#> [185] "nyc-taxi/year=2010/month=7/part-0.parquet"      
#> [186] "nyc-taxi/year=2010/month=8/part-0.parquet"      
#> [187] "nyc-taxi/year=2010/month=9/part-0.parquet"      
#> [188] "nyc-taxi/year=2011/month=1/part-0.parquet"      
#> [189] "nyc-taxi/year=2011/month=10/part-0.parquet"     
#> [190] "nyc-taxi/year=2011/month=11/part-0.parquet"     
#> [191] "nyc-taxi/year=2011/month=12/part-0.parquet"     
#> [192] "nyc-taxi/year=2011/month=2/part-0.parquet"      
#> [193] "nyc-taxi/year=2011/month=3/part-0.parquet"      
#> [194] "nyc-taxi/year=2011/month=4/part-0.parquet"      
#> [195] "nyc-taxi/year=2011/month=5/part-0.parquet"      
#> [196] "nyc-taxi/year=2011/month=6/part-0.parquet"      
#> [197] "nyc-taxi/year=2011/month=7/part-0.parquet"      
#> [198] "nyc-taxi/year=2011/month=8/part-0.parquet"      
#> [199] "nyc-taxi/year=2011/month=9/part-0.parquet"      
#> [200] "nyc-taxi/year=2012/month=1/part-0.parquet"      
#> [201] "nyc-taxi/year=2012/month=10/part-0.parquet"     
#> [202] "nyc-taxi/year=2012/month=11/part-0.parquet"     
#> [203] "nyc-taxi/year=2012/month=12/part-0.parquet"     
#> [204] "nyc-taxi/year=2012/month=2/part-0.parquet"      
#> [205] "nyc-taxi/year=2012/month=3/part-0.parquet"      
#> [206] "nyc-taxi/year=2012/month=4/part-0.parquet"      
#> [207] "nyc-taxi/year=2012/month=5/part-0.parquet"      
#> [208] "nyc-taxi/year=2012/month=6/part-0.parquet"      
#> [209] "nyc-taxi/year=2012/month=7/part-0.parquet"      
#> [210] "nyc-taxi/year=2012/month=8/part-0.parquet"      
#> [211] "nyc-taxi/year=2012/month=9/part-0.parquet"      
#> [212] "nyc-taxi/year=2013/month=1/part-0.parquet"      
#> [213] "nyc-taxi/year=2013/month=10/part-0.parquet"     
#> [214] "nyc-taxi/year=2013/month=11/part-0.parquet"     
#> [215] "nyc-taxi/year=2013/month=12/part-0.parquet"     
#> [216] "nyc-taxi/year=2013/month=2/part-0.parquet"      
#> [217] "nyc-taxi/year=2013/month=3/part-0.parquet"      
#> [218] "nyc-taxi/year=2013/month=4/part-0.parquet"      
#> [219] "nyc-taxi/year=2013/month=5/part-0.parquet"      
#> [220] "nyc-taxi/year=2013/month=6/part-0.parquet"      
#> [221] "nyc-taxi/year=2013/month=7/part-0.parquet"      
#> [222] "nyc-taxi/year=2013/month=8/part-0.parquet"      
#> [223] "nyc-taxi/year=2013/month=9/part-0.parquet"      
#> [224] "nyc-taxi/year=2014/month=1/part-0.parquet"      
#> [225] "nyc-taxi/year=2014/month=10/part-0.parquet"     
#> [226] "nyc-taxi/year=2014/month=11/part-0.parquet"     
#> [227] "nyc-taxi/year=2014/month=12/part-0.parquet"     
#> [228] "nyc-taxi/year=2014/month=2/part-0.parquet"      
#> [229] "nyc-taxi/year=2014/month=3/part-0.parquet"      
#> [230] "nyc-taxi/year=2014/month=4/part-0.parquet"      
#> [231] "nyc-taxi/year=2014/month=5/part-0.parquet"      
#> [232] "nyc-taxi/year=2014/month=6/part-0.parquet"      
#> [233] "nyc-taxi/year=2014/month=7/part-0.parquet"      
#> [234] "nyc-taxi/year=2014/month=8/part-0.parquet"      
#> [235] "nyc-taxi/year=2014/month=9/part-0.parquet"      
#> [236] "nyc-taxi/year=2015/month=1/part-0.parquet"      
#> [237] "nyc-taxi/year=2015/month=10/part-0.parquet"     
#> [238] "nyc-taxi/year=2015/month=11/part-0.parquet"     
#> [239] "nyc-taxi/year=2015/month=12/part-0.parquet"     
#> [240] "nyc-taxi/year=2015/month=2/part-0.parquet"      
#> [241] "nyc-taxi/year=2015/month=3/part-0.parquet"      
#> [242] "nyc-taxi/year=2015/month=4/part-0.parquet"      
#> [243] "nyc-taxi/year=2015/month=5/part-0.parquet"      
#> [244] "nyc-taxi/year=2015/month=6/part-0.parquet"      
#> [245] "nyc-taxi/year=2015/month=7/part-0.parquet"      
#> [246] "nyc-taxi/year=2015/month=8/part-0.parquet"      
#> [247] "nyc-taxi/year=2015/month=9/part-0.parquet"      
#> [248] "nyc-taxi/year=2016/month=1/part-0.parquet"      
#> [249] "nyc-taxi/year=2016/month=10/part-0.parquet"     
#> [250] "nyc-taxi/year=2016/month=11/part-0.parquet"     
#> [251] "nyc-taxi/year=2016/month=12/part-0.parquet"     
#> [252] "nyc-taxi/year=2016/month=2/part-0.parquet"      
#> [253] "nyc-taxi/year=2016/month=3/part-0.parquet"      
#> [254] "nyc-taxi/year=2016/month=4/part-0.parquet"      
#> [255] "nyc-taxi/year=2016/month=5/part-0.parquet"      
#> [256] "nyc-taxi/year=2016/month=6/part-0.parquet"      
#> [257] "nyc-taxi/year=2016/month=7/part-0.parquet"      
#> [258] "nyc-taxi/year=2016/month=8/part-0.parquet"      
#> [259] "nyc-taxi/year=2016/month=9/part-0.parquet"      
#> [260] "nyc-taxi/year=2017/month=1/part-0.parquet"      
#> [261] "nyc-taxi/year=2017/month=10/part-0.parquet"     
#> [262] "nyc-taxi/year=2017/month=11/part-0.parquet"     
#> [263] "nyc-taxi/year=2017/month=12/part-0.parquet"     
#> [264] "nyc-taxi/year=2017/month=2/part-0.parquet"      
#> [265] "nyc-taxi/year=2017/month=3/part-0.parquet"      
#> [266] "nyc-taxi/year=2017/month=4/part-0.parquet"      
#> [267] "nyc-taxi/year=2017/month=5/part-0.parquet"      
#> [268] "nyc-taxi/year=2017/month=6/part-0.parquet"      
#> [269] "nyc-taxi/year=2017/month=7/part-0.parquet"      
#> [270] "nyc-taxi/year=2017/month=8/part-0.parquet"      
#> [271] "nyc-taxi/year=2017/month=9/part-0.parquet"      
#> [272] "nyc-taxi/year=2018/month=1/part-0.parquet"      
#> [273] "nyc-taxi/year=2018/month=10/part-0.parquet"     
#> [274] "nyc-taxi/year=2018/month=11/part-0.parquet"     
#> [275] "nyc-taxi/year=2018/month=12/part-0.parquet"     
#> [276] "nyc-taxi/year=2018/month=2/part-0.parquet"      
#> [277] "nyc-taxi/year=2018/month=3/part-0.parquet"      
#> [278] "nyc-taxi/year=2018/month=4/part-0.parquet"      
#> [279] "nyc-taxi/year=2018/month=5/part-0.parquet"      
#> [280] "nyc-taxi/year=2018/month=6/part-0.parquet"      
#> [281] "nyc-taxi/year=2018/month=7/part-0.parquet"      
#> [282] "nyc-taxi/year=2018/month=8/part-0.parquet"      
#> [283] "nyc-taxi/year=2018/month=9/part-0.parquet"      
#> [284] "nyc-taxi/year=2019/month=1/part-0.parquet"      
#> [285] "nyc-taxi/year=2019/month=10/part-0.parquet"     
#> [286] "nyc-taxi/year=2019/month=11/part-0.parquet"     
#> [287] "nyc-taxi/year=2019/month=12/part-0.parquet"     
#> [288] "nyc-taxi/year=2019/month=2/part-0.parquet"      
#> [289] "nyc-taxi/year=2019/month=3/part-0.parquet"      
#> [290] "nyc-taxi/year=2019/month=4/part-0.parquet"      
#> [291] "nyc-taxi/year=2019/month=5/part-0.parquet"      
#> [292] "nyc-taxi/year=2019/month=6/part-0.parquet"      
#> [293] "nyc-taxi/year=2019/month=7/part-0.parquet"      
#> [294] "nyc-taxi/year=2019/month=8/part-0.parquet"      
#> [295] "nyc-taxi/year=2019/month=9/part-0.parquet"      
#> [296] "nyc-taxi/year=2020/month=1/part-0.parquet"      
#> [297] "nyc-taxi/year=2020/month=10/part-0.parquet"     
#> [298] "nyc-taxi/year=2020/month=11/part-0.parquet"     
#> [299] "nyc-taxi/year=2020/month=12/part-0.parquet"     
#> [300] "nyc-taxi/year=2020/month=2/part-0.parquet"      
#> [301] "nyc-taxi/year=2020/month=3/part-0.parquet"      
#> [302] "nyc-taxi/year=2020/month=4/part-0.parquet"      
#> [303] "nyc-taxi/year=2020/month=5/part-0.parquet"      
#> [304] "nyc-taxi/year=2020/month=6/part-0.parquet"      
#> [305] "nyc-taxi/year=2020/month=7/part-0.parquet"      
#> [306] "nyc-taxi/year=2020/month=8/part-0.parquet"      
#> [307] "nyc-taxi/year=2020/month=9/part-0.parquet"      
#> [308] "nyc-taxi/year=2021/month=1/part-0.parquet"      
#> [309] "nyc-taxi/year=2021/month=10/part-0.parquet"     
#> [310] "nyc-taxi/year=2021/month=11/part-0.parquet"     
#> [311] "nyc-taxi/year=2021/month=12/part-0.parquet"     
#> [312] "nyc-taxi/year=2021/month=2/part-0.parquet"      
#> [313] "nyc-taxi/year=2021/month=3/part-0.parquet"      
#> [314] "nyc-taxi/year=2021/month=4/part-0.parquet"      
#> [315] "nyc-taxi/year=2021/month=5/part-0.parquet"      
#> [316] "nyc-taxi/year=2021/month=6/part-0.parquet"      
#> [317] "nyc-taxi/year=2021/month=7/part-0.parquet"      
#> [318] "nyc-taxi/year=2021/month=8/part-0.parquet"      
#> [319] "nyc-taxi/year=2021/month=9/part-0.parquet"      
#> [320] "nyc-taxi/year=2022/month=1/part-0.parquet"      
#> [321] "nyc-taxi/year=2022/month=2/part-0.parquet"      
#> [322] "pums"                                           
#> [323] "pums/2016"                                      
#> [324] "pums/2016/5year"                                
#> [325] "pums/2016/5year/housing"                        
#> [326] "pums/2016/5year/housing/part-0.parquet"         
#> [327] "pums/2016/5year/people"                         
#> [328] "pums/2016/5year/people/part-0.parquet"          
#> [329] "pums/person_pums.duckdb"

Maybe something to do with the fact that the pums directory contains files as well as directories?

Component(s)

R
@thisisnic
Copy link
Member Author

I think this may be intentional behaviour based on how FileSelector works, but it would be useful to have a way of listing all directories in a bucket. Maybe there's another bit of the FileSystem API we can use here instead?

CC @westonpace

@amoeba
Copy link
Member

amoeba commented Jun 9, 2023

This looks related to or the same as #34525 and I think the behavior may be impacted by this draft PR: #35440.

@westonpace
Copy link
Member

Hmm, based on my understanding of FileSelector I would expect it to list directories. In the S3 filesystem we have to jump through some hoops to make this happen because S3 doesn't really have directories. I'm guessing we probably need something similar for GCS. I think this is a legitimate feature request.

@thisisnic thisisnic changed the title [R] Calling bucket$ls on GCS bucket without recursive = TRUE doesn't list full contents [R][C++] Calling bucket$ls on GCS bucket without recursive = TRUE doesn't list full contents Jun 13, 2023
@thisisnic
Copy link
Member Author

Thanks @westonpace, I'll change the ticket type then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Component: C++ Component: R Type: enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@amoeba @westonpace @thisisnic