Data validation list #347

JanMarvin · 2022-09-29T16:10:12Z

as discussed in ycphs/openxlsx#386 I've prepared a pull request to test if data valiadation lists work with Excel 2007. This is included in this pull request of openxlsx2. Please see the code below. This works in my MS365.

To check:

library(openxlsx2)

wb <- wb_workbook()$
  add_worksheet("Tab_1", zoom = 80, gridLines = FALSE)$
  add_data(x = rbind(2016:2019), dims = "C1:F1", colNames = FALSE)$
  add_data_validation(rows = 1, col = 1, type = "list", value = '"2016,2017,2018,2019"')$
  add_formula(startRow = 1, startCol = 2, array = TRUE, 
              x = '= HYPERLINK(CELL("address", INDEX(C1:F1, MATCH(A1, C1:F1, 0))), "Go to the selected column")'
  )

wb$open()

If you could have a look. You can build with ref="data_validation_list". Though we are currently in the process to release the package to CRAN and this pull request was a little late and needs some double checking.

Todo

fix tests
what's the actual difference?
add check what happens if old and new data validation list appear in the same file
new tests

olivier7121 · 2022-09-29T17:24:09Z

Hi @JanMarvin.

This code works:

install.packages(c("Rcpp", "remotes"), dependencies = TRUE, lib = "G:/Programmation/R/R-4.2.1/library")
remotes::install_github("JanMarvin/openxlsx2", lib = "G:/Programmation/R/R-4.2.1/library", ref = "data_validation_list")

OutputFolder <- file.path(".", "Output")
if(!dir.exists(OutputFolder))	dir.create(OutputFolder)

OutputFile <- file.path(OutputFolder, "Reprex_Openxlsx2_dataValidation_list.xlsx")

wb <- wb_workbook()$
	add_worksheet("Tab_1", zoom = 80, gridLines = FALSE)$
	add_data(x = rbind(2016:2019), dims = "C1:F1", colNames = FALSE)$
	add_data(x = 2017, startCol = 1, startRow = 1, colNames = FALSE)$
	add_data_validation(rows = 1, col = 1, type = "list", value = '"2016,2017,2018,2019"')$
	add_formula(startRow = 1, startCol = 2, array = TRUE, x = '= HYPERLINK(CELL("address", INDEX(C1:F1, MATCH(A1, C1:F1, 0))), "Go to the selected column")')

wb$save(path = OutputFile, overwrite = TRUE)

But the hyperlink doesn't work because it doesn't know to which tab from which worksheet it should point.
If I add in the formula, within Excel 2007, "#Tab_1!" & CELL(... then the hyperlink works (as far as I understand, # is a shortcut for the name of the current/active worksheet).

But if I add this string to the formula directly in the R code, like this:

install.packages(c("Rcpp", "remotes"), dependencies = TRUE, lib = "G:/Programmation/R/R-4.2.1/library")
remotes::install_github("JanMarvin/openxlsx2", lib = "G:/Programmation/R/R-4.2.1/library", ref = "data_validation_list")

OutputFolder <- file.path(".", "Output")
if(!dir.exists(OutputFolder))	dir.create(OutputFolder)

OutputFile <- file.path(OutputFolder, "Reprex_Openxlsx2_dataValidation_list.xlsx")

wb <- wb_workbook()$
	add_worksheet("Tab_1", zoom = 80, gridLines = FALSE)$
	add_data(x = rbind(2016:2019), dims = "C1:F1", colNames = FALSE)$
	add_data(x = 2017, startCol = 1, startRow = 1, colNames = FALSE)$
	add_data_validation(rows = 1, col = 1, type = "list", value = '"2016,2017,2018,2019"')$
	add_formula(startRow = 1, startCol = 2, array = TRUE, x = '= HYPERLINK("#Tab_1!" & CELL("address", INDEX(C1:F1, MATCH(A1, C1:F1, 0))), "Go to the selected column")')

wb$save(path = OutputFile, overwrite = TRUE)

then I cannot open the Excel file anymore, it is corrupted.

JanMarvin · 2022-09-29T17:30:25Z

Sigh, it worked for me 😞

If you already know the file name, you could use the formula from your original post. I just wanted to be more flexible and use whatever sheet is available. I'll have a look at it tomorrow.

olivier7121 · 2022-09-29T17:35:01Z

Sorry...

I just tried:

install.packages(c("Rcpp", "remotes"), dependencies = TRUE, lib = "G:/Programmation/R/R-4.2.1/library")
remotes::install_github("JanMarvin/openxlsx2", lib = "G:/Programmation/R/R-4.2.1/library", ref = "data_validation_list")

OutputFolder <- file.path(".", "Output")
if(!dir.exists(OutputFolder))	dir.create(OutputFolder)

OutputFile <- file.path(OutputFolder, "Reprex_Openxlsx2_dataValidation_list.xlsx")

wb <- wb_workbook()$
	add_worksheet("Tab_1", zoom = 80, gridLines = FALSE)$
	add_data(x = rbind(2016:2019), dims = "C1:F1", colNames = FALSE)$
	add_data(x = 2017, startCol = 1, startRow = 1, colNames = FALSE)$
	add_data_validation(rows = 1, col = 1, type = "list", value = '"2016,2017,2018,2019"')$
	#add_formula(startRow = 1, startCol = 2, array = TRUE, x = '= HYPERLINK("#Tab_1!" & CELL("address", INDEX(C1:F1, MATCH(A1, C1:F1, 0))), "Go to the selected column")')
	add_formula(startRow = 1, startCol = 2, array = TRUE, x = '= HYPERLINK("[Reprex_Openxlsx2_dataValidation_list.xlsx]Tab_1!" & CELL("address", INDEX(C1:F1, MATCH(A1, C1:F1, 0))), "Go to the selected column")')

wb$save(path = OutputFile, overwrite = TRUE)

but I got the same result as before (corrupted).

JanMarvin · 2022-09-29T17:37:01Z

Even if the formula does not work for whatever reason, the data validation list does?

olivier7121 · 2022-09-29T17:40:24Z

Yes, it does.
Well, the formula doesn't work simply because it doesn't know to which tab from which worksheet it should point.

JanMarvin · 2022-09-29T17:41:32Z

Alright, we'll figure it out :) Thanks for testing

olivier7121 · 2022-09-29T17:43:40Z

You're welcome, thanks for developing and improving!

JanMarvin · 2022-09-29T18:08:11Z

Guess that's it (tested in libreoffice). Formula expects escaped xml characters &. A bit inkonvenient I'll open another issue for this

wb <- wb_workbook()$
  add_worksheet("Tab_1", zoom = 80, gridLines = FALSE)$
  add_data(x = rbind(2016:2019), dims = "C1:F1", colNames = FALSE)$
  # add_data(x = 2017, startCol = 1, startRow = 1, colNames = FALSE)$
  add_data_validation(rows = 1, col = 1, type = "list", value = '"2016,2017,2018,2019"')$
  add_formula(startRow = 1, startCol = 2, x = '= HYPERLINK("#Tab_1!" &amp; CELL("address", INDEX(C1:F1, MATCH(A1, C1:F1, 0))), "Go to the selected column")')

wb$open()

olivier7121 · 2022-09-29T18:49:11Z

Yes, well done! It also works on Excel 2007 now.
Would be indeed nice with a more convenient (well, the regular one) symbol ;)

JanMarvin · 2022-09-29T19:17:12Z

Hehe, will fix this soon

JanMarvin · 2022-10-02T11:39:46Z

The only remaining question is: what is the actual difference between old and new data validations list. I assume there must be a reason someone came up with the ext part for x14:dataValidations. Either someone working on Excel simply forgot that it was already possible and added it this way (similar like creating hyperlink colors as blue, even though they specify a certain theme color explicitly for hyperlinks) or more likely, there is a certain feature of data validation lists that was added in Excel 2010 that we do not support and/or do not know.

Though I guess this should be ready to merge.

olivier7121 · 2022-10-07T05:24:14Z

Hi @JanMarvin.
Many thanks for your work!
I have no idea why there is indeed this, a priori, stupid difference between the 2 versions of validation lists...
Talking about hyperlinks and formatting, do you know why the hyperlink in the reprex doesn't appear blue and underlined? I will be forced to create an hyperlink style for that...

add_formula(startRow = 1, startCol = 2, x = '= HYPERLINK("#Tab_1!" &amp; CELL("address", INDEX(C1:F1, MATCH(A1, C1:F1, 0))), "Go to the selected column")')

JanMarvin · 2022-10-07T07:01:51Z

@olivier7121 , thanks for your kind words and that's because I sometimes have to sleep and a job to attend :)

But to answer the question, the color does nothing aside from being blue. It does not impact the XLSX file in any other way. There already is a pull request to fix this #353 , but this needs to be updated again 🤷

While openxlsx2 is already well tested in many areas, I can't test everything and I have my own priorities and needs this package should fullfil. I almost exclusively need to load and read from workbooks. Everything else is just bonus to me. Therefore, I never saw the issue, because the files I was working with seldomly use hyperlinks and if they do, they already were colored.

That's why raising issues is important. Just from the issues raised since the first CRAN release a couple of things have popped up. Like this here, or #356, #360, and #362. But they all require chunks of my time to research and fix. But we'll get there.

JanMarvin · 2022-10-07T08:00:09Z

Until this is fixed, you could use something like this:
wb_add_font("B1", name = "Arial", color= wb_colour("blue"))

olivier7121 · 2022-10-07T08:31:17Z

But to answer the question, the color does nothing aside from being blue. It does not impact the XLSX file in any other way. There already is a pull request to fix this #353 , but this needs to be updated again 🤷

@JanMarvin I didn't see you opened a new issue #344 for that, and pulled a new request #353, many thanks once again!
I noticed a difference between 'simple' hyperlinks (i.e. without 'complicated' formulas, e.g. that point to hard-coded cells' addresses) that were colored and more 'sophisticated' hyperlinks (i.e. defined thanks to more elaborated formulas) that were not colored.
Yes, it's really minor, I know. It can just create communication issues with third parties that are not familiar with the spreadsheet as I am (as in my project the spreadsheet will be shared) and don't know that hyperlinks in some predetermined cells are expected.
I noticed this issue already a while ago but didn't report it here (that's bad, I know!) because it is minor.

While openxlsx2 is already well tested in many areas, I can't test everything and I have my own priorities and needs this package should fullfil.

Of course, that's fully understandable.

Until this is fixed, you could use something like this:

Yes, thank you, I already picked up the exact color of Excel hyperlinks :)

But they all require chunks of my time to research and fix. But we'll get there.

I would be happy to help with tests etc.

JanMarvin · 2022-10-08T13:18:28Z

@olivier7121 , if you want to have a look I've opened a pull request #365 this should fix a couple of issues with data styling including hyperlink styling.

I've checked that a few of the basics still work, but this could use some additional testing.

previously data_validation was an x14 extension which broke backward comparabilty with Excel 2007. Apparently this is not required.

compare old and new data validations list

JanMarvin · 2022-10-15T10:28:12Z

@olivier7121 merged it with main. Thanks again for bringing it to my attention.

olivier7121 · 2022-10-23T17:50:41Z

Hi @JanMarvin. Apologies for the late reaction, was ill and then quite busy with work.

@olivier7121 , if you want to have a look I've opened a pull request #365 this should fix a couple of issues with data styling including hyperlink styling.

I tried this code:

install.packages(c("Rcpp", "remotes"), dependencies = TRUE, lib = "G:/Programmation/R/R-4.2.1/library")
remotes::install_github("JanMarvin/openxlsx2", lib = "G:/Programmation/R/R-4.2.1/library", ref = "data_validation_list")
library(openxlsx2, lib.loc = "G:/Programmation/R/R-4.2.1/library")

OutputFolder <- file.path(".", "Output")
if(!dir.exists(OutputFolder))	dir.create(OutputFolder)

OutputFile <- file.path(OutputFolder, "Reprex_Openxlsx2_dataValidation_list.xlsx")

wb <- wb_workbook()$
	add_worksheet("Tab_1", zoom = 80, gridLines = FALSE)$
	add_data(x = rbind(2016:2019), dims = "C1:F1", colNames = FALSE)$
	add_data(x = 2017, startCol = 1, startRow = 1, colNames = FALSE)$
	add_data_validation(rows = 1, col = 1, type = "list", value = '"2016,2017,2018,2019"')$
	add_formula(startRow = 1, startCol = 2, x = '= HYPERLINK("#Tab_1!" &amp; CELL("address", INDEX(C1:F1, MATCH(A1, C1:F1, 0))), "Go to the selected column")')

wb$save(path = OutputFile, overwrite = TRUE)

This worked both on Excel 2007 and Excel 2013 but without any blue-coloured hyperlinks in both cases.

I first tried with the penultimate row like this (& instead of &):

add_formula(startRow = 1, startCol = 2, x = '= HYPERLINK("#Tab_1!" & CELL("address", INDEX(C1:F1, MATCH(A1, C1:F1, 0))), "Go to the selected column")')

But this didn't work at all on both versions of Excel I have on my computer.

Is this what you wanted me to test? Or did I misunderstand?

JanMarvin · 2022-10-23T18:02:58Z

Hi @olivier7121 , all required pull requests should be merged with main already. If you want you can give that branch a spin. I plan to push 0.3.1 to CRAN this month, it will be picked from main.

olivier7121 · 2022-10-23T18:38:18Z

OK, I think what I tried was not correct then. I am not used to trying new versions on Github.
What should I write exactly to try the new code?

remotes::install_github("JanMarvin/openxlsx2", lib = "G:/Programmation/R/R-4.2.1/library", ref = "add_data_and_add_style")

?
Or

remotes::install_github("JanMarvin/openxlsx2", lib = "G:/Programmation/R/R-4.2.1/library", ref = c("add_data_and_add_style", "data_validation_list"))

?
I really have no idea.

JanMarvin · 2022-10-23T18:43:59Z

Hi, you can simply build our current main branch:

# this should be identical to ref  = "main"
remotes::install_github("JanMarvin/openxlsx2", lib = "G:/Programmation/R/R-4.2.1/library")

Or you could run the lines below to install from r-universe. No need to build on your end, similar to installation from CRAN:

# Enable repository from janmarvin
options(repos = c(
  janmarvin = 'https://janmarvin.r-universe.dev',
  CRAN = 'https://cloud.r-project.org'))
# Download and install openxlsx2 in R
install.packages('openxlsx2')

olivier7121 · 2022-10-23T18:47:50Z

OK, thanks.

I arrive at the same conclusion:

This worked both on Excel 2007 and Excel 2013 but without any blue-coloured hyperlinks in both cases.

JanMarvin · 2022-10-23T18:53:45Z

You have to use =HYPERLINK( not = HYPERLINK( this should work

olivier7121 · 2022-10-23T18:55:51Z

I obtain the same outcome.

JanMarvin · 2022-10-23T19:00:41Z

Maybe blue is not supported on Excel 2007 dunno ;-)

It works on MS365 which is good enough for me. Sorry 🤷

olivier7121 · 2022-10-23T19:03:26Z

No no, I tried with Excel 2013 too ;)
Really strange...

JanMarvin · 2022-10-23T19:09:31Z

I run this code, minor adjustments from your code snippet (no whitespace in HYPERLINK and no more &). Could you give this a try? Maybe something got mixed up when saving and opening the file?

library(openxlsx2)

wb <- wb_workbook()$
  add_worksheet("Tab_1", zoom = 80, gridLines = FALSE)$
  add_data(x = rbind(2016:2019), dims = "C1:F1", colNames = FALSE)$
  add_data(x = 2017, startCol = 1, startRow = 1, colNames = FALSE)$
  add_data_validation(rows = 1, col = 1, type = "list", value = '"2016,2017,2018,2019"')$
  add_formula(startRow = 1, startCol = 2, x = '=HYPERLINK("#Tab_1!" & CELL("address", INDEX(C1:F1, MATCH(A1, C1:F1, 0))), "Go to the selected column")')

wb$open()

olivier7121 · 2022-10-23T19:18:31Z

I tried with Excel 2007, 2013 and 365: with & instead of & the file is corrupted in these 3 versions of Excel.
I now tried with & and Excel 365: the formula is modified in &@ and there is no blue-coloured hyperlink.

Maybe something got mixed up when saving and opening the file?
Perhaps.

olivier7121 · 2022-10-23T19:24:26Z

the formula is modified in &@

I just meant: an @ is added to the character & in the formula (amp; no longer appears in the formula once in Excel).

JanMarvin · 2022-10-23T19:24:34Z

Okay, sounds like you're still running the old openxlsx2 version. Please remove openxlsx2 entirely (enter the folder with the R libs. find openxlsx2, right click and delete the folder. after that run one of the solutions from above.

olivier7121 · 2022-10-23T19:31:09Z

OK, I thought that the previous version was replaced but apparently not.
You were right: now the hyperlinks are blue, even with Excel 2007 ;)
But the (absence of) space between = and HYPERLINK seems indeed to be important as with a space the colour of the hyperlink disappears =D

JanMarvin · 2022-10-23T19:41:56Z

Glad that it works! And I know about the whitespace. It's required by our hyperlink detection :)

olivier7121 · 2022-10-23T19:44:19Z

It's required by our hyperlink detection :)

Can you not perform a trimmed search?

JanMarvin · 2022-10-23T19:55:16Z

Have a look here:

openxlsx2/R/write.R

Lines 1036 to 1038 in 29ac147

    
           if (any(grepl("^(=|)HYPERLINK\\(", x, ignore.case = TRUE))) { 
        
             class(dfx$X) <- c("character", "formula", "hyperlink") 
        
           }

If you have any other idea for this, please submit a pull request. I guess you can write the trimws wrapper around your hyperlink function if you need one.

In all honesty, I do not care at all for hyperlinks or their colors and feel that I have spent already enough time on this issue.

olivier7121 · 2022-10-23T20:19:05Z

if (any(grepl("^(=|)HYPERLINK\\(", sub(" ", "", x), ignore.case = TRUE))) { 
   class(dfx$X) <- c("character", "formula", "hyperlink") 
 }

In all honesty, I do not care at all for hyperlinks or their colors and feel that I have spent already enough time on this issue.

As you wish. I do care about hyperlinks (they are quite useful :D) and their color is here for a reason.

JanMarvin · 2022-10-23T22:59:04Z

Because it fixes another issue with formulas: #388

This will treat every cell with HYPERLINK in it as hyperlink. This goes beyond what Excel does. In Excel this kind of hyperlink coming form a formula would not be colored. If this should be avoided, I'd like to do it in a single regex. Check that there is nothing meaningful between = and HYPERLINK(). (It's stuff like this that takes an insane amount of time.)

# works only with #388
wb <- wb_workbook()$
  add_worksheet("Tab_1", zoom = 80, gridLines = FALSE)$
  add_data(dims = "C1:F1", x = rbind(2016:2019), colNames = FALSE)$
  add_data(x = 2017, startCol = 1, startRow = 1, colNames = FALSE)$
  add_data_validation(rows = 1, col = 1, type = "list", value = '"2016,2017,2018,2019"')$
  add_formula(dims = "B1", x = '=HYPERLINK("#Tab_1!" & CELL("address", INDEX(C1:F1, MATCH(A1, C1:F1, 0))), "Go to the selected column")')$
  add_formula(dims = "B2", x = '=IF(2017 = VALUE(A1), HYPERLINK("github.com","github.com"), A1)')

olivier7121 · 2022-10-24T17:15:45Z

I don't understand your message (except the R code).

JanMarvin mentioned this pull request Sep 29, 2022

Formula requires escaped xml #348

Closed

JanMarvin force-pushed the data_validation_list branch from e7fd6d3 to 62553c0 Compare September 30, 2022 08:21

JanMarvin mentioned this pull request Oct 13, 2022

Roadmap for release 0.3.1 #374

Closed

5 tasks

JanMarvin added 4 commits October 15, 2022 12:07

update data validation type list

6d250d5

previously data_validation was an x14 extension which broke backward comparabilty with Excel 2007. Apparently this is not required.

fix tests

2d49a81

[test] old and new work together

d906996

compare old and new data validations list

Update NEWS

727a48d

JanMarvin force-pushed the data_validation_list branch from 969309f to 727a48d Compare October 15, 2022 10:12

JanMarvin merged commit be9d272 into main Oct 15, 2022

JanMarvin deleted the data_validation_list branch October 15, 2022 10:28

Data validation list #347

Data validation list #347

Conversation

JanMarvin commented Sep 29, 2022 • edited Loading

Todo

olivier7121 commented Sep 29, 2022

JanMarvin commented Sep 29, 2022

olivier7121 commented Sep 29, 2022

JanMarvin commented Sep 29, 2022

olivier7121 commented Sep 29, 2022

JanMarvin commented Sep 29, 2022

olivier7121 commented Sep 29, 2022

JanMarvin commented Sep 29, 2022

olivier7121 commented Sep 29, 2022

JanMarvin commented Sep 29, 2022

JanMarvin commented Oct 2, 2022

olivier7121 commented Oct 7, 2022

JanMarvin commented Oct 7, 2022

JanMarvin commented Oct 7, 2022

olivier7121 commented Oct 7, 2022

JanMarvin commented Oct 8, 2022

JanMarvin commented Oct 15, 2022

olivier7121 commented Oct 23, 2022

JanMarvin commented Oct 23, 2022

olivier7121 commented Oct 23, 2022 • edited Loading

JanMarvin commented Oct 23, 2022

olivier7121 commented Oct 23, 2022

JanMarvin commented Oct 23, 2022

olivier7121 commented Oct 23, 2022

JanMarvin commented Oct 23, 2022

olivier7121 commented Oct 23, 2022

JanMarvin commented Oct 23, 2022

olivier7121 commented Oct 23, 2022

olivier7121 commented Oct 23, 2022 • edited Loading

JanMarvin commented Oct 23, 2022

olivier7121 commented Oct 23, 2022

JanMarvin commented Oct 23, 2022

olivier7121 commented Oct 23, 2022

JanMarvin commented Oct 23, 2022

olivier7121 commented Oct 23, 2022 • edited Loading

JanMarvin commented Oct 23, 2022

olivier7121 commented Oct 24, 2022

JanMarvin commented Sep 29, 2022 •

edited

Loading

olivier7121 commented Oct 23, 2022 •

edited

Loading

olivier7121 commented Oct 23, 2022 •

edited

Loading

olivier7121 commented Oct 23, 2022 •

edited

Loading