Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project-based Workflow #651

Draft
wants to merge 86 commits into
base: new-project-structure
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
fac1fbf
Rename createDefaultProjectConfiguration to createProjectConfiguratio…
Felixmil Apr 18, 2024
4c4dc69
Harmonize Excel configuration files (#633)
Felixmil Apr 18, 2024
9d5e142
renamed the `Parameters` folder to `Configurations` (#631)
Felixmil Apr 18, 2024
b91a6b4
Created helpers for frequently used paths/ProjectConfiguration in tests
Felixmil Apr 18, 2024
e483c03
fix devtools::document() errors
Felixmil Apr 18, 2024
5e3d508
skip some test on CI (instead of CRAN)
Felixmil Apr 18, 2024
304a209
Update NEWS
Felixmil Apr 18, 2024
3ed3769
Merge remote-tracking branch 'origin/new-project-structure' into para…
Felixmil Apr 18, 2024
bded53d
enable GitHub actions when making PR in any branch
Felixmil Apr 18, 2024
9b8b75a
remove snapshot for projectConfiguration as it contains absolute path…
Felixmil Apr 18, 2024
9d9f89d
start working on project definition
Felixmil Apr 19, 2024
c3f77f3
Add populationsCSV path in projectConfiguration
Felixmil Apr 19, 2024
4a88401
Fix "Indiv" to "Indiv1" to prevent warnings
Felixmil Apr 19, 2024
060de9a
Merge remote-tracking branch 'origin/new-project-structure' into para…
Felixmil Apr 19, 2024
8a5c783
devtools::document()
Felixmil Apr 19, 2024
db97314
Update test expectation
Felixmil Apr 19, 2024
ae68c65
Merge branch 'paramsFile-to-modelParamsFile' into implement-project
Felixmil Apr 19, 2024
fa85e3d
Add Project object that contains all esqlabsR workflow components (co…
Felixmil May 6, 2024
c2de183
Improve projectConfiguration print method
Felixmil May 6, 2024
11bec57
style pkg
Felixmil May 6, 2024
3990ce7
keep inactivated scenarios status when changing configurations
Felixmil May 6, 2024
8ba5ec3
ignore excel temporary files
Felixmil May 15, 2024
14c2202
Merge branch 'new-project-structure' into implement-project
Felixmil May 15, 2024
24c7dde
Add an optional path argument to save ProjectCOnfiguration
Felixmil May 16, 2024
2c45a72
Add test for Project
Felixmil May 16, 2024
baf1b05
rename file and relocate functions
Felixmil May 29, 2024
f8ce9fd
Rename files and add "-legacy" to older versions (temporary)
Felixmil May 29, 2024
324b151
style + document
Felixmil May 29, 2024
bf39aaa
replace function with latest name
Felixmil May 29, 2024
a6ffb83
Better user feedback on simulation results
Felixmil May 30, 2024
82b9d94
Add documentation
Felixmil May 30, 2024
a334d50
Add simulationTime objects
Felixmil May 30, 2024
2360f80
Rename time and timeUnit to simulationTime and simulationTimeUnit
Felixmil May 31, 2024
46dee0b
Apply simulationTime output intervals to simulation
Felixmil May 31, 2024
51c0e44
Add OutputPaths
Felixmil Jun 5, 2024
2075bc2
Add simulationTimeUnit validation
Felixmil Jun 6, 2024
0ae6e46
Add steadyState simulation run
Felixmil Jun 6, 2024
902d537
Add pretty printing method for individuals
Felixmil Jun 6, 2024
f0eae00
Rename validation function
Felixmil Jun 6, 2024
604571b
Rename ModelParameter to SimulationParameter
Felixmil Jun 11, 2024
e9ffb05
Add Population object and pop simulation capabilities
Felixmil Jun 11, 2024
84a7289
devtools::document()
Felixmil Jun 11, 2024
c5e768d
minor fix and optimizations
Felixmil Jun 11, 2024
07bfb1e
merge branch `main` into `implement-project`
Felixmil Jul 30, 2024
c673ca4
git merge
Felixmil Jul 30, 2024
e9f2da4
Merge branch 'new-project-structure' into implement-project
Felixmil Jul 30, 2024
ee5a512
Merge branch 'new-project-structure' into implement-project
Felixmil Aug 28, 2024
cabbeea
Merge branch 'implement-project' of https://github.com/esqLABS/esqlab…
Felixmil Aug 28, 2024
e0a1a7f
Update project.R
AKostiv8 Sep 17, 2024
108dddb
Prevent user from modifying scenarios directly
Felixmil Sep 18, 2024
9856304
Refactor repeated error messages
Felixmil Sep 19, 2024
4858a9f
config validation
AKostiv8 Oct 1, 2024
3b52677
Feedback implemented
AKostiv8 Oct 2, 2024
9edd4c8
Update project.R
AKostiv8 Oct 2, 2024
d365de0
Project status tests
AKostiv8 Oct 3, 2024
15457b4
Test adjustment to the new project structure.
AKostiv8 Oct 7, 2024
a6b9298
update time param warning
AKostiv8 Oct 7, 2024
4ae81b5
Merge commit '7d4271027aff77abe19c41cf1749106570fce118'
Felixmil Oct 10, 2024
6ee1646
replace deprecated function
Felixmil Oct 11, 2024
758f034
pull from main
Felixmil Oct 11, 2024
032a45d
Update path to where results are stored now
Felixmil Oct 11, 2024
b41e61d
Remove embeded shiny apps
Felixmil Oct 11, 2024
ef67c18
Update plots test for new project structure
Felixmil Oct 11, 2024
cda1c45
remove/relocate outdated test
Felixmil Oct 11, 2024
5ff0443
update snaps
Felixmil Oct 11, 2024
bdbc595
shiny app remove
AKostiv8 Oct 14, 2024
872e3d2
shiny app remove
AKostiv8 Oct 14, 2024
803c2a3
conflicts resolved
AKostiv8 Oct 14, 2024
a9d5a1e
Rename / Comment out of date tests
Felixmil Oct 14, 2024
860c0d7
remove shiny app launching script
Felixmil Oct 14, 2024
0bccc89
merge from main
Felixmil Oct 14, 2024
fe55689
devtools::documen()
Felixmil Oct 14, 2024
17081d0
merge new-project-structure into branch
Felixmil Oct 14, 2024
42c8b55
Merge branch 'individual-validation-demo' into implement-project
AKostiv8 Oct 14, 2024
a8df746
Merge branch 'implement-project' of https://github.com/esqLABS/esqlab…
AKostiv8 Oct 14, 2024
b4d589a
Revert "Validation object & individual modification validation" (#726)
AKostiv8 Oct 14, 2024
e42ecdb
update snapshot after updating packages
Felixmil Oct 14, 2024
f5d01f1
Merge branch 'implement-project' of https://github.com/esqLABS/esqlab…
Felixmil Oct 14, 2024
7cb9054
fix code examples in documentation
Felixmil Oct 14, 2024
34b110d
fix path to Example Project Configuration File
Felixmil Oct 14, 2024
85a04d1
remove old test files
Felixmil Oct 14, 2024
d67fd79
fix exampleProject
Felixmil Oct 14, 2024
247d269
Move some objects to Internal
Felixmil Oct 14, 2024
84aea3b
disable vignettes (temporary)
Felixmil Oct 14, 2024
050fbd7
restore original build options
Felixmil Oct 14, 2024
e267a54
remove unsuported character in doc
Felixmil Oct 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,4 @@ tests/testthat/Rplots.pdf
*.code-workspace
docs
pkgdown
**/~\$*\.xlsx
1 change: 1 addition & 0 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Depends:
ospsuite (>= 12.1.0),
R (>= 4.1)
Imports:
cli,
colorspace,
dplyr,
fs,
Expand Down
19 changes: 6 additions & 13 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,9 @@ export(ExportConfiguration)
export(GenderInt)
export(LLOQMode)
export(ProjectConfiguration)
export(Scenario)
export(ScenarioConfiguration)
export(ScenarioConfigurationLegacy)
export(ScenarioLegacy)
export(ULOQMode)
export(applyIndividualParameters)
export(calculateMeanDataSet)
export(col2hsv)
export(compareSimulations)
Expand All @@ -21,12 +20,13 @@ export(createEsqlabsPlotConfiguration)
export(createEsqlabsPlotGridConfiguration)
export(createPlotsFromExcel)
export(createProjectConfiguration)
export(createScenarios)
export(enumPutList)
export(escapeForRegex)
export(esqLABS_colors)
export(esqlabsRSettingNames)
export(example_ProjectConfiguration)
export(exampleProject)
export(exampleProjectConfiguration)
export(exampleProjectConfigurationFile)
export(executeInParallel)
export(exportParametersToXLS)
export(extendParameterStructure)
Expand All @@ -41,35 +41,28 @@ export(getIndexClosestToValue)
export(getMoleculeNameFromQuantity)
export(hillFunction)
export(init_project)
export(initializeSimulation)
export(isParametersEqual)
export(isTableFormulasEqual)
export(loadObservedData)
export(loadObservedDataFromPKML)
export(loadScenarioResults)
export(pathFromClipboard)
export(readExcel)
export(readIndividualCharacteristicsFromXLS)
export(readParametersFromXLS)
export(readPopulationCharacteristicsFromXLS)
export(readScenarioConfigurationFromExcel)
export(removeFromList)
export(runScenarios)
export(sampleRandomValue)
export(saveScenarioResults)
export(sensitivityCalculation)
export(sensitivitySpiderPlot)
export(sensitivityTimeProfiles)
export(sensitivityTornadoPlot)
export(setApplications)
export(setParameterValuesByPathWithCondition)
export(sineFunction)
export(sourceAll)
export(startFunctionVisualizer)
export(startUnitConverter)
export(stringToNum)
export(writeIndividualToXLS)
export(writeParameterStructureToXLS)
import(cli)
import(fs)
import(ggplot2)
import(grDevices)
Expand Down
219 changes: 137 additions & 82 deletions R/utilities-parameters.R → R/SimulationParameter.R
Original file line number Diff line number Diff line change
@@ -1,99 +1,154 @@
#' Read parameter values from a structured Excel file.
#' Each excel sheet must consist of columns 'Container Path', 'Parameter Name',
#' 'Value', and 'Units'
#' @title Model Parameter
#' @description A class representing a parameter that can be applied to a simulation.
SimulationParameter <- R6::R6Class(
classname = "SimulationParameter",
public = list(
#' @description Creates a new instance of SimulationParameter
#' @param project A Project in which the parameter is defined.
#' @param modelParameterData A data frame row containing the parameter data with the columns: "Container Path", "Parameter Name", "Value", "Units".
initialize = function(project, modelParameterData) {
private$.project <- project
self$containerPath <- modelParameterData$`Container Path`
self$parameterName <- modelParameterData$`Parameter Name`
self$value <- modelParameterData$Value
self$units <- modelParameterData$Units
},
#' @description Prints the parameter.
print = function() {
print(private$.parameter)
},
#' @description Converts the parameter to a data frame.
toDataFrame = function() {
return(
tibble::tibble(
`Container Path` = self$containerPath,
`Parameter Name` = self$parameterName,
Value = self$value,
Units = self$units
)
)
}
),
active = list(
#' @field containerPath The container path of the parameter.
containerPath = function(value) {
if (!missing(value)) {
private$.parameter$containerPath <- value
}
return(private$.parameter$containerPath)
},
#' @field parameterName The name of the parameter.
parameterName = function(value) {
if (!missing(value)) {
private$.parameter$parameterName <- value
}
return(private$.parameter$parameterName)
},
#' @field value The value of the parameter.
value = function(value) {
if (!missing(value)) {
private$.parameter$value <- value
}

Check warning on line 51 in R/SimulationParameter.R

View check run for this annotation

Codecov / codecov/patch

R/SimulationParameter.R#L51

Added line #L51 was not covered by tests
return(private$.parameter$value)
},

Check warning on line 53 in R/SimulationParameter.R

View check run for this annotation

Codecov / codecov/patch

R/SimulationParameter.R#L53

Added line #L53 was not covered by tests
#' @field units The units of the parameter.
units = function(value) {
if (!missing(value)) {
if (is.na(value)){
value <- ""
}
private$.parameter$units <- value
}
return(private$.parameter$units)
},
#' @field parameterObject Returns the parameter as a `{ospsuite}` compatible list.
parameterObject = function() {
return(
list(
paths = paste(self$containerPath, self$parameterName, sep = "|"),
values = self$value,
units = self$units
)
)
}
),
private = list(
.project = NULL,
.parameter = list()
)
)

#' @title flattenParametersObjects

Check warning on line 81 in R/SimulationParameter.R

View check run for this annotation

Codecov / codecov/patch

R/SimulationParameter.R#L73-L81

Added lines #L73 - L81 were not covered by tests
#' Transform a set of `SimulationParameter` objects into one list of parameters.
#'
#' @param paramsXLSpath Path to the excel file
#' @param sheets Names of the excel sheets containing the information about the
#' parameters. Multiple sheets can be processed. If no sheets are provided,
#' the first one in the Excel file is used.
#' @param simulationParameters list of `SimulationParameter`
#'
#' @return A list containing vectors 'paths' with the full paths to the
#' parameters, 'values' the values of the parameters, and 'units' with the
#' units the values are in.
#' @export
readParametersFromXLS <- function(paramsXLSpath, sheets = NULL) {
#' @keywords internal
flattenParameterObjects <- function(simulationParameters) {

Check warning on line 87 in R/SimulationParameter.R

View check run for this annotation

Codecov / codecov/patch

R/SimulationParameter.R#L86-L87

Added lines #L86 - L87 were not covered by tests
list(
paths = purrr::map(simulationParameters, ~ purrr::pluck(.x, "paths")) %>% purrr::list_c(),
values = purrr::map(simulationParameters, ~ purrr::pluck(.x, "values")) %>% purrr::list_c(),
units = purrr::map(simulationParameters, ~ purrr::pluck(.x, "units")) %>% purrr::list_c()
)
}

validateParametersFileStructure <- function(filePath, data) {
columnNames <- c("Container Path", "Parameter Name", "Value", "Units")
validateIsString(paramsXLSpath)
validateIsString(sheets, nullAllowed = TRUE)

if (is.null(sheets)) {
sheets <- c(1)
if (!all(columnNames %in% names(data))) {
stop(messages$errorWrongXLSStructure(filePath = filePath, expectedColNames = columnNames))

Check warning on line 99 in R/SimulationParameter.R

View check run for this annotation

Codecov / codecov/patch

R/SimulationParameter.R#L99

Added line #L99 was not covered by tests
}
}

pathsValuesVector <- vector(mode = "numeric")
pathsUnitsVector <- vector(mode = "character")

createParametersFromFile <- function(project, filePath) {
fileModelParameters <- list()
sheets <- readxl::excel_sheets(filePath)
for (sheet in sheets) {
data <- readExcel(path = paramsXLSpath, sheet = sheet)
fileModelParameters[[sheet]] <- createParametersFromSheet(project = project, filePath = filePath, sheet = sheet)
}
return(fileModelParameters)
}

if (!all(columnNames %in% names(data))) {
stop(messages$errorWrongXLSStructure(filePath = paramsXLSpath, expectedColNames = columnNames))
createParametersFromSheet <- function(project, filePath, sheet) {
sheetModelParametersData <- readExcel(filePath, sheet = sheet)
validateParametersFileStructure(
filePath = filePath,
data = sheetModelParametersData
)
sheetModelParameters <- list()
for (i in 1:nrow(sheetModelParametersData)) {
parameterData <- sheetModelParametersData[i, ]
parameterID <- parameterData$`Parameter Name`

i <- 0
while (parameterID %in% names(sheetModelParameters)) {
parameterID <- parameterData$`Parameter Name`
parameterID <- paste(
stringr::str_replace_all(
stringr::str_extract(
parameterData$`Container Path`,
paste0(paste(rep("[^\\|]*", i), collapse = "\\|"), "[^\\|]*$")
), "\\|", "_"
),
parameterID,
sep = "_"
)
i <- i + 1

Check warning on line 136 in R/SimulationParameter.R

View check run for this annotation

Codecov / codecov/patch

R/SimulationParameter.R#L125-L136

Added lines #L125 - L136 were not covered by tests
}

fullPaths <- paste(data[["Container Path"]], data[["Parameter Name"]], sep = "|")
pathsValuesVector[fullPaths] <- as.numeric(data[["Value"]])

pathsUnitsVector[fullPaths] <- tidyr::replace_na(data = as.character(data[["Units"]]), replace = "")
sheetModelParameters[[parameterID]] <-
SimulationParameter$new(
project = project,
modelParameterData = parameterData
)
}

return(.parametersVectorToList(pathsValuesVector, pathsUnitsVector))
return(sheetModelParameters)
}

#' Write parameter structure to excel that can be loaded in MoBi
#'
#' @param parameterStructure A list containing vectors 'paths' with the full paths to the parameters,
#' 'values' the values of the parameters, and 'units' with the units the values are in.
#'
#' @param paramsXLSpath Path to the excel file
#' @param sheet (Optional) name of the excel sheet
#' @param append If TRUE, the existing excel file/sheet will be appended with the new
#' parameter structure. If FALSE (default), the existing file will be
#' overwritten.
#'
#' @export
#'
#' @examples
#' \dontrun{
#' params <- list(paths = c("Container1|Path1", "Container|Second|Third|Path2"), values = c(1, 2), units = c("", "µmol"))
#'
#' writeParameterStructureToXLS(params, "test.xlsx")
#' }
#'
writeParameterStructureToXLS <- function(parameterStructure, paramsXLSpath, sheet = NULL, append = FALSE) {
if (isTRUE(append)) {
existingData <- readParametersFromXLS(paramsXLSpath = paramsXLSpath, sheets = sheet)
parameterStructure$paths <- c(existingData$paths, parameterStructure$paths)
parameterStructure$values <- c(existingData$values, parameterStructure$values)
parameterStructure$units <- c(existingData$units, parameterStructure$units)
}

.validateParametersStructure(parameterStructure, "parameterStructure")
# Split full parameter paths into container path and parameter name
containerPaths <- unlist(lapply(parameterStructure$paths, \(x){
.splitParameterPathIntoContainerAndName(x)$containerPath
}), use.names = FALSE)
parameterNames <- unlist(lapply(parameterStructure$paths, \(x){
.splitParameterPathIntoContainerAndName(x)$parameterName
}), use.names = FALSE)

# Create a data frame with the parameter structure
output <- data.frame(
"Container Path" = containerPaths,
"Parameter Name" = parameterNames,
Value = parameterStructure$values,
Units = parameterStructure$units,
check.names = FALSE
)

# Write the results into an excel file.
# Wrap the output data frame into a list and name the list if sheet name
# has been provided
data <- list(output)
if (!is.null(sheet)) {
names(data) <- sheet
}
.writeExcel(data = data, path = paramsXLSpath)
}

# Legacy code -------------------------------------------------------------

#' Export simulation parameters to excel
#'
Expand Down Expand Up @@ -278,7 +333,7 @@
# Check for formula type equality
if (!all(
c(formula1$isConstant, formula1$isDistributed, formula1$isExplicit, formula1$isTable) ==
c(formula2$isConstant, formula2$isDistributed, formula2$isExplicit, formula2$isTable)
c(formula2$isConstant, formula2$isDistributed, formula2$isExplicit, formula2$isTable)

Check warning on line 336 in R/SimulationParameter.R

View check run for this annotation

Codecov / codecov/patch

R/SimulationParameter.R#L336

Added line #L336 was not covered by tests
)) {
return(FALSE)
}
Expand Down
Loading
Loading