-
Notifications
You must be signed in to change notification settings - Fork 51
/
logging.R
256 lines (242 loc) · 7.95 KB
/
logging.R
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
#------------------------------------------------------------------------------#
#
# _ _ _ _ _
# (_) | | | | | | | |
# _ __ ___ _ _ __ | |_ | |__ | | __ _ _ __ | | __
# | '_ \ / _ \ | || '_ \ | __|| '_ \ | | / _` || '_ \ | |/ /
# | |_) || (_) || || | | || |_ | |_) || || (_| || | | || <
# | .__/ \___/ |_||_| |_| \__||_.__/ |_| \__,_||_| |_||_|\_\
# | |
# |_|
#
# This file is part of the 'rstudio/pointblank' project.
#
# Copyright (c) 2017-2024 pointblank authors
#
# For full copyright and license information, please look at
# https://rstudio.github.io/pointblank/LICENSE.html
#
#------------------------------------------------------------------------------#
# nocov start
#' Enable logging of failure conditions at the validation step level
#'
#' @description
#'
#' The `log4r_step()` function can be used as an action in the [action_levels()]
#' function (as a list component for the `fns` list). Place a call to this
#' function in every failure condition that should produce a log (i.e., `warn`,
#' `stop`, `notify`). Only the failure condition with the highest severity for a
#' given validation step will produce a log entry (skipping failure conditions
#' with lower severity) so long as the call to `log4r_step()` is present.
#'
#' @param x A reference to the x-list object prepared by the `agent`. This
#' version of the x-list is the same as that generated via
#' `get_agent_x_list(<agent>, i = <step>)` except this version is internally
#' generated and hence only available in an internal evaluation context.
#'
#' @param message The message to use for the log entry. When not provided, a
#' default glue string is used for the messaging. This is dynamic since the
#' internal `glue::glue()` call occurs in the same environment as `x`, the
#' x-list that's constrained to the validation step. The default message, used
#' when `message = NULL` is the glue string `"Step {x$i} exceeded the {level}
#' failure threshold (f_failed = {x$f_failed}) ['{x$type}']"`. As can be seen,
#' a custom message can be crafted that uses other elements of the x-list with
#' the `{x$<component>}` construction.
#'
#' @param append_to The file to which log entries at the warn level are
#' appended. This can alternatively be one or more **log4r** appenders.
#'
#' @return Nothing is returned however log files may be written in very specific
#' conditions.
#'
#' @section YAML:
#'
#' A **pointblank** agent can be written to YAML with [yaml_write()] and the
#' resulting YAML can be used to regenerate an agent (with [yaml_read_agent()])
#' or interrogate the target table (via [yaml_agent_interrogate()]). Here is an
#' example of how `log4r_step()` can be expressed in R code (within
#' [action_levels()], itself inside [create_agent()]) and in the corresponding
#' YAML representation.
#'
#' R statement:
#'
#' ```r
#' create_agent(
#' tbl = ~ small_table,
#' tbl_name = "small_table",
#' label = "An example.",
#' actions = action_levels(
#' warn_at = 1,
#' fns = list(
#' warn = ~ log4r_step(
#' x, append_to = "example_log"
#' )
#' )
#' )
#' )
#' ```
#'
#' YAML representation:
#'
#' ```yaml
#' type: agent
#' tbl: ~small_table
#' tbl_name: small_table
#' label: An example.
#' lang: en
#' locale: en
#' actions:
#' warn_count: 1.0
#' fns:
#' warn: ~log4r_step(x, append_to = "example_log")
#' steps: []
#' ```
#'
#' Should you need to preview the transformation of an *agent* to YAML (without
#' any committing anything to disk), use the [yaml_agent_string()] function. If
#' you already have a `.yml` file that holds an *agent*, you can get a glimpse
#' of the R expressions that are used to regenerate that agent with
#' [yaml_agent_show_exprs()].
#'
#' @section Examples:
#'
#' For the example provided here, we'll use the included `small_table` dataset.
#' We are also going to create an `action_levels()` list object since this is
#' useful for demonstrating a logging scenario. It will have a threshold for
#' the `warn` state, and, an associated function that should be invoked
#' whenever the `warn` state is entered. Here, the function call with
#' `log4r_step()` will be invoked whenever there is one failing test unit.
#'
#' ```{r}
#' al <-
#' action_levels(
#' warn_at = 1,
#' fns = list(
#' warn = ~ log4r_step(
#' x, append_to = "example_log"
#' )
#' )
#' )
#' ```
#'
#' Within the [action_levels()]-produced object, it's important to match things
#' up: notice that `warn_at` is given a threshold and the list of functions
#' given to `fns` has a `warn` component.
#'
#' Printing `al` will show us the settings for the `action_levels` object:
#'
#' ```{r}
#' al
#' ```
#'
#' Let's create an agent with `small_table` as the target table. We'll apply the
#' `action_levels` object created above as `al`, add two validation steps, and
#' then [interrogate()] the data.
#'
#' ```r
#' agent <-
#' create_agent(
#' tbl = ~ small_table,
#' tbl_name = "small_table",
#' label = "An example.",
#' actions = al
#' ) %>%
#' col_vals_gt(columns = d, 300) %>%
#' col_vals_in_set(columns = f, c("low", "high")) %>%
#' interrogate()
#'
#' agent
#' ```
#'
#' \if{html}{
#' \out{
#' `r pb_get_image_tag(file = "man_log4r_step_1.png")`
#' }
#' }
#'
#' From the agent report, we can see that both steps have yielded warnings upon
#' interrogation (i.e., filled yellow circles in the `W` column).
#'
#' What's not immediately apparent is that when entering the `warn` state
#' in each validation step during interrogation, the `log4r_step()` function
#' call was twice invoked! This generated an `"example_log"` file in the working
#' directory (since it was not present before the interrogation) and log entries
#' were appended to the file. Here are the contents of the file:
#'
#' ```
#' WARN [2022-06-28 10:06:01] Step 1 exceeded the WARN failure threshold
#' (f_failed = 0.15385) ['col_vals_gt']
#' WARN [2022-06-28 10:06:01] Step 2 exceeded the WARN failure threshold
#' (f_failed = 0.15385) ['col_vals_in_set']
#' ```
#'
#' @family Logging
#' @section Function ID:
#' 5-1
#'
#' @export
log4r_step <- function(
x,
message = NULL,
append_to = "pb_log_file"
) {
if (!requireNamespace("log4r", quietly = TRUE)) {
stop(
"Using the `log4r_step()` function requires the log4r package:\n",
"* It can be installed with `install.packages(\"log4r\")`.",
call. = FALSE
)
}
type <- x$this_type
warn_val <- x$warn
stop_val <- x$stop
notify_val <- x$notify
log4r_fn_present <-
vapply(
c("warn", "stop", "notify"),
FUN.VALUE = logical(1),
USE.NAMES = FALSE,
FUN = function(y) {
grepl(
"log4r_step(x",
paste(
as.character(x$actions[[paste0("fns.", y)]]),
collapse = ""
),
fixed = TRUE
)
}
)
level <- toupper(type)
level_val <-
switch(
level,
"WARN" = 3,
"STOP" = 4,
"NOTIFY" = 5,
3
)
# Skip logging at this level if a higher severity
# condition is present for this validation step *and*
# there is a `log4r_step()` function ready for
# evaluation at those higher severities
if (warn_val && log4r_fn_present[1]) highest_level <- 3
if (stop_val && log4r_fn_present[2]) highest_level <- 4
if (notify_val && log4r_fn_present[3]) highest_level <- 5
if (highest_level > level_val) {
return(invisible(NULL))
}
if (is.character(append_to)) {
appenders <- log4r::file_appender(file = append_to[1])
}
logger <- log4r::logger(appenders = appenders)
log4r::levellog(
logger = logger,
level = level_val,
message = glue::glue(
"Step {x$i} exceeded the {level} failure threshold \\
(f_failed = {x$f_failed}) ['{x$type}']"
)
)
}
# nocov end