lupa is an auditable R toolkit for profiling tabular data, defining what
quality means for a particular use, measuring it, cleaning a copy without
silently changing the input, and finding approximate duplicate records at
scale. It reports the scope, evidence, and uncertainty of every result.
The public API, help pages, and vignettes are in Spanish. The names are stable and can be copied from this English guide; the Spanish README tells the same story in Spanish.
The public names are Spanish in both examples and help pages:
| Spanish API | English meaning |
|---|---|
perfilar() |
profile |
analizar() |
analyse |
marco_calidad() |
quality framework |
planificar_limpieza() |
plan a cleanup |
guiar_limpieza() |
guide a cleanup |
aplicar() |
apply a selected cleanup |
medir() / evaluar() |
measure / evaluate |
detectar_duplicados_aproximados() |
find approximate duplicates |
reportar() |
create a report |
- Profiles a delivery and surfaces missingness, types, patterns, dates, and personal-data evidence.
- Finds keys, relationships, dependencies, and measurement granularities that were never declared.
- Lets a project define its own quality framework instead of forcing a global score.
- Measures and evaluates explicit metrics, scales, validity rules, and referential domains.
- Produces editable cleanup plans, applies only selected actions to a copy, and keeps an audit log.
- Finds approximate duplicates with exact tiles, deterministic MinHash/LSH, blocking, cost estimates, and disk-backed lots.
- Repairs encoding damage in R, including repeated mojibake and CESU-8, while refusing unsafe lossy conversions.
- Follows quality through time and creates self-contained HTML reports.
Until the first CRAN release, install the development version directly from GitHub:
# install.packages("pak")
pak::pak("sebollin/lupa")For a local clone or release tarball, use:
R CMD build .
R CMD INSTALL lupa_0.1.0.tar.gzOr install a local source tarball from R:
install.packages("lupa_0.1.0.tar.gz", repos = NULL)library(lupa)
data(datos_operativos)
perfil <- perfilar(datos_operativos, analizar_dependencias = FALSE)
head(perfil$hallazgos[, c("columna", "tipo_hallazgo", "severidad")], 5)
analisis <- analizar(datos_operativos)
archivo <- tempfile(fileext = ".html")
reportar(analisis, archivo = archivo)
stopifnot(file.exists(archivo))
unlink(archivo)The profile is read-only: it never changes the input table. Findings are ordinary inspectable data frames, and personal-data evidence is masked when the classification warrants it. A real console preview is shown below.
The pkgdown reference and the linked vignettes are the detailed manual. This table is the short map:
| Task | Main functions | Read more |
|---|---|---|
| Look at data for the first time | perfilar(), analizar(), distribucion_valores(), detectar_asociaciones(), analizar_tiempo(), clasificar_variables(), inferir_tipo(), descubrir_patrones(), detectar_formatos_fecha(), sentinelas_naniar |
Getting started |
| Find undeclared structure | detectar_claves(), detectar_relaciones(), detectar_dependencias(), granularidades(), transiciones_granularidad() |
Getting started |
| Define quality | marco_calidad(), marco_agesic(), marco_iso25012(), catalogo_agesic(), metrica(), especializar(), instanciar(), modelo(), metricas_nucleo(), metricas_referencial(), proponer_modelo(), modelo_desde_propuesta(), perfiles_madurez(), cobertura_analisis() |
Quality model |
| Measure and evaluate | medir(), agregar(), evaluar(), regla_evaluacion(), perfil_evaluacion(), escala(), referencial(), vigencia() |
Quality model |
| Clean safely | planificar_limpieza(), guiar_limpieza(), aplicar() |
Cleaning plan |
| Find approximate duplicates | detectar_duplicados_aproximados(), estimar_costo() |
Scale and duplicates |
| Repair encoding damage | reparar_codificacion through planificar_limpieza() and aplicar() |
Cleanup reference |
| Follow quality over time | historico_calidad(), acumular_historico(), guardar_historico(), leer_historico(), detectar_deriva_calidad(), comparar_perfiles(), comparar_evaluaciones() |
History and drift |
| Share results | reportar(), guardar_analisis(), leer_analisis() |
Reporting reference |
| Validate and extend | validadores_internacionales(), validadores_uruguay(), pack_validadores(), validar_ci_uy(), validar_rut_uy(), validar_luhn(), validar_mod97(), validar_iso3166(), validar_iso4217(), validar_correo() |
Reference |
library(lupa)
data(datos_operativos)
marco <- marco_calidad(
"Marco operativo",
list(Estructura = c("Ausencias observadas", "Duplicacion exacta"))
)
propuesta <- proponer_modelo(perfilar(datos_operativos,
analizar_dependencias = FALSE))
list(marco = marco, propuesta = propuesta)The API has a few boundaries worth knowing. There is no global quality score:
dimensions, units, and rules stay visible. The core is universal and
catalogues are pluggable; AGESIC
v1.6 is a reference implementation, not a country lock. The package has one
required import, cli;
stringdist is optional.
skimr and
DataExplorer explore;
pointblank,
validate, and
dataquieR express or evaluate
rules; zoomerjoin,
textreuse, and
reclin2 focus on text comparison
or record linkage. calidad, maintained
by Klaus Lehmann and
Ricardo Pizarro, is a complementary axis:
it evaluates the quality of survey estimates, while lupa evaluates the
tabular data that produces an estimate.
Encoding repair follows the approach and frozen data of
ftfy 6.3.1 by
Robyn Speer, in R. It includes eleven byte tables,
CESU-8 and Java C0 80 handling, and five deliberate extensions documented in
the NEWS. It reproduces 159 of the 161 distributed corpus cases and
leaves all 31 negative cases untouched. It deliberately does not provide
ftfy's style-oriented fix_text steps such as HTML unescaping, quote
curling, width normalization, or Unicode normalization: changing legitimate
data silently is not repair.
citation("lupa")Conceptual references are Batini and Scannapieco (2016), the AGESIC Digital Government Data Quality Framework v1.6, and ISO/IEC 25012:2008.
Please use the issue tracker for bugs, proposals, and documentation fixes. The stable contracts are the declared units, scope, protection, and audit trail; implementation details and benchmark times can change between releases when those contracts remain true.
lupa is released under the GPL-3.
See LICENSE.note for the Apache-2.0 data derived from
ftfy and the MIT data derived from
naniar.