LABE

Merging citations and catalog data at SLUB Dresden.

Status: testing

Project

The project is composed of a couple of command line tools, both written in Python and Go.

• ckit, citation toolkit contains an API server, plus a few command line tools (go)
• python, orchestration helper to assemble data files regularly (python, based on luigi)

Meeting Minutes

• 2022-02-03 (wrap-up)
• 2022-01-20
• 2022-01-13
• 2022-01-06
• 2021-12-16
• 2021-11-25
• 2021-11-18
• 2021-11-04
• 2021-10-21
• 2021-10-14
• 2021-10-07
• 2021-09-30
• 2021-09-23
• 2021-09-16
• 2021-09-14 (kick-off)

Project structure

$ tree -d
.
├── ansible
│   └── roles
│       ├── common
│       │   └── tasks
│       └── labe
│           ├── defaults
│           ├── tasks
│           └── templates
├── data
├── extra
│   └── perfstats
├── go
│   └── ckit
│       ├── cache
│       ├── cmd
│       │   ├── doisniffer
│       │   ├── labed
│       │   ├── makta
│       │   └── tabjson
│       ├── doi
│       ├── fixtures
│       ├── packaging
│       │   └── deb
│       │       └── ckit
│       │           └── DEBIAN
│       ├── set
│       ├── static
│       ├── tabutils
│       ├── testdata
│       └── xflag
├── notes
│   └── 2022_01_30_performance_report_files
├── python
│   ├── labe
│   ├── packaging
│   │   └── deb
│   │       └── labe
│   │           └── DEBIAN
│   └── tests
└── static

40 directories

SLOC

$ tokei -C -t=Go,Python,yaml
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Go                     19         2666         2134          341          191
 Python                 16         2881         2345          126          410
 YAML                    4          249          193           35           21
===============================================================================
 Total                  39         5796         4672          502          622
===============================================================================

Ideas

• stats on combined oci, refcat graph; notes
• stats on combined oci, openalex (mag), refcat graph
• include "cited by" count in documents; may need a separate mapping database (with about 70M rows) for (doi, cited by count) -- could also be a COUNT on oci, but may want to have separate lookup table for performance (e.g. result would be just an int; db is about 4GB)

Misc

A data web service, lightning talk on the Go side of things at Leipzig Gophers 2021-11-23