All-in-one text de-duplication
-
Updated
May 21, 2024 - Python
#
de-duplication
Entity resolution (also known as data matching, data linkage, record linkage, and many other terms) is the task of finding entities in a dataset that refer to the same entity across different data sources (e.g., data files, books, websites, and databases). Entity resolution is necessary when joining different data sets based on entities that may or may not share a common identifier (e.g., database key, URI, National identification number), which may be due to differences in record shape, storage location, or curator style or preference.
Here are 10 public repositories matching this topic...
CD4Py: Code De-Duplication for Python
-
Updated
Dec 13, 2020 - Python
Container image converter aiming to minimize image size and speed up boot time dramatically with block-level de-dupliction and lazy-pull technology.
-
Updated
Mar 14, 2019 - C
Preprocesses the query logs which can be used by suggesters like Most Popular Suggester (MPS).
parser
autocomplete
preprocessor
autosuggest
autocomplete-search
spell-checker
de-duplication
profanity-filter
symspell
-
Updated
Aug 20, 2024 - Python
A collection of algorithms to generate a signature/fingerprint/hash in order to be used for detecting duplicate/near duplicate documents.
rust
profile
text
fingerprint
hash
cargo
deduplication
de-duplication
text-profile-signature
lookup3
-
Updated
Aug 9, 2017 - Rust
Efficient Event Streamlining and Dynamic De-Duplication Across Message Brokers - A Technology-Agnostic approach
-
Updated
Apr 7, 2023 - TypeScript
Repository contains java application code to stream records to a Kinesis Stream, consume , de-duplicate and strictly order records and display records on a dashboard
dashboard
streams
kinesis-stream
kcl
kpl
de-duplication
iam-role
kinesis-producer-librarry
strict-ordering
high-velocity-data
kinesis-client-library
real-time-streaming
-
Updated
May 15, 2017 - Java
A misc project of python based function to track, survey and manage files from mutiple systems
-
Updated
Oct 6, 2021 - Python
A secure image storage application written in Python
-
Updated
Nov 22, 2017 - Python
Created by Halbert L. Dunn
Released 1946
- Followers
- 38 followers
- Organization
- entity-resolution
- Wikipedia
- Wikipedia