From fb72abd267f0fcfd744f3ebb7ea738b47f68ae46 Mon Sep 17 00:00:00 2001 From: Tanguy Abel Date: Mon, 13 Feb 2023 14:15:57 +0100 Subject: [PATCH] docs: remove mention about future support for big data loading (#1118) Signed-off-by: Tanguy Abel --- docs/get-started/what-is.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/get-started/what-is.md b/docs/get-started/what-is.md index e9724d67431..41473ace02c 100644 --- a/docs/get-started/what-is.md +++ b/docs/get-started/what-is.md @@ -59,7 +59,7 @@ There are three other packages that people often compare DocArray to, yet I have [Hugging Face datasets](https://huggingface.co/docs/datasets/) is a library for easily accessing and sharing datasets for NLP, computer vision, and audio tasks. One of the highlights is its efficient loading of large datasets, which is highly appreciated during training. -In DocArray, there will also be a couple of feature releases soon to allow big data loading with constant memory consumption. However, the biggest difference is that DocArray is focused on **data in transit**, whereas HF Datasets is about **data at rest**. DocArray is focused on active data that is subject to frequent change; and allows efficient transfer between threads, processes and microservices. Data in transit often traverses a network or temporarily resides in memory to be read or updated. That's opposed to Datasets, where training data is stored physically and statically, and is subject to very occasional changes. The figure below depicts the differences: +The biggest difference is that DocArray is focused on **data in transit**, whereas HF Datasets is about **data at rest**. DocArray is focused on active data that is subject to frequent change; and allows efficient transfer between threads, processes and microservices. Data in transit often traverses a network or temporarily resides in memory to be read or updated. That's opposed to Datasets, where training data is stored physically and statically, and is subject to very occasional changes. The figure below depicts the differences: ```{figure} compare-data-type.svg :width: 90%