Very slow vector indexing when loading a dataset with embedding via GraphImporter

[warframesam]

Tried loading a dataset with embeddings via GraphImporter (~13 million vertices containing 768 dim. embeddings and ~300 million edges). The program slowed down with time when building vector index.
NOTE: Program was manually killed after about 21 hours
arcadedb server(dockerized) was shutdown before running GraphImporter

Logs -> arcadedb_26.11.1-SNAPSHOT_GraphImporter_with_embedding.log

Observations

• Vertices processing throughput is much slower compared to data load without embedding
• Embedding vector index build becomes slow with time(almost stalled at the very end of the log)

Setup

arcadedb version

docker image: 26.11.1-SNAPSHOT
arcadedb-integration jar: 26.11.1-SNAPSHOT

arcadedb schema

CREATE VERTEX TYPE WORK;

-- Define Properties on WORK Type
CREATE PROPERTY WORK.id IF NOT EXISTS LONG;
CREATE PROPERTY WORK.title IF NOT EXISTS STRING;
CREATE PROPERTY WORK.abstract IF NOT EXISTS STRING;
CREATE PROPERTY WORK.authors IF NOT EXISTS LIST OF STRING;
CREATE PROPERTY WORK.publication_year IF NOT EXISTS INT;
CREATE PROPERTY WORK.embeddings ARRAY_OF_FLOATS;

-- Create Unique Index on id
CREATE INDEX WORK_id ON WORK (id) UNIQUE;

-- Create vector index with 384 dimensions using COSINE similarity and INT8 quantization
CREATE INDEX ON WORK (embeddings) LSM_VECTOR METADATA {
  "dimensions": 768,
  "similarity": "COSINE",
  "quantization": "INT8"
};

-- Define Edge Type
CREATE EDGE TYPE CITE;

JVM opts

-Xmx24g

Java code

public class ArcadeDBGraphImporter {
    public static void main(String[] args) {

        ArcadeDBGraphImporterConfig arcadeDBGraphImporterConfig = new ArcadeDBGraphImporterConfig();

        try (DatabaseFactory arcade = new DatabaseFactory(arcadeDBGraphImporterConfig.getDatabasePath())) {
            try (Database database = arcade.open()) {
                try (GraphImporter importer = GraphImporter.builder(database)
                        .vertex("WORK", new JsonlRowSource(arcadeDBGraphImporterConfig.getVertexSourcePath()), v -> {
                            v.id("id");
                            v.longProperty("id", "id");
                            v.property("title", "title");
//                            v.property("title_lang", "title_lang");
                            v.property("abstract", "abstract");
                            v.property("field", "field");
//                            v.property("abstract_lang", "abstract_lang");
                            v.listProperty("authors", "authors");
                            v.intProperty("publication_year", "publication_year");
                    	    v.floatArrayProperty("embeddings", "embeddings");
                        })
                        .edgeSource("CITE", new CsvRowSource(arcadeDBGraphImporterConfig.getEdgeSourcePath()), e -> {
                            e.from("from_id", "WORK");
                            e.to("to_id", "WORK");
                        })
                        .build()) {

                    importer.run();
                    System.out.printf("\nVertices: %,d, Edges: %,d%n",
                            importer.getVertexCount(), importer.getEdgeCount());
                } catch (Exception e) {
                    throw new RuntimeException(e);
                }
            }
        }
    }
}

Dataset

meta

• ~13 million vertices in jsonl
• ~300 million edges in csv
• embeddings dimension: 768

sample edges.csv

from_id,to_id
2184705490,96540192
2184705490,135511430

sample vertices.jsonl

{"id": "2184705490", "type": "dissertation", "title": "Studies on storage behaviour of tomatoes coated with chitosan-lysozyme films", "abstract": "Simple technologies are required for reducing the post harvest losses of horticultural produce. Edible films are being studied extensively for application on fresh and cut fruits and vegetables. Tomato, being a very nutritious and important food and a highly perishable climacteric fruit, this study was planned to investigate the application of chitosan films. Chitosan is a biodegradable waste product from sea food and is safe for consumption. With a view to broaden its antimicrobial activity it was combined with lysozyme, a lytic enzyme. Since the edible films are sensitive to changes in temperature and humidity, they were studied under ambient and optimal conditions of storage. This study showed that 1% chitosan was more suitable for tomatoes for storage at both conditions of ambient and low temperature. Respiration study showed that 1% chitosan treatments resulted in more favorable levels of CO 2 production and internal O2. This was reflected in the quality of tomatoes held under these conditions. Two per cent chitosan films were unsuitable due to their high CO2 production and low internal O 2 levels. Spoilage was more apparent in this treatment. Lysozyme addition did not show any additional benefit. The research in this study has demonstrated that the selection of edible films for horticultural produce needs to be integrated with the requirement of storage conditions of the produce.", "field": "Materials Science", "publication_year": 2006, "citations": ["96540192", "135511430", "628752770", "804347849", "1491875823", "1543988247", "1555179716", "1909329750", "1942191010", "1964668323", "1969385044", "1973550038", "1978115062", "1978813157", "1986589335", "1988065641", "1988971376", "1990334932", "1997007503", "1998361448", "2002876130", "2011791131", "2014221899", "2020322655", "2021198456", "2030237720", "2035649622", "2035991985", "2038621830", "2048325428", "2048732892", "2065236790", "2065801599", "2069601971", "2076131153", "2076888186", "2078295833", "2086353422", "2087393294", "2094902926", "2095388819", "2095402235", "2097734000", "2098426135", "2105157072", "2108959254", "2109588582", "2109715889", "2113376388", "2121822945", "2125002907", "2125809273", "2126739759", "2130428456", "2138292215", "2138362454", "2144181560", "2145958489", "2149470780", "2149663276", "2153124349", "2154225810", "2161523807", "2182259680", "2230728210", "2273091895", "2296840117", "2409081401", "2471639665", "2479211581", "2499481382", "2509743902", "2977985009", "3164147305", "3171213794"], "authors": ["Thumula, Padmini."], "embeddings": [0.3269, 0.3374, -0.77148, 0.47046, 0.0466, -0.7627, -0.56543, 0.67041, -0.34155, -0.44165, 0.12042, 0.4978, 0.34033, 0.27881, -0.14807, -0.84912, -1.46387, -0.7373, 0.5083, 0.59033, 0.14685, 0.14758, -0.68164, 0.02087, 0.02808, 1.04199, -0.38599, 1.09277, 0.24133, -0.04782, 0.49146, -0.38037, 0.26172, 0.10822, 0.60205, 0.3103, 0.55762, -0.05157, 0.34033, 0.45166, -0.66406, -0.27563, 0.64404, -0.52148, -0.26685, 0.37695, 0.47974, 0.39331, -0.20203, -0.49292, 0.85742, -1.14355, 0.35791, 0.9375, 0.71387, 0.06879, -0.5249, -0.44995, 0.15454, -0.79102, -0.15588, -0.05237, 0.49072, -0.23608, 1.07715, -0.1731, 0.09747, -0.08307, 0.80713, 1.25781, 0.82666, -1.19238, 0.09222, 1.06348, 0.0802, -0.46216, 0.58008, -0.12225, -0.78467, 0.13806, 0.14197, -0.02887, 0.21631, -0.81543, 0.06842, 0.20605, 0.22803, 0.58594, -0.49951, 0.76221, 0.51367, 0.60938, -0.04868, 0.36865, 0.27295, 1.27148, -0.21375, -0.71338, 0.40454, 0.44971, -0.82861, 0.26978, -0.06537, 0.27734, 1.50195, 0.16821, -0.13403, -0.28857, 0.46094, -0.67578, 0.91406, -0.82861, 0.625, 0.11133, -0.64404, -0.67969, -0.44556, -0.05804, -0.25439, -0.29614, 0.01945, -0.72949, 0.8374, 0.47949, -0.23657, 0.52441, 0.9209, -0.58203, 0.35889, -0.69727, -1.16113, -0.32959, -0.02007, -0.34131, -0.04633, 0.28882, -1.40527, -0.62256, -0.75635, 0.77881, -0.29272, 0.25098, 0.84863, 0.7041, -0.9834, 1.17188, -0.35962, -0.83105, -0.14771, 0.31152, 0.34497, 0.13501, 0.08661, -0.43506, 0.198, -0.68066, 0.08862, 0.69385, -0.43018, -0.26782, 0.27686, 0.30786, 0.68701, 0.15308, -1.79492, 0.94775, 0.55176, 0.04993, -0.01863, -0.60693, -0.55908, 0.56494, -0.34106, 0.29492, 0.04593, -0.88232, 0.42065, 0.03702, -0.45044, -0.25195, -0.81299, 1.01953, 0.13953, -0.87549, 0.93945, -0.33179, -0.30176, 1.00098, -0.72949, -0.63428, 0.40039, 0.31738, 0.8125, -0.8584, -0.90527, -0.00227, -0.85986, -0.09149, -0.20178, 1.46191, -0.49756, 1.22754, -0.15698, -0.43433, -0.40332, -0.61279, -0.34131, 0.11322, 0.10309, 0.35522, -0.25488, 0.49951, 0.13599, -0.09192, -0.06219, 0.81982, -0.28687, -0.16614, 0.27734, 0.65137, 0.53271, 0.37964, 0.7583, -0.7334, -0.78418, -0.10217, -0.80566, 0.66016, 0.96191, -0.20752, 0.77344, -0.30835, 0.51367, 1.17578, 0.64697, -0.29004, 0.26758, 0.79541, -1.38867, -0.75, 0.30981, 0.31323, -0.32153, 0.49194, -0.33667, -0.3479, -0.52344, 0.28833, -0.25977, -0.06537, 0.09631, -0.80713, -0.08112, -0.65527, -0.10248, -0.10211, 0.23132, 0.17944, 0.3894, -0.15869, -0.01365, 0.27368, 0.75879, 0.66602, 0.38281, -0.09869, -0.1626, 0.01138, -0.32275, -0.13049, -0.20605, -0.33667, -0.22888, -0.54102, 0.4707, 0.1853, 0.93848, -0.84814, -0.65576, -0.36743, 1.14648, 0.01691, -0.46338, 0.76123, -1.16602, -0.42505, 0.01479, -0.59082, -0.2644, 0.14197, 0.75635, 0.26807, -0.07825, 0.55029, 0.1991, -1.53418, -0.58838, -0.92627, 0.26001, -0.56055, 0.15088, 0.65723, -0.27539, -0.08368, -0.81006, -0.65234, -0.40454, -0.3103, 1.12402, 0.00864, 0.39038, -0.35596, 0.96924, 0.21582, -0.25562, 0.66797, -0.6665, -0.49854, -0.04547, -0.70508, 0.09698, -0.60986, 0.64844, -0.00205, -0.62305, 0.31616, 0.38818, -0.40381, 0.00315, -0.36548, 0.3269, -0.33789, -0.44263, 0.15173, 0.69873, 0.55859, -0.06058, 0.16406, 0.47119, -0.90967, -0.01257, -0.57568, -0.56885, -0.32544, 0.84326, -0.27393, -0.65625, 0.58838, 0.01923, -0.59521, -0.38916, 0.32715, 0.38525, -0.23083, -1.33594, -1.68945, -0.75537, -0.36914, 0.63525, 0.25366, 0.3186, -0.93896, 0.44556, -0.39087, 0.60645, 0.79199, 0.65088, 0.25732, -0.53271, -0.53418, 0.76562, 0.70361, 0.22705, 1.18652, -0.52393, -0.67432, 0.48071, -0.85742, 0.39844, 0.25146, 0.35156, -0.31519, 0.03799, -0.03769, 1.39941, -0.21472, 0.09369, 0.05814, 0.1543, 0.55615, 0.80127, 0.38379, -0.38794, 0.1416, 0.63477, -0.56543, -0.01622, -0.32373, 0.61963, 1.19629, 0.70801, -1.07812, 0.10925, 0.36475, -0.56885, -0.4436, 0.047, 1.68457, -0.04636, -0.3645, 0.13098, 0.71924, 0.47266, -0.37817, -0.60205, -0.72949, -0.36743, 0.13782, 0.66553, 0.10425, -1.0498, -0.24792, 14.69531, -0.33472, -0.59521, -0.07452, 0.31958, 0.78271, -0.55859, 0.04819, -0.73438, 0.4104, 1.10059, 0.85791, -0.21765, 0.21936, -0.23999, -0.98047, 0.06567, 0.03366, 0.61719, -0.69092, -0.46118, 0.51855, 0.30762, 0.01842, 0.30103, -0.24658, -0.2019, 0.3147, 0.80713, -0.06744, 0.83447, -0.00941, 0.70557, 0.16577, -1.25488, -0.42505, -0.66113, -0.65723, 0.3064, 0.21265, -0.61963, -0.60938, 0.09009, 0.50439, -0.52393, 0.63867, -0.18823, 0.58789, -0.22205, -0.61279, -0.1438, 0.24365, 0.13171, -0.3457, -0.25366, 0.08319, 0.35474, -0.40137, -0.25732, 0.229, -0.95947, -0.62109, -0.05807, 0.32886, -0.73389, 0.56201, 0.05658, 0.62549, -0.23572, -0.073, 0.20093, -0.05405, -0.04205, 0.05728, -0.25928, -0.11969, -0.45752, -0.60889, -0.3186, 0.50195, 0.50146, -0.33667, -1.26074, 0.21423, 0.26929, -0.28223, 1.63574, -0.73828, -0.74023, 0.17957, -0.91113, -0.21204, -0.20581, -0.57373, -0.07184, 0.62549, 1.08984, -0.79248, -1.03711, -0.33154, 0.4187, -0.79199, 0.46167, -0.68311, 0.22888, 0.24011, -0.604, 0.05902, -0.50537, -0.81738, -0.90137, 0.19873, 0.34497, -0.73926, -0.67285, -0.68652, -0.81641, -0.20435, 1.08887, -0.64014, 0.83838, -0.77051, 0.62988, -0.25098, -1.04883, -0.40039, 0.19507, -0.94434, 0.06494, 0.90381, 0.24243, -0.40405, -1.51562, 0.18054, 0.63525, -0.17944, 0.76416, -0.50928, -0.30566, 0.87695, -0.40356, -0.43896, 0.10229, -0.81445, 0.1687, 0.54199, 0.08698, -0.54639, 0.36182, 1.85156, 0.65527, -0.40039, 0.28027, -0.3728, -0.34253, 0.83105, -0.60596, -0.271, 0.08856, -0.21875, 0.35986, 0.7002, 0.2467, -0.08014, 0.16284, -0.34546, 0.39111, 0.42578, -0.1106, -0.55127, -0.67969, -0.51172, -0.05038, -0.73389, -0.63916, -0.43335, -0.35815, -0.37769, 0.70898, -0.30127, -0.23511, 0.08795, -0.44629, 0.04712, 0.59375, 0.01205, -0.24121, -0.07562, 0.01459, 0.10309, 0.56592, -1.06348, -0.65283, -0.99854, 0.2522, -0.43896, -0.49561, -1.2666, 0.84766, 0.19312, -0.13574, -0.22217, 0.63037, -1.4209, 0.93359, 0.44434, -1.1084, -0.13013, -0.35425, 0.20728, 0.13245, -0.35767, 0.55127, -1.1543, 0.42554, -0.24048, -0.7334, -0.74463, 0.86719, -1.32227, -0.74316, 0.65234, -0.86182, 0.01452, 1.16699, 0.46606, -0.1366, -0.99707, 1.14355, 0.28101, -0.48486, -0.28979, -0.00201, 0.57471, -0.77734, 0.55029, -0.02689, -0.30737, -1.35742, 0.27441, -0.09058, -0.09863, 0.29883, 0.89502, -0.29419, 0.01084, 0.81787, -1.39062, -0.59961, -0.3877, 0.20801, 0.41992, -0.34229, 0.43799, -0.45801, 0.80664, 0.54004, 0.12878, 0.60645, -0.56885, 0.01005, 1.17285, 0.0098, 0.20654, -0.87451, 0.24329, -0.12238, -0.44043, -0.23804, -0.44458, -1.10156, 0.34497, 0.78613, 0.68555, 0.52637, -0.18274, 0.18896, 0.59668, 0.32593, -0.43677, 0.55664, -0.3418, 1.74023, -0.49878, 0.81787, -1.11328, -1.27637, 1.31641, 0.62402, -0.23425, 1.04004, -0.55859, 0.43921, -0.13208, -0.74902, 0.31982, -0.30713, 0.93359, 0.79443, 0.55859, -0.21387, 0.27344, 1.01465, 0.34326, 1.2041, 1.17285, -0.44849, -0.57422, -0.02281, 0.21838, -0.84619, 0.32349, -0.77002, 0.76904, -0.14563, 1.16406, 0.3125, -0.27588, 0.63672, -0.17664, -0.29321, -0.37915, -0.92041, 0.78467, 0.22522, -0.03867, -0.9917, -0.54785, 0.59717, 0.29565, -1.0498, 0.32129, 1.26074, -0.60889, 0.4541, -0.27197, -0.15442, -0.65332, -0.29639, -0.58838, -0.79736, -0.31152, -0.37305, -0.05954, -0.41479, -0.11456]}
{"id": "2188378280", "type": "article", "title": "Mechanochemical route to synthesize Co3O4/CuO composite nanopowders", "abstract": "/CuO composite nanopowders have been successfully synthesized by heating the precursor obtained via a mechanochemical reaction. The precursor and the nanopowders were characterized using thermogravimetric (TG), X-ray diffraction (XRD) and transmission electron spectroscopy (TEM). Subsequent thermal treatment of the precursor at 550-700 °C in air for 2 h resulted in the formation of Co", "field": "Materials Science", "publication_year": 2006, "citations": ["1518314399", "1971138078", "1978940862", "1980126325", "1984807562", "1987760332", "1998626888", "2011122973", "2011679767", "2021368575", "2033057955", "2037488779", "2053354951", "2053641943", "2057395445", "2066612550", "2070357240", "2074475022", "2078949651", "2086063792", "2091858186", "2094177561"], "authors": ["Aidong Tang", "HuamingYang", "Xiangchao Zhang"], "embeddings": [0.37964, 0.14478, -0.73975, 1.59766, -0.14282, 0.3938, 0.43994, -0.11719, -0.23218, 0.4314, 0.32764, 0.05078, 0.49731, -0.19873, -0.58057, -0.75439, -0.06247, -0.36792, -0.15723, 0.59229, -0.00067, -0.70215, -1.1543, -0.11462, 0.84229, 1.37598, 0.51514, 0.94971, -0.08606, -0.07709, 1.11328, 0.60791, 0.11487, -0.29517, -0.13708, 0.42505, -0.15295, 0.3208, -0.00606, 0.55225, -0.35962, -0.41406, 0.88086, -0.521, -0.67285, 0.44238, 0.19849, 0.51172, 0.32959, -0.80469, 0.9375, -0.78076, 0.54443, 0.74609, -0.16748, 0.73584, -0.57373, -0.64355, -0.52295, 0.08264, -0.06262, -0.38232, 0.50049, -0.49243, 1.70996, -0.28589, 0.1311, -0.16736, 0.38428, 0.95166, 1.26953, -1.75781, -0.02898, 0.53955, -0.07849, 0.0058, 0.73193, 0.01704, -0.38916, 0.07361, -0.11371, -0.03958, 0.47827, -0.49365, -0.10986, 0.26733, 0.03769, 0.5249, -0.56299, 1.46875, 0.40796, -0.28442, -0.33154, 0.52588, 0.53857, 0.84424, 0.56299, -0.20581, 0.18372, 0.81494, -0.3501, 0.80078, 0.46265, 0.36768, 1.11035, 0.5957, 0.51562, -0.27539, 0.6748, -0.63965, 0.72217, 0.12927, 0.40723, 0.21533, -0.31445, -0.68945, -0.61572, -0.44434, -0.74609, -0.45483, 0.24976, -0.67334, 0.91357, 0.55273, 0.38062, 1.00879, 0.33472, -0.13574, 1.21094, -1.18945, -0.59619, -0.27637, -0.0708, 0.03461, -0.34302, -0.64502, -0.93945, -0.47827, -1.16211, 0.93115, -0.44824, 0.28369, 1.28906, 0.42773, 0.27295, 0.38062, -0.55225, -0.93555, 0.02939, -0.44922, 0.41968, -0.29761, -0.19666, 0.23254, 0.85107, -1.03711, -0.58496, -0.04984, -0.4397, -0.13245, 0.98828, -0.97949, 0.73389, 0.69971, -1.38379, 0.03537, 0.06708, 0.27954, -0.01061, -0.02356, -0.7417, 0.01773, 0.45166, 0.38574, 0.37085, -0.79199, -0.14905, 0.64844, -0.51465, -0.72998, -0.34985, 1.03613, 0.35034, -0.46411, 0.46362, -0.02075, -0.3457, -0.43604, -1.375, -0.79639, 0.3457, 1.1875, 0.61865, -0.17371, -0.9873, 0.42212, -0.0564, -0.33301, -0.24158, 0.93506, -0.84033, 0.22876, -0.34033, 0.70508, -0.48218, -0.27881, 0.24878, -0.06183, 0.13794, 0.85938, -0.31323, 0.2771, -0.02895, -0.25562, -0.61084, 0.43286, -0.52734, -0.4502, 0.19885, 0.91797, 0.25122, -0.14783, 0.70605, -0.36841, -0.50781, 0.46069, -1.13477, 0.81299, -0.32275, -0.46069, 0.8833, -0.99951, -0.00227, 0.90723, 0.69336, -1.06152, 0.7876, 1.03223, -0.74561, -1.2832, 0.17651, 0.6123, -0.42603, 0.73584, 0.27515, -0.12524, 0.15076, -0.24451, 0.73047, -0.26562, 0.146, -0.89258, 0.56494, 0.00194, -0.09296, 0.50391, 0.54297, -0.04013, -0.02312, -0.73096, -0.78076, 0.02542, 0.45947, 1.12402, 0.18433, 0.16174, -0.50928, 0.15979, -0.14856, -0.45312, -0.12854, 0.25244, -0.15308, -1.56836, -0.08215, 0.2301, 1.18945, -0.58447, -0.57568, 0.42188, 0.74951, -0.72314, -0.35352, 1.01562, -0.20886, -1.0293, -0.29468, -0.1803, 0.4209, 0.57812, 0.01729, -0.8335, -1.10156, 0.39185, 0.14221, -1.43359, -1.05273, -0.31689, 0.57422, -0.30347, -0.08673, -0.32373, -0.34277, -0.30273, -0.6626, -0.29736, 0.56348, -0.1076, 0.73291, -0.27637, 0.4436, -0.6167, 1.25488, -0.12415, 0.03195, -0.21814, -0.89502, 0.14575, 0.4895, -0.45972, 0.08453, -0.53027, 0.35986, 0.38477, -0.99316, 0.13708, 0.18347, -0.37158, 0.34912, 0.0625, 0.80322, -0.19128, 0.16602, -0.10284, 0.77979, -0.07349, -0.15247, 0.42212, 0.41577, -1.04199, -0.07294, 0.25684, 0.065, -0.36646, 0.41406, -0.49487, -0.92969, 0.0271, -0.51562, -0.23169, 0.44434, 0.4707, 0.20532, -0.84521, -0.62939, -0.66162, -1.30566, -0.93945, -0.52832, 0.54541, -0.35449, -0.39038, 0.05185, 0.28149, 0.896, 0.51172, 0.89648, 0.70557, -0.18164, -0.42139, -0.22937, 0.3125, -0.08899, 0.52295, -0.2074, 0.50928, -0.40527, -0.65967, 0.71777, 0.46899, 0.3855, -0.1261, -0.2666, -0.05719, 1.57129, -0.03845, 0.33667, 0.68555, 0.97314, 0.07983, -0.06952, 0.2561, 0.49805, 0.42969, 0.59619, -0.11902, -0.35278, -0.01024, 0.03284, 0.85205, -0.35278, 0.81006, -0.70947, 0.59912, -0.64893, -0.06055, 0.20715, 0.46143, 0.03708, -0.3208, 0.10651, 0.66162, 0.72461, -0.2179, -0.10919, -0.81348, 0.16943, 0.74854, 0.80518, 0.84619, -0.70654, 0.39819, 14.60938, 0.66602, -0.12476, 0.37451, -0.52344, 0.69922, -0.12952, 0.2688, -0.79834, 0.09375, 0.37817, 0.1626, 0.15576, 0.65869, -0.15442, -1.2168, 0.19666, 0.47852, 0.07391, -1.57715, -0.16211, 0.5083, 0.84082, 0.35327, -0.11578, -0.06848, 0.09467, -0.06311, 0.51611, 0.00419, 0.84717, 0.12054, 0.12323, -0.12433, -1.60742, -0.40112, -0.15564, -1.12598, 0.03384, 0.66309, -0.31519, -0.24792, 0.21497, -0.28564, 0.26074, 0.62402, -0.46875, -0.02623, -0.12549, -1.00977, -0.21509, 0.50537, 0.26294, -0.43726, -0.23096, -0.24963, -0.19031, 0.29468, -0.1853, -0.16003, -0.6665, -0.10083, 0.14465, 0.92578, 0.25171, 0.2135, -0.0132, -0.42065, -0.21875, -0.13452, -0.03049, -0.93945, 0.41797, -0.02931, -0.36011, -0.19336, -0.68896, -0.44458, -0.29541, 0.34839, 0.1181, -0.20544, -0.49756, 0.07538, -0.42676, 0.07605, 1.39355, -0.86816, -1.00098, -0.21021, -1.26758, -0.27051, 0.17163, -0.59424, -0.85938, 0.24524, 0.97803, -0.18018, -0.95801, -0.69141, 0.37378, -0.18469, -0.3374, -0.25928, -0.26782, 0.28589, -1.03223, -0.75391, 0.25586, -0.12482, -1.25, -0.06198, 0.3894, -0.13647, -0.13428, -0.65625, -0.32935, -0.75928, 1.01465, -0.02347, 0.51416, -0.70557, -0.23279, -0.72949, -0.61523, 0.33521, 0.16223, -1.03027, -0.43335, -0.14856, -0.1156, 0.05469, -0.92188, 0.51025, 1.14551, 0.25391, 0.91992, -0.47559, 1.27832, 0.59131, -0.12219, -0.76953, -0.14404, -0.51904, -0.15039, 0.41382, 0.896, -0.54053, 0.10059, 1.03613, -0.07703, -0.76611, -0.8418, -0.41016, -0.354, -0.04764, 0.02132, -0.03519, 0.58057, -0.15076, 0.53809, 0.39722, 0.39307, -0.16846, 0.46338, 0.23792, 0.12585, -0.64111, -0.21655, -0.4624, -0.13208, -1.40918, -0.12213, -1.05273, -0.26196, -0.35254, -0.13831, -0.13892, 0.04834, -0.2146, -0.88672, -0.47119, -0.46875, 0.69336, 0.62891, -0.40332, -0.99561, -0.77588, -0.30273, -0.20496, 0.61572, -0.2627, 0.46509, -1.12305, -0.30518, -0.08154, -0.46753, -2.07617, 0.56787, -0.04529, -0.13062, 0.43945, -0.48364, -0.65576, 0.07843, 0.70215, -0.08063, -0.65967, -0.84961, -0.1875, 0.27417, 0.56836, 0.35767, -1.29883, -0.21594, -0.15271, -1.29199, -0.3833, 0.81445, -0.55713, -0.9917, 0.53906, -0.56934, 0.16772, -0.27832, 0.83545, -0.07056, -1.70117, 0.68066, -0.17834, 0.40942, -0.42651, 0.28979, 1.19043, -1.02637, 1.31348, -0.29932, -0.09619, -0.39722, 0.90332, 0.22046, 0.45532, 0.04523, -0.01238, -0.48364, -0.28467, -0.12817, -0.7666, -0.40454, -0.104, 0.16992, 0.74414, -0.20789, 0.71582, -0.32153, 0.56445, 0.53369, -0.27856, 0.87695, -0.40186, -0.10956, -0.02922, -0.07245, -0.21118, -0.42651, -0.77881, -0.45117, 0.11877, -0.30981, 0.36475, -0.82031, 0.10968, 0.08435, 0.54004, -0.57031, 0.89941, 0.19885, 0.09357, -0.01342, -1.15527, -0.117, 0.59277, 1.04883, -0.82373, 0.6665, -1.23828, -0.93506, 1.28809, 0.61572, -0.4187, 0.77539, -0.47656, 0.07727, 0.13696, -1.0293, -0.92969, 0.10077, 1.09082, 0.59033, 1.22363, -1.01562, 0.83154, 0.35815, -0.21167, 1.51465, 1.40625, -0.74023, -0.15479, -0.979, 0.66895, -0.98779, 0.53516, 0.03766, 0.15942, 0.19043, 0.59961, 0.30713, -0.46167, 0.47632, 0.08344, 0.52734, -0.26636, 0.38843, 0.72412, 0.25317, 0.01964, -0.84717, -0.29639, -0.02017, 0.21497, -0.64258, 0.32617, 1.23828, 1.04688, 0.63672, 0.37817, 0.28491, -1.00586, 0.33984, -0.8501, -0.16321, -0.86426, -0.52051, -0.82666, -0.21741, -0.94238]}

[lvca]

Fixed in #10550 (tracked in #10543, released in 26.11.1).

What was wrong: when an INT8 or BINARY vector index was larger than the build cache, the graph build kept re-reading vectors from the index pages and converting them to new float arrays. At this scale that kept the heap almost full and the build slowed to a crawl.

What changed: the build now keeps the compact quantized vectors in memory (4x smaller for INT8) and converts them only when needed, without allocating. A build that fits the float cache works as before.

Measured: 1M x 768, 6 GB heap, default settings: 444 s (751 full GCs) -> 196 s (0 full GCs). With a tight heap the difference is larger.

For your setup: the build log now tells you how many vectors are kept in memory. If not all of them are, raise arcadedb.vectorIndex.graphBuildCacheMaxHeapPercent (the log suggests a value) so the whole index stays in memory.

Not covered here: indexes without quantization (float32) still use the old cache, and the slow parts of the import itself (parsing the JSONL embeddings, updating vertex edge pointers) are unchanged.

---

❤️ Thanks for helping make ArcadeDB better! ArcadeDB is free and open source, sustained by its community. If it's useful to you or your company, please consider becoming a sponsor to keep development going.

[warframesam]

@lvca Thanks for the fix.
re-run was successfull. Here are the logs -> arcadedb_FIX_10550.log