bug: free page can not use again

[yjhjstz]

ivfflatbulkdelete() {
....
/* Set to first free page */
if (!BlockNumberIsValid(insertPage))
insertPage = searchPage;
...
}

Here only set the first free page, other deleted pages can not be used anymore, this caused index grow larger and larger.

any solution? thanks.

[ankane]

Hey @yjhjstz, existing pages with free space are reused (pages aren't deleted) - see InsertTuple function.

[yjhjstz]

yes, but if marked insertPage have N pages, How to manage N free pages ?

[ankane]

insertPage is only a single page. InsertTuple will start at insertPage and follow nextblkno until it finds a page with free space.

[yjhjstz]

I see, but to find the free page to insert must seq scan the list , not efficient.
refer to contrib/bloom

/* Metadata of bloom index */
typedef struct BloomMetaPageData
{
	uint32		magickNumber;
	uint16		nStart;
	uint16		nEnd;
	BloomOptions opts;
	FreeBlockNumberArray notFullPage;
} BloomMetaPageData;

is that ok ?

[ankane]

There are more efficient ways, but I'd like to keep it simple for now. I don't think it'll be an issue for typical usage patterns (in the worst case scenario, it'll end up scanning the list once after a vacuum). Tables and some index types use a free space map, but pgvector needs one for each list (rather than one for the entire index, which isn't supported). Using an array like Bloom indexes would probably work, but it'll add more complexity.

[ankane]

Hey @yjhjstz, just fyi, found a bug in the vacuum code that caused indexes to not reuse space at all. There's a fix in 0.2.4 (and a test case to prevent regressions - it still uses a single insertPage, but there was a logic error).

You can shrink existing indexes with:

-- Postgres 12+
REINDEX INDEX CONCURRENTLY index_name;

-- Postgres < 12
CREATE INDEX CONCURRENTLY temp_name ON table USING ivfflat (column opclass);
DROP INDEX CONCURRENTLY index_name;
ALTER INDEX temp_name RENAME TO index_name;