Skip to content

doc(server): sync backend-hstore with master - #486

Merged
imbajin merged 1 commit into
apache:masterfrom
bitflicker64:doc-sync/server-backend-hstore
Sep 6, 2026
Merged

doc(server): sync backend-hstore with master#486
imbajin merged 1 commit into
apache:masterfrom
bitflicker64:doc-sync/server-backend-hstore

Conversation

@bitflicker64

@bitflicker64 bitflicker64 commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Adds the missing configuration page for the hstore distributed backend, in English and Chinese. Every fact
traces to the HugeGraph master branch.

The page is not yet linked from the sidebar, because nav registration for all three new backend
pages has to land as a single later change.

page what was wrong what changed source (file:line on master)
config/config-backend-hstore.md (en + cn) Page did not exist. hstore had no configuration page at all, so the backend was only reachable through the PD and Store quickstart pages. New page, weight 6, after the complete configuration manual in the Server Config section. new file
... section 1 Overview Nothing described what the backend is or which Server behavior it switches. Explains that PD owns metadata, schema and schema id counters while Store owns the data, and tabulates the seven behaviors Server switches on the backend type (schema storage, schema ids, system store, task scheduler, auth manager, version check, init-store). hugegraph-hstore/src/main/java/org/apache/hugegraph/backend/store/hstore/HstoreProvider.java:30-53, hugegraph-core/src/main/java/org/apache/hugegraph/StandardHugeGraph.java:266-271,281-285,613-621,664-669,1651-1656, hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/SchemaTransactionV2.java:72,84,648-650, hugegraph-core/src/main/java/org/apache/hugegraph/backend/store/BackendStoreInfo.java:42-48, hugegraph-dist/src/main/java/org/apache/hugegraph/cmd/InitStore.java:113-122
... section 2 Prerequisites Startup order and the ports Server talks to were not written down anywhere in the config section. Documents the PD then Store then Server order, the Up versus Pending store registration rule, and the PD/Store gRPC and REST default ports. Links to the PD and Store quickstart pages. hugegraph-pd/hg-pd-core/src/main/java/org/apache/hugegraph/pd/StoreNodeService.java:160-167,218-220, hugegraph-pd/hg-pd-dist/src/assembly/static/conf/application.yml:38-55, hugegraph-store/hg-store-dist/src/assembly/static/conf/application.yml:32-47
... section 3 Selecting the backend No page said how to turn the backend on, that serializer=binary is needed, how store becomes the graph name PD sees, or that pd.peers is read from the graph config. Documents the four graph keys, the usePD switch in rest-server.properties, the inheritance of pd.peers from the server config, the shipped hstore.properties.template, the hstore Docker image swap and the rocksdb-only Maven profile. conf/graphs/hstore.properties.template:25-33, conf/rest-server.properties:26-27, hugegraph-core/src/main/java/org/apache/hugegraph/config/CoreOptions.java:47-82,649-654,679-685, hugegraph-core/src/main/java/org/apache/hugegraph/backend/store/BackendProviderFactory.java:48-49,61-62,70, hugegraph-core/src/main/java/org/apache/hugegraph/backend/store/BackendStoreProvider.java:28-30, hugegraph-dist/src/main/java/org/apache/hugegraph/dist/RegisterUtil.java:77-126, hugegraph-api/src/main/java/org/apache/hugegraph/core/GraphManager.java:251-260,342-372,1603-1642, hugegraph-server/Dockerfile-hstore:39-41, hugegraph-dist/pom.xml:38,294-304, hugegraph-dist/src/assembly/travis/start-server.sh:72-75
... section 4 hstore options hstore.partition_count and hstore.shard_count were documented nowhere, including in the complete configuration manual. Both keys with default 0 and their descriptions, the validation error for a negative partition count, how PD interprets 0 and clamps a value above its own total, and the fact that no Server code reads hstore.shard_count in this release. hugegraph-hstore/src/main/java/org/apache/hugegraph/backend/store/hstore/HstoreOptions.java:27-38, .../HstoreSessionsImpl.java:126-140, hugegraph-pd/hg-pd-core/src/main/java/org/apache/hugegraph/pd/PartitionService.java:425-445, hugegraph-pd/hg-pd-core/src/main/java/org/apache/hugegraph/pd/config/PDConfig.java:104-107,263-278
... section 5 Options that only apply in hstore mode The keys that only matter with PD were spread across the manual with no indication that they are distributed-mode only. Table of eight keys with file, default, effect and the declaring file:line. Adds the fail-closed rules init-store.sh applies when init_store.enabled is false. hugegraph-api/src/main/java/org/apache/hugegraph/config/ServerOptions.java:187-201,355-361,371-380,390-396,591-598, hugegraph-core/src/main/java/org/apache/hugegraph/config/CoreOptions.java:649-654,679-685, hugegraph-dist/src/main/java/org/apache/hugegraph/cmd/InitStore.java:86-92,113-122,202-238
... section 6 How Server finds the stores The routing path from a key to a store node was undocumented. Describes the one-time PD client and store client setup and the five routing cases the PD-backed partitioner handles, including leader-moved cache invalidation. .../HstoreSessionsImpl.java:109-122, hugegraph-store/hg-store-client/src/main/java/org/apache/hugegraph/store/HgStoreClient.java:92-101, hugegraph-store/hg-store-client/src/main/java/org/apache/hugegraph/store/client/HgStoreNodePartitionerImpl.java:54-193
... section 7 Backend capabilities Nothing listed the query forms the backend does not support, so users hit the limits at runtime. Table of 21 features with yes/no, plus the two reasons the code gives for input-id ordering and property updates being unsupported. hugegraph-hstore/src/main/java/org/apache/hugegraph/backend/store/hstore/HstoreFeatures.java:24-135
... section 8 Verification No way to check that Server actually reached PD and found stores. Documents GET /metrics/backend and what a nodes value of 0 means. hugegraph-api/src/main/java/org/apache/hugegraph/api/metrics/MetricsAPI.java:85,113-134, .../HstoreMetrics.java:37-43, .../HstoreSessionsImpl.java:810-816

Paths above are relative to hugegraph-server/ unless they start with hugegraph-pd/, hugegraph-store/,
docker/ or conf/.

Add config/config-backend-hstore.md in English and Chinese. The hstore
distributed backend had no configuration page: how to select it, the two
hstore.* options with their defaults, the rest-server and graph keys that
only take effect with PD, how the Server routes requests to Stores through
PD, and the query features the backend does not support were all
undocumented.
@bitflicker64
bitflicker64 force-pushed the doc-sync/server-backend-hstore branch from 6c16fe8 to 987fa8c Compare September 5, 2026 19:12

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Summary: The page is carefully sourced and most of it holds up against apache/hugegraph master at 36811483a, but two statements are wrong (the error string quoted for a rocksdb-only build, and nodes: 0 read as proof that PD was reached) and two are misleading as framed (section 7 is presented as a diff against the local backends when 18 of its 21 rows match RocksDBFeatures, and hstore.partition_count is described as applied once when it is re-sent on every Server start). The two hand-maintained section index pages were also not updated. Evidence: both files read at head 987fa8c; claims checked against apache/hugegraph master 36811483a in HstoreFeatures, RocksDBFeatures, BackendFeatures, HstoreProvider, HstoreOptions, HstoreSessionsImpl, HstoreMetrics, BackendProviderFactory, BackendStoreProvider, BackendStoreInfo, StandardHugeGraph, SchemaTransactionV2, RegisterUtil, GraphManager, InitStore, CoreOptions, ServerOptions, PartitionService, PDConfig, StoreNodeService, HgStoreNodePartitionerImpl, hugegraph-dist/pom.xml, backend.properties, hstore.properties.template, Dockerfile-hstore and both application.yml files; and against apache/hugegraph-doc master 493ef79 for themes/docsy, config.toml and content/{en,cn}/docs/config/.

Comment on lines +208 to +235
`hstore` does not support every query form the local backends do. The differences visible to a user:

| Feature | Supported |
|----------------------------|-----------|
| Scan by key prefix | yes |
| Scan by key range | yes |
| Query with range condition | yes |
| Query with order by | yes |
| Query by page | yes |
| OLAP properties | yes |
| Task and server vertex | yes |
| Scan token | no |
| Query schema by name | no |
| Query by label | no |
| Query with `in` condition | no |
| Query with `contains` | no |
| Query with `contains key` | no |
| Sort results by input ids | no |
| Delete edge by label | no |
| Update vertex property | no |
| Update edge property | no |
| Transaction | no |
| Number type | no |
| Aggregate property | no |
| TTL | no |

Sorting by input ids is off because multi node batch scans group the input keys by store and lose the global
order. Vertex and edge property updates are off because the properties are stored in a single cell.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ This table is framed as the differences from the local backends, but 18 of its 21 rows are identical to RocksDBFeatures on apache/hugegraph master (36811483a). Only three entries actually differ:

  • supportsQuerySortByInputIds: rocksdb true, hstore false
  • supportsTransaction: rocksdb true, hstore false
  • supportsTaskAndServerVertex: rocksdb false through the BackendFeatures default, hstore true

Every other no row is false on rocksdb too. RocksDBFeatures.supportsUpdateVertexProperty() even carries the same // Vertex properties are stored in a cell(column value) comment that the closing paragraph attributes to hstore.

The table also omits supportsMergeVertexProperty() (false), listing 21 of the 22 overrides in HstoreFeatures.

Please retitle the section to describe this backend's own capabilities, or add a rocksdb column marking the three real differences. Same at content/cn/docs/config/config-backend-hstore.md:190-219.

Comment on lines +121 to +123
A locally built distribution has the `hstore` provider compiled in by default. The `rocksdb-only` Maven
profile narrows the compiled backend list to `rocksdb`, and a distribution built that way rejects
`backend=hstore` with `Unsupported backend type`.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ The quoted error string is not the one a rocksdb-only build produces.

Unsupported backend type '%s' comes from RegisterUtil.registerBackend(), whose switch only sees the entries of DistOptions.BACKENDS. Under -Drocksdb-only that list is narrowed at hugegraph-dist/pom.xml:302 (<hugegraph.backends>rocksdb</hugegraph.backends>) and reaches the runtime through hugegraph-dist/src/main/resources/backend.properties (backends=[${hugegraph.backends}]), so hstore is never passed to that switch and the message never fires.

BackendProviderFactory.ALLOWED_BACKENDS is a static List.of("memory", "rocksdb", "hbase", "hstore"), so the illegal-backend guard passes and execution reaches:

BackendException.check(clazz != null, "Not exists BackendStoreProvider: %s", backend);

Please quote Not exists BackendStoreProvider: hstore instead, so the string a reader greps for in the server log is the one actually written. Same at content/cn/docs/config/config-backend-hstore.md:115-116.

Comment on lines +246 to +248
The `nodes` value in the response is the count of active stores PD returns. A `nodes` value of `0` means the
Server reached PD but PD has no store in state `Up`, which usually means the Store nodes have not registered
yet, or registered as `Pending` because they are not in PD's `pd.initial-store-list`.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ A nodes value of 0 does not establish that the Server reached PD.

HstoreMetrics.metrics() reports session.getActiveStoreSize(), and that method in HstoreSessionsImpl swallows the failure:

public int getActiveStoreSize() {
    try {
        return defaultPdClient.getActiveStores().size();
    } catch (PDException ignore) {
    }
    return 0;
}

Any PDException produces the same 0: PD down, PD unreachable from the Server, wrong pd.peers, or credentials rejected. As written the page tells a reader to rule out the most likely cause first, and there is no log line at this call site to correct them.

Please add the PD-unreachable case. Same at content/cn/docs/config/config-backend-hstore.md:229-231.

Comment on lines +148 to +149
The number is applied when the store is first registered with PD, so changing it later in the properties file
does not repartition an existing graph.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ The number is re-sent on every Server start, not once at first registration.

infoInitializedGraph, the guard around setGraph(...) in HstoreSessionsImpl.open(), is a private static final Set<String> on the class. It is empty again in a fresh JVM, so each start sends whatever hstore.partition_count currently holds. PartitionService.updateGraph() accepts any value in 1..totalCount and rewrites the graph's partitionCount, logging partition count changed from {} to {}.

That matters because newPartition() derives boundaries from the current value:

int partitionSize = PartitionUtils.MAX_VALUE / graph.getPartitionCount();

Partitions created afterwards use the new count while existing ones keep their old boundaries. Saying it does not repartition an existing graph is true, but it reads as a reassurance that the key is safe to edit.

Please say the value is re-sent at each start and must not change once the graph holds data. Same at content/cn/docs/config/config-backend-hstore.md:140.


- `0`, the default, means let PD decide. For a graph data store PD uses its own cluster wide partition total,
which it derives from the number of entries in `pd.initial-store-list`, `partition.store-max-shard-count`
and `partition.default-shard-count`. For the `/m` and `/s` stores it uses a fixed count of `1`.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 The /s store here contradicts section 1 of this page.

Line 35 states that hstore has no system store, and HstoreProvider.newSystemStore() does return null, so an hstore graph never registers a /s entry with PD. Line 87 correctly lists only /g and /m. The /s branch in PartitionService.updateGraph exists for graphs that do have a system store.

Please drop /s from this sentence, or note that it does not arise for hstore. Same at content/cn/docs/config/config-backend-hstore.md:136.

---
title: "Configuring the HStore Distributed Backend"
linkTitle: "Config HStore Backend"
weight: 6

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 The description says the page is not yet linked from the sidebar, but the linkage actually missing is elsewhere.

Docsy generates the section nav from the page tree: themes/docsy/layouts/partials/sidebar-tree.html:48 takes (union $s.Pages $s.Sections).ByWeight and filters on toc_hide alone. This page sets no toc_hide, so it does appear under Server Config on merge, after Config HTTPS (weight 4).

What does not update itself is content/en/docs/config/_index.md, which lists the section's pages as four hand-written bullets, and its CN counterpart. Both still list four pages after this merges.

Please add a bullet for this page to both _index.md files, and correct that line in the description.

Comment on lines +5 to +9
search_keywords:
- hstore
- pd.peers
- usePD
- hstore.partition_count

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 search_keywords is not read by anything in this repository.

No other markdown page under content/ uses it, layouts/ holds only 404.html, and nothing in themes/docsy or config.toml references it. The only front matter fields docsy reads for its index are title, categories, tags, description and exclude_search (themes/docsy/assets/json/offline-search-index.json). All three search backends are off in config.toml besides: offlineSearch = false, algolia_docsearch = false, and gcs_engine_id commented out. The sibling pages in this section carry title, linkTitle and weight and nothing else.

Please drop the block, or replace it with a description, which docsy does read for the search excerpt and the page meta description. Same at content/cn/docs/config/config-backend-hstore.md:5-9.

Comment on lines +161 to +162
only change behavior, when PD and the `hstore` backend are in use. The source column gives the file and line
on the HugeGraph master branch where the option is declared.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 The source column pins line numbers in another repository's moving branch.

I checked all eight ranges against apache/hugegraph master at 36811483a, the two CoreOptions.java rows included, and every one is correct today. Nothing keeps them correct after the next edit to either file, and this repository has no check that would catch the drift, so the column decays into wrong pointers on a published page.

Please keep the file names and drop the line ranges. Same at content/cn/docs/config/config-backend-hstore.md:151-162.

Comment on lines +82 to +84
- `serializer=binary` is required. Registering the `hstore` backend adds a config space and a store provider
but no serializer of its own, and the adapter is written against the binary serializer. The built-in default
of `serializer` is `text`, so this value has to be written out.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Worth a follow-up: this contradicts the manual in the same section.

The statement here is right, CoreOptions.SERIALIZER declares default text. But content/en/docs/config/config-option.md:74 lists serializer with default value binary, and the CN manual carries the same row at the same line. Once this merges, two pages under Server Config state different defaults for one key, and the manual is the one readers reach for first.

That file is outside this diff, so nothing needs to change here. Please open a follow-up for the serializer row and link it from the description.

Comment on lines +175 to +179
`init-store.sh` never initializes an `hstore` graph. On the enabled path it scans `conf/graphs` and skips
every graph whose backend is `hstore`. If you turn the whole step off with `init_store.enabled=false`, it
validates instead that the admin account can still be created on the PD startup path: `usePD` has to be true,
the auth graph has to exist locally with backend `hstore`, and `auth.admin_pa` has to be set to an explicit
non-empty value. Otherwise startup fails rather than handing out the public default password.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 These three conditions only apply when the built-in local authenticator is configured.

InitStore.checkAdminBootstrapReachable() returns before any of them otherwise:

if (!requiresLocalBuiltinAdmin(conf)) {
    return;
}

The javadoc above it states that remote auth and custom authenticators keep their identities elsewhere and are exempt. As written the paragraph reads as an unconditional requirement, so an operator on remote auth would set usePD, move the auth graph to hstore and configure auth.admin_pa for no reason.

Please add that gate to the sentence. Same at content/cn/docs/config/config-backend-hstore.md:164-167.

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

中文部分与源码保持一致且表述清晰,评分 9.2/10,按规则可直接合并。

@imbajin
imbajin merged commit 2d0547d into apache:master Sep 6, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants