Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update and document experimental features #13348

Merged
merged 8 commits into from
Nov 29, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/configuration/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -1375,7 +1375,7 @@ For GCE's properties, please refer to the [gce-extensions](../development/extens

This section contains the configuration options for the processes that reside on Data servers (MiddleManagers/Peons and Historicals) in the suggested [three-server configuration](../design/processes.md#server-types).

Configuration options for the experimental [Indexer process](../design/indexer.md) are also provided here.
Configuration options for the [Indexer process](../design/indexer.md) are also provided here.

### MiddleManager and Peons

Expand Down Expand Up @@ -2188,7 +2188,7 @@ Supported query contexts:
|Key|Description|Default|
|---|-----------|-------|
|`druid.expressions.useStrictBooleans`|Controls the behavior of Druid boolean operators and functions, if set to `true` all boolean values will be either a `1` or `0`. See [expression documentation](../misc/math-expr.md#logical-operator-modes)|false|
|`druid.expressions.allowNestedArrays`|If enabled, Druid array expressions can create nested arrays. This is experimental and should be used with caution.|false|
|`druid.expressions.allowNestedArrays`|If enabled, Druid array expressions can create nested arrays.|false|
### Router

#### Router Process Configs
Expand Down
68 changes: 68 additions & 0 deletions docs/development/experimental-features.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
id: experimental-features
title: "Experimental features"
---

<!--
~ Licensed to the Apache Software Foundation (ASF) under one
~ or more contributor license agreements. See the NOTICE file
~ distributed with this work for additional information
~ regarding copyright ownership. The ASF licenses this file
~ to you under the Apache License, Version 2.0 (the
~ "License"); you may not use this file except in compliance
~ with the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing,
~ software distributed under the License is distributed on an
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
~ KIND, either express or implied. See the License for the
~ specific language governing permissions and limitations
~ under the License.
-->

The following features are marked [experimental](./experimental.md) in the Druid docs.

This document includes each page that mentions an experimental feature. To graduate a feature, remove all mentions of its experimental status on all relevant pages.

Note that this document does not track the status of contrib extensions, some of which are experimental.

## SQL-based ingestion

- [SQL-based ingestion](../multi-stage-query/index.md)
- [SQL-based ingestion concepts](../multi-stage-query/concepts.md)
- [SQL-based ingestion and multi-stage query task API](../multi-stage-query/api.md)

## Nested columns

- [Nested columns](../querying/nested-columns.md)

## Indexer process

- [Indexer process](../design/indexer.md)
- [Processes and servers](../design/processes.md#indexer-process-optional)

## Kubernetes

- [Kubernetes](../development/extensions-core/kubernetes.md)
writer-jill marked this conversation as resolved.
Show resolved Hide resolved

## Segment locking

- [Configuration reference](../configuration/index.md#overlord-operations)
- [Task reference](../ingestion/tasks.md#locking)
- [Design](../design/architecture.md#availability-and-consistency)

## Moments sketch

- [Aggregations](../querying/aggregations.md#moments-sketch-experimental)

## Front coding

- [Ingestion spec reference](../ingestion/ingestion-spec.md#front-coding)

## Other configuration properties

- [Configuration reference](../configuration/index.md)
- `CLOSED_SEGMENTS_SINKS` mode
- Expression processing configuration `druid.expressions.allowNestedArrays`
3 changes: 0 additions & 3 deletions docs/development/extensions-core/druid-lookups.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,6 @@ title: "Cached Lookup Module"
~ under the License.
-->


> Please note that this is an experimental module and the development/testing still at early stage. Feel free to try it and give us your feedback.

## Description
This Apache Druid module provides a per-lookup caching mechanism for JDBC data sources.
The main goal of this cache is to speed up the access to a high latency lookup sources and to provide a caching isolation for every lookup source.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,6 @@ This topic contains configuration reference information for the Apache Kafka sup

## Task Autoscaler Properties

> Note that Task AutoScaler is currently designated as experimental.

| Property | Description | Required |
| ------------- | ------------- | ------------- |
| `enableTaskAutoScaler` | Enable or disable autoscaling. `false` or blank disables the `autoScaler` even when `autoScalerConfig` is not null| no (default == false) |
Expand Down
2 changes: 0 additions & 2 deletions docs/development/extensions-core/kinesis-ingestion.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,8 +149,6 @@ Where the file `supervisor-spec.json` contains a Kinesis supervisor spec:

#### Task Autoscaler Properties

> Note that Task AutoScaler is currently designated as experimental.

| Property | Description | Required |
| ------------- | ------------- | ------------- |
| `enableTaskAutoScaler` | Enable or disable the auto scaler. When false or absent, Druid disables the `autoScaler` even when `autoScalerConfig` is not null.| no (default == false) |
Expand Down
4 changes: 2 additions & 2 deletions docs/querying/lookups.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,8 +115,8 @@ will not detect this automatically.

Dynamic Configuration
---------------------
> Dynamic lookup configuration is an [experimental](../development/experimental.md) feature. Static
> configuration is no longer supported.
> Static configuration is no longer supported.

The following documents the behavior of the cluster-wide config which is accessible through the Coordinator.
The configuration is propagated through the concept of "tier" of servers.
A "tier" is defined as a group of services which should receive a set of lookups.
Expand Down