Closed
Conversation
Implement pushTaskPayload/streamTaskPayload as introduced in apache#14887 for HDFS storage to allow larger mm-less ingestion payloads when using HDFS as the deep storage location.
* Add deprecated com.google.common.io.Files#write to forbiddenApis * Replace deprecated Files.write()
Mistakenly categories under deep storage instead of metadata store.
Changes --------- - Bind `SegmentMetadataCache` only once to `HeapMemorySegmentMetadataCache` in `SQLMetadataStorageDruidModule` - Invoke start and stop of the cache from `DruidOverlord` rather than on lifecycle start/stop - Do not override the binding in `CliOverlord`
…task time (apache#17770) Changes --------- - Use `maxIntervalToKill` to determine search interval for killing unused segments. - If no segment has been killed for the datasource yet, use durationToRetain
…pec was unmodified (apache#17707) Add an optional query parameter called skipRestartIfUnmodified to the /druid/indexer/v1/supervisor endpoint. Callers can set skipRestartIfUnmodified=true to not restart the supervisor if the spec is unchanged. Example: curl -X POST --header "Content-Type: application/json" -d @supervisor.json localhost:8888/druid/indexer/v1/supervisor?skipRestartIfUnmodified=true
Changes --------- - Emit time lag from Kafka similar to Kinesis as metrics `ingest/kafka/lag/time`, `ingest/kafka/maxLag/time`, `ingest/kafka/avgLag/time` - Add new method in `KafkaSupervisor` to fetch timestamps of latest records in stream to compute time lag - Add new field `emitTimeLagMetrics` in `KafkaSupervisorIOConfig` to toggle emission of new metrics
* suggest filter values when known * update snapshots * add more d * fix load rule clamp * better segment timeline init
Changes --------- - Usages of skife config had been deprecated in apache#14695 and `LegacyBrokerParallelMergeConfig` is the last config class that still uses it. - Remove `org.skife.config` from pom, licenses, log4j2.xml, etc. - Add validation for deleted property paths in `StartupInjectorBuilder.PropertiesValidator` - Use the replacement flattened configs (which remove the `.task` and `.pool` substring)
Changes --------- - Add field `taskLimits` to the following worker select strategies `equalDistribution`, `equalDistributionWithCategorySpec`, `fillCapacityWithCategorySpec`, `fillCapacity` - Add sub-fields `maxSlotCountByType` and `maxSlotRatioByType` to `taskLimits` - Apply these limits per worker when assigning new tasks --------- Co-authored-by: sviatahorau <mikhail.sviatahorau@deep.bi> Co-authored-by: Benedict Jin <asdf2014@apache.org> Co-authored-by: Kashif Faraz <kashif.faraz@gmail.com>
* Docs: Add query example * Update after review * Update query * Update docs/api-reference/sql-api.md --------- Co-authored-by: Victoria Lim <vtlim@users.noreply.github.com>
remove usage of dependency:go-offline from build scripts - as it tries to download excluded artifacts --------- Co-authored-by: Zoltan Haindrich <kirk@rxd.hu>
…7790) Currently, query stack traces are logged only when "debug: true" is set in the query context. This patch additionally logs stack traces targeted at the DEVELOPER or OPERATOR personas, because for these personas, stack traces are useful more often than not. We continue to omit stack traces by default for USER and ADMIN, because these personas are meant to interact with the API, not with code or logs. Skipping stack traces minimizes clutter in the logs.
* fix go to task selecting correct task type * support autocompact also * support scheduled_batch, refactor * one more state and update tests
Enables Calcite*Test-s and quidem tests to run queries with Dart.
needed some minor tweaks:
changed to use interfaces at some places
renamed DartWorkerClient to DartWorkerClientImpl and made DartWorkerClient an interface
reused existing parts of the MSQ test system to run the query
* Fix single container config creates failing peon tasks * More obvious array error output
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
…I response (apache#17840) Adds support for an optional filename query parameter to the /druid/v2/sql/statements/{queryId}/results API. When provided, the response will include a header Content-Disposition: attachment; filename="{filename}", which will instruct a web browser to save the response as a file rather than displaying it inline. This save-as-attachment behavior could be achieved by adding a "download" attribute to the results link, but this only works for same-origin URLs (as in the Web Console). If the UI origin is different from the Druid API origin, browsers will ignore the attribute and serve the results inline, which is poor UX for files that are potentially very large. For the sake of consistency, all successful responses in SqlStatementResource.doGetResults may include this header, even if there are no results. Release note Improved: The "Get query results" statements API supports an optional filename query parameter. When provided, the response will instruct web browsers to save the results as a file instead of showing them inline (via the Content-Disposition header).
* set filename * update download button * added markdown support * add test * better download * fix TSV * better download behaviour and tests * always show download all button
…est (apache#17841) Changes: - Fix flakiness in SegmentBootstrapperTest - Make TestSegmentCacheManager thread safe by moving from ArrayList to CopyOnWriteArrayList - Modify assertions to disregard list ordering since order of list modifications is not always deterministic - Fix flaky KinesisIndexTask tests.
…ixing bugs (apache#17844) * better debounce * better cumpose filter * hook up preview filters * better stack handling * fix some props * refactor stack to facet * fix hover part 1 * line hover part 2 * start adding moduleWhere * info popover * add filter icon * toggle button * module filter bar
github-actions
bot
added
Area - Batch Ingestion
Area - Querying
Area - Segment Format and Ser/De
Area - MSQ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #XXXX.
Description
Fixed the bug ...
Renamed the class ...
Added a forbidden-apis entry ...
Release note
Key changed/added classes in this PR
MyFooOurBarTheirBazThis PR has: