Update SECURITY.md#3
Merged
Merged
Conversation
lviiii
pushed a commit
to lviiii/gluten
that referenced
this pull request
Jul 25, 2022
* fix columnar bhj Signed-off-by: Yuan Zhou <yuan.zhou@intel.com> * fix Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
lviiii
pushed a commit
to lviiii/gluten
that referenced
this pull request
Jul 25, 2022
* [ARROW-DATA-SOURCE-1] Reorganize the source code for the new repository organization * [ARROW-DATA-SOURCE-1] Make compiling succeed * [ADS-2] Fix RAM usage CI (apache#3) * Update tpch.yml * Update report_ram_log.yml * Update report_ram_log.yml * Delete github-ci-fix * Delete github-ci-fix2 * [Arrow-Data-Source-4]Add mkdocs.yml and update docs * [ADS-6] Add utility methods to check leaked Allocators/MemoryPools (apache#7) Close apache#6 * [NSE-51] Update ArrowWritableColumnVector * [ADS-9][Parquet] Parquet data source not replaced by default (apache#10) * [ADS-9][Parquet] Parquet data source not replaced by default * Code style * [ADS-13] Validate metric TaskMetrics.peakExecutionMemory for native SQL engine (apache#14) Closes apache#13 * [ADS-11]Modify title check and automatic link to Issues for PRs (apache#12) * [ADS-16] Upgrade Arrow version to 3.0.0 (apache#17) Closes apache#16 * Initialize new repo * Move arrow data source files to arrow-data-source directory * move native sql files to native-sql-engine folder * [NSE-86] Add root pom.xml; Remove native-sql-engine/core/ArrowWritableColumnVector.java (apache#88) * fix github actions Signed-off-by: Yuan Zhou <yuan.zhou@intel.com> * fix building & CI Signed-off-by: Yuan Zhou <yuan.zhou@intel.com> Co-authored-by: Chen Haifeng <haifeng.chen@intel.com> Co-authored-by: zhixingheyi-tian <xiangxiang.shen@intel.com> Co-authored-by: Hongze Zhang <mailtozhz@126.com> Co-authored-by: HongW2019 <hong2.wang@intel.com> Co-authored-by: Hongze Zhang <hongze.zhang@intel.com> Co-authored-by: Rui Mo <rui.mo@intel.com>
liujiayi771
added a commit
to liujiayi771/gluten
that referenced
this pull request
Mar 13, 2023
* support decimal UnscaledValue * rename DecimalUtil to GlutenDecimalUtil
jinchengchenghh
referenced
this pull request
in jinchengchenghh/gluten
Mar 20, 2023
* support decimal UnscaledValue * rename DecimalUtil to GlutenDecimalUtil
jinchengchenghh
referenced
this pull request
in jinchengchenghh/gluten
Mar 21, 2023
* support decimal UnscaledValue * rename DecimalUtil to GlutenDecimalUtil
rui-mo
pushed a commit
to rui-mo/gazelle-jni
that referenced
this pull request
Mar 22, 2023
* support decimal UnscaledValue * rename DecimalUtil to GlutenDecimalUtil
rui-mo
pushed a commit
to rui-mo/gazelle-jni
that referenced
this pull request
Mar 23, 2023
* support decimal UnscaledValue * rename DecimalUtil to GlutenDecimalUtil
SinghAsDev
pushed a commit
to SinghAsDev/gluten
that referenced
this pull request
Mar 5, 2024
## What changes were proposed in this pull request? Adding support for Ubuntu 18 ## How was this patch tested? Ran a query successfully that has been validated on Ubuntu 20 before. No performance regression was observed. <!-- https://github.com/pinternal/gluten/pull/3 -->
This was referenced May 14, 2024
sharkdtu
pushed a commit
to sharkdtu/gluten
that referenced
this pull request
Nov 11, 2024
sharkdtu
pushed a commit
to sharkdtu/gluten
that referenced
this pull request
Nov 11, 2024
j7nhai
pushed a commit
to j7nhai/gluten
that referenced
this pull request
Apr 22, 2025
baibaichen
referenced
this pull request
in baibaichen/gluten
Mar 31, 2026
- #1 SPARK-36286: Fixed in Velox C++ (TimestampConversion.cpp) - "2015-03-18T" now correctly returns null in kSparkCast mode - #2 cast from timestamp II: testGluten rewrite skipping Long.MinValue which overflows in collect() path rebaseGregorianToJulianMicros - #3/#4 timezone: testGluten rewrite syncing session timezone with per-expression timezone via withSQLConf(SESSION_LOCAL_TIMEZONE) Excludes original failing tests in VeloxTestSettings and adds testGluten overrides in GlutenCastWithAnsiOffSuite for both spark40 and spark41. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Mar 31, 2026
- #1/#2 (TZ workaround): testGluten rewrite for "data type casting" and "cast string to timestamp" with withSQLConf to sync session timezone - #3: Add PRESERVE_CHAR_VARCHAR_TYPE_INFO=true in beforeAll - Exclude #4 (out-of-range to long) and #5 (Decimal to string) pending Velox C++ fixes Verified: spark41 95 passed/5 ignored, spark40 79 passed/5 ignored Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Mar 31, 2026
Enable ANSI mode session config and disable ANSI fallback in GlutenCastWithAnsiOnSuite so Velox receives the correct ANSI flag. This fixes: - #3 Casting to char/varchar (now ANSI config propagates correctly) - #5 SPARK-39749: cast Decimal to string (Velox isScientific() reads sparkAnsiEnabled from session config) - #4 out-of-range to long (Velox Conversions.h float→int64 boundary fix, committed separately in Velox upstream) Velox C++ changes (not in this commit, require Velox upstream PR): - SparkCastHooks::isScientific() returns !config_.sparkAnsiEnabled() - Conversions.h: fix float/double→int64 boundary detection UB by comparing in double precision against exact 2^63 boundary Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 9, 2026
- #1 SPARK-36286: Fixed in Velox C++ (TimestampConversion.cpp) - "2015-03-18T" now correctly returns null in kSparkCast mode - #2 cast from timestamp II: testGluten rewrite skipping Long.MinValue which overflows in collect() path rebaseGregorianToJulianMicros - #3/#4 timezone: testGluten rewrite syncing session timezone with per-expression timezone via withSQLConf(SESSION_LOCAL_TIMEZONE) Excludes original failing tests in VeloxTestSettings and adds testGluten overrides in GlutenCastWithAnsiOffSuite for both spark40 and spark41. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 9, 2026
- #1/#2 (TZ workaround): testGluten rewrite for "data type casting" and "cast string to timestamp" with withSQLConf to sync session timezone - #3: Add PRESERVE_CHAR_VARCHAR_TYPE_INFO=true in beforeAll - Exclude #4 (out-of-range to long) and #5 (Decimal to string) pending Velox C++ fixes Verified: spark41 95 passed/5 ignored, spark40 79 passed/5 ignored Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 9, 2026
Enable ANSI mode session config and disable ANSI fallback in GlutenCastWithAnsiOnSuite so Velox receives the correct ANSI flag. This fixes: - #3 Casting to char/varchar (now ANSI config propagates correctly) - #5 SPARK-39749: cast Decimal to string (Velox isScientific() reads sparkAnsiEnabled from session config) - #4 out-of-range to long (Velox Conversions.h float→int64 boundary fix, committed separately in Velox upstream) Velox C++ changes (not in this commit, require Velox upstream PR): - SparkCastHooks::isScientific() returns !config_.sparkAnsiEnabled() - Conversions.h: fix float/double→int64 boundary detection UB by comparing in double precision against exact 2^63 boundary Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 10, 2026
- #1 SPARK-36286: Fixed in Velox C++ (TimestampConversion.cpp) - "2015-03-18T" now correctly returns null in kSparkCast mode - #2 cast from timestamp II: testGluten rewrite skipping Long.MinValue which overflows in collect() path rebaseGregorianToJulianMicros - #3/#4 timezone: testGluten rewrite syncing session timezone with per-expression timezone via withSQLConf(SESSION_LOCAL_TIMEZONE) Excludes original failing tests in VeloxTestSettings and adds testGluten overrides in GlutenCastWithAnsiOffSuite for both spark40 and spark41. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 10, 2026
- #1/#2 (TZ workaround): testGluten rewrite for "data type casting" and "cast string to timestamp" with withSQLConf to sync session timezone - #3: Add PRESERVE_CHAR_VARCHAR_TYPE_INFO=true in beforeAll - Exclude #4 (out-of-range to long) and #5 (Decimal to string) pending Velox C++ fixes Verified: spark41 95 passed/5 ignored, spark40 79 passed/5 ignored Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 10, 2026
Enable ANSI mode session config and disable ANSI fallback in GlutenCastWithAnsiOnSuite so Velox receives the correct ANSI flag. This fixes: - #3 Casting to char/varchar (now ANSI config propagates correctly) - #5 SPARK-39749: cast Decimal to string (Velox isScientific() reads sparkAnsiEnabled from session config) - #4 out-of-range to long (Velox Conversions.h float→int64 boundary fix, committed separately in Velox upstream) Velox C++ changes (not in this commit, require Velox upstream PR): - SparkCastHooks::isScientific() returns !config_.sparkAnsiEnabled() - Conversions.h: fix float/double→int64 boundary detection UB by comparing in double precision against exact 2^63 boundary Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 11, 2026
- #1 SPARK-36286: Fixed in Velox C++ (TimestampConversion.cpp) - "2015-03-18T" now correctly returns null in kSparkCast mode - #2 cast from timestamp II: testGluten rewrite skipping Long.MinValue which overflows in collect() path rebaseGregorianToJulianMicros - #3/#4 timezone: testGluten rewrite syncing session timezone with per-expression timezone via withSQLConf(SESSION_LOCAL_TIMEZONE) Excludes original failing tests in VeloxTestSettings and adds testGluten overrides in GlutenCastWithAnsiOffSuite for both spark40 and spark41. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 11, 2026
- #1/#2 (TZ workaround): testGluten rewrite for "data type casting" and "cast string to timestamp" with withSQLConf to sync session timezone - #3: Add PRESERVE_CHAR_VARCHAR_TYPE_INFO=true in beforeAll - Exclude #4 (out-of-range to long) and #5 (Decimal to string) pending Velox C++ fixes Verified: spark41 95 passed/5 ignored, spark40 79 passed/5 ignored Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 11, 2026
Enable ANSI mode session config and disable ANSI fallback in GlutenCastWithAnsiOnSuite so Velox receives the correct ANSI flag. This fixes: - #3 Casting to char/varchar (now ANSI config propagates correctly) - #5 SPARK-39749: cast Decimal to string (Velox isScientific() reads sparkAnsiEnabled from session config) - #4 out-of-range to long (Velox Conversions.h float→int64 boundary fix, committed separately in Velox upstream) Velox C++ changes (not in this commit, require Velox upstream PR): - SparkCastHooks::isScientific() returns !config_.sparkAnsiEnabled() - Conversions.h: fix float/double→int64 boundary detection UB by comparing in double precision against exact 2^63 boundary Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 12, 2026
- #1 SPARK-36286: Fixed in Velox C++ (TimestampConversion.cpp) - "2015-03-18T" now correctly returns null in kSparkCast mode - #2 cast from timestamp II: testGluten rewrite skipping Long.MinValue which overflows in collect() path rebaseGregorianToJulianMicros - #3/#4 timezone: testGluten rewrite syncing session timezone with per-expression timezone via withSQLConf(SESSION_LOCAL_TIMEZONE) Excludes original failing tests in VeloxTestSettings and adds testGluten overrides in GlutenCastWithAnsiOffSuite for both spark40 and spark41. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 12, 2026
- #1/#2 (TZ workaround): testGluten rewrite for "data type casting" and "cast string to timestamp" with withSQLConf to sync session timezone - #3: Add PRESERVE_CHAR_VARCHAR_TYPE_INFO=true in beforeAll - Exclude #4 (out-of-range to long) and #5 (Decimal to string) pending Velox C++ fixes Verified: spark41 95 passed/5 ignored, spark40 79 passed/5 ignored Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 12, 2026
Enable ANSI mode session config and disable ANSI fallback in GlutenCastWithAnsiOnSuite so Velox receives the correct ANSI flag. This fixes: - #3 Casting to char/varchar (now ANSI config propagates correctly) - #5 SPARK-39749: cast Decimal to string (Velox isScientific() reads sparkAnsiEnabled from session config) - #4 out-of-range to long (Velox Conversions.h float→int64 boundary fix, committed separately in Velox upstream) Velox C++ changes (not in this commit, require Velox upstream PR): - SparkCastHooks::isScientific() returns !config_.sparkAnsiEnabled() - Conversions.h: fix float/double→int64 boundary detection UB by comparing in double precision against exact 2^63 boundary Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 18, 2026
- #1 SPARK-36286: Fixed in Velox C++ (TimestampConversion.cpp) - "2015-03-18T" now correctly returns null in kSparkCast mode - #2 cast from timestamp II: testGluten rewrite skipping Long.MinValue which overflows in collect() path rebaseGregorianToJulianMicros - #3/#4 timezone: testGluten rewrite syncing session timezone with per-expression timezone via withSQLConf(SESSION_LOCAL_TIMEZONE) Excludes original failing tests in VeloxTestSettings and adds testGluten overrides in GlutenCastWithAnsiOffSuite for both spark40 and spark41. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 18, 2026
- #1/#2 (TZ workaround): testGluten rewrite for "data type casting" and "cast string to timestamp" with withSQLConf to sync session timezone - #3: Add PRESERVE_CHAR_VARCHAR_TYPE_INFO=true in beforeAll - Exclude #4 (out-of-range to long) and #5 (Decimal to string) pending Velox C++ fixes Verified: spark41 95 passed/5 ignored, spark40 79 passed/5 ignored Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen
referenced
this pull request
in baibaichen/gluten
Apr 18, 2026
Enable ANSI mode session config and disable ANSI fallback in GlutenCastWithAnsiOnSuite so Velox receives the correct ANSI flag. This fixes: - #3 Casting to char/varchar (now ANSI config propagates correctly) - #5 SPARK-39749: cast Decimal to string (Velox isScientific() reads sparkAnsiEnabled from session config) - #4 out-of-range to long (Velox Conversions.h float→int64 boundary fix, committed separately in Velox upstream) Velox C++ changes (not in this commit, require Velox upstream PR): - SparkCastHooks::isScientific() returns !config_.sparkAnsiEnabled() - Conversions.h: fix float/double→int64 boundary detection UB by comparing in double precision against exact 2^63 boundary Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
yaooqinn
added a commit
to yaooqinn/gluten
that referenced
this pull request
May 13, 2026
Layer 2 mid-review apache#3 BLOCKER (cross-persona consensus: PMC#5 + skeptic SB-1). Velox/Timestamp.h:183 toMicros() = seconds * 1e6 + nanos / 1000 (integer floor). Naive use in PA-6.2.E floored BOTH lo and hi: - floor(lo) is OK: widens prune interval downward, conservative - floor(hi) SHRINKS prune interval -> false-negative drops rows whose true timestamp has nanos%1000 != 0 (Iceberg/Parquet/Arrow source data is naturally ns precision) Fix: floor(lo), ceil(hi). ceil = floor + 1 us when nanos%1000 != 0. RED: CB_1_testTimestampNanosCeilUpperFloorLower — Timestamp(s, 999500), failure: hiMicros=999 want 1000 (toMicros floors away the residue) CB_1_2_testTimestampExactMicrosNoCeil — regression sentinel, Timestamp(s, 1000) exact 1us must NOT over-ceil GREEN: framedSerialize TIMESTAMP arm, +1 us when getNanos() % 1000 != 0. Tests: cpp gtest 15/15 (was 13), JVM 9-suite 38/38 + E2ESuite 11/11 unchanged (JVM 0 changes, wire shape unchanged). refs: todos/features/gluten-inmemory-cache-stats/reviews/batch-pa9-synthesis.md (CB-1)
yaooqinn
added a commit
to yaooqinn/gluten
that referenced
this pull request
May 13, 2026
Layer 2 mid-review apache#3 BLOCKER (cross-persona consensus PMC#2 + skeptic SB-5 + perf#5). Before: cpp emitted full string bytes; JVM truncated to 256B + carry + demote on read. Two sources of truth, JNI wire bloat (6.4MB/partition for 64KB strings), and vanilla parity gap not declared. After: cpp truncates at SOURCE (kStatsStringTruncateLen=256). JVM keeps encodeStringBounds as defense-in-depth for legacy V2 fallback; JVM carry path never triggers in production cpp+JVM path. Wire bytes capped at 256B per bound regardless of source string size. RED: CB_2_testVarcharCppTruncates256B — 300B 'a'/'m', expect lo=256B 'a', hi=255B 'm'+'n' (carry); failure: cpp emitted full 300B + wire layout offset shifted. CB_2_2_testVarcharCppCarryOverflowDemote — 300B 0xFF, expect supported=0; failure: cpp emitted supported=1 + raw bytes. GREEN: VeloxColumnarBatchSerializer.cc computeStats VARCHAR case adds truncate + carry + demote. variant(std::move(...)) avoids redundant string copy. Mirrors JVM PA-9 encodeStringBounds semantics. CB_2.3 regression sentinel: short string ('apple'/'banana') round-trip unchanged (no accidental rewrite of <= 256B inputs). Tests: cpp gtest 18/18 PASS (was 13). JVM 9-suite 38/38 + E2ESuite 11/11 unchanged (JVM 0 changes; defense-in-depth carry path still tested by ColumnarCacheShipBlockerMarshalSuite PA-9 cases). refs: todos/features/gluten-inmemory-cache-stats/reviews/batch-pa9-synthesis.md (CB-2) todos/features/gluten-inmemory-cache-stats/docs/0008-layerA-fulltype-extension.md sec 3.1.a
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.