Skip to content

Update SECURITY.md#3

Merged
weiting-chen merged 1 commit into
apache:masterfrom
weiting-chen:security.md
Dec 14, 2021
Merged

Update SECURITY.md#3
weiting-chen merged 1 commit into
apache:masterfrom
weiting-chen:security.md

Conversation

@weiting-chen
Copy link
Copy Markdown
Contributor

No description provided.

@weiting-chen weiting-chen merged commit 0a00431 into apache:master Dec 14, 2021
lviiii pushed a commit to lviiii/gluten that referenced this pull request Jul 25, 2022
* fix columnar bhj

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

* fix

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
lviiii pushed a commit to lviiii/gluten that referenced this pull request Jul 25, 2022
* [ARROW-DATA-SOURCE-1] Reorganize the source code for the new repository organization

* [ARROW-DATA-SOURCE-1] Make compiling succeed

* [ADS-2] Fix RAM usage CI (apache#3)

* Update tpch.yml

* Update report_ram_log.yml

* Update report_ram_log.yml

* Delete github-ci-fix

* Delete github-ci-fix2

* [Arrow-Data-Source-4]Add mkdocs.yml and update docs

* [ADS-6]  Add utility methods to check leaked Allocators/MemoryPools (apache#7)

Close apache#6

* [NSE-51] Update ArrowWritableColumnVector

* [ADS-9][Parquet] Parquet data source not replaced by default (apache#10)

* [ADS-9][Parquet] Parquet data source not replaced by default

* Code style

* [ADS-13] Validate metric TaskMetrics.peakExecutionMemory for native SQL engine (apache#14)

Closes apache#13

* [ADS-11]Modify title check and automatic link to Issues for PRs (apache#12)

* [ADS-16] Upgrade Arrow version to 3.0.0 (apache#17)

Closes apache#16

* Initialize new repo

* Move arrow data source files to arrow-data-source directory

* move native sql files to native-sql-engine folder

* [NSE-86] Add root pom.xml; Remove native-sql-engine/core/ArrowWritableColumnVector.java (apache#88)

* fix github actions

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

* fix building & CI

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Co-authored-by: Chen Haifeng <haifeng.chen@intel.com>
Co-authored-by: zhixingheyi-tian  <xiangxiang.shen@intel.com>
Co-authored-by: Hongze Zhang <mailtozhz@126.com>
Co-authored-by: HongW2019 <hong2.wang@intel.com>
Co-authored-by: Hongze Zhang <hongze.zhang@intel.com>
Co-authored-by: Rui Mo <rui.mo@intel.com>
liujiayi771 added a commit to liujiayi771/gluten that referenced this pull request Mar 13, 2023
* support decimal UnscaledValue

* rename DecimalUtil to GlutenDecimalUtil
jinchengchenghh referenced this pull request in jinchengchenghh/gluten Mar 20, 2023
* support decimal UnscaledValue

* rename DecimalUtil to GlutenDecimalUtil
jinchengchenghh referenced this pull request in jinchengchenghh/gluten Mar 21, 2023
* support decimal UnscaledValue

* rename DecimalUtil to GlutenDecimalUtil
rui-mo pushed a commit to rui-mo/gazelle-jni that referenced this pull request Mar 22, 2023
* support decimal UnscaledValue

* rename DecimalUtil to GlutenDecimalUtil
rui-mo pushed a commit to rui-mo/gazelle-jni that referenced this pull request Mar 23, 2023
* support decimal UnscaledValue

* rename DecimalUtil to GlutenDecimalUtil
SinghAsDev pushed a commit to SinghAsDev/gluten that referenced this pull request Mar 5, 2024
## What changes were proposed in this pull request?

Adding support for Ubuntu 18

## How was this patch tested?

Ran a query successfully that has been validated on Ubuntu 20 before. No
performance regression was observed.

<!-- https://github.com/pinternal/gluten/pull/3 -->
sharkdtu pushed a commit to sharkdtu/gluten that referenced this pull request Nov 11, 2024
sharkdtu pushed a commit to sharkdtu/gluten that referenced this pull request Nov 11, 2024
j7nhai pushed a commit to j7nhai/gluten that referenced this pull request Apr 22, 2025
baibaichen referenced this pull request in baibaichen/gluten Mar 31, 2026
- #1 SPARK-36286: Fixed in Velox C++ (TimestampConversion.cpp) -
  "2015-03-18T" now correctly returns null in kSparkCast mode
- #2 cast from timestamp II: testGluten rewrite skipping Long.MinValue
  which overflows in collect() path rebaseGregorianToJulianMicros
- #3/#4 timezone: testGluten rewrite syncing session timezone with
  per-expression timezone via withSQLConf(SESSION_LOCAL_TIMEZONE)

Excludes original failing tests in VeloxTestSettings and adds
testGluten overrides in GlutenCastWithAnsiOffSuite for both
spark40 and spark41.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Mar 31, 2026
- #1/#2 (TZ workaround): testGluten rewrite for "data type casting" and
  "cast string to timestamp" with withSQLConf to sync session timezone
- #3: Add PRESERVE_CHAR_VARCHAR_TYPE_INFO=true in beforeAll
- Exclude #4 (out-of-range to long) and #5 (Decimal to string) pending
  Velox C++ fixes

Verified: spark41 95 passed/5 ignored, spark40 79 passed/5 ignored

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Mar 31, 2026
Enable ANSI mode session config and disable ANSI fallback in
GlutenCastWithAnsiOnSuite so Velox receives the correct ANSI flag.
This fixes:
- #3 Casting to char/varchar (now ANSI config propagates correctly)
- #5 SPARK-39749: cast Decimal to string (Velox isScientific() reads
  sparkAnsiEnabled from session config)
- #4 out-of-range to long (Velox Conversions.h float→int64 boundary
  fix, committed separately in Velox upstream)

Velox C++ changes (not in this commit, require Velox upstream PR):
- SparkCastHooks::isScientific() returns !config_.sparkAnsiEnabled()
- Conversions.h: fix float/double→int64 boundary detection UB by
  comparing in double precision against exact 2^63 boundary

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 9, 2026
- #1 SPARK-36286: Fixed in Velox C++ (TimestampConversion.cpp) -
  "2015-03-18T" now correctly returns null in kSparkCast mode
- #2 cast from timestamp II: testGluten rewrite skipping Long.MinValue
  which overflows in collect() path rebaseGregorianToJulianMicros
- #3/#4 timezone: testGluten rewrite syncing session timezone with
  per-expression timezone via withSQLConf(SESSION_LOCAL_TIMEZONE)

Excludes original failing tests in VeloxTestSettings and adds
testGluten overrides in GlutenCastWithAnsiOffSuite for both
spark40 and spark41.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 9, 2026
- #1/#2 (TZ workaround): testGluten rewrite for "data type casting" and
  "cast string to timestamp" with withSQLConf to sync session timezone
- #3: Add PRESERVE_CHAR_VARCHAR_TYPE_INFO=true in beforeAll
- Exclude #4 (out-of-range to long) and #5 (Decimal to string) pending
  Velox C++ fixes

Verified: spark41 95 passed/5 ignored, spark40 79 passed/5 ignored

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 9, 2026
Enable ANSI mode session config and disable ANSI fallback in
GlutenCastWithAnsiOnSuite so Velox receives the correct ANSI flag.
This fixes:
- #3 Casting to char/varchar (now ANSI config propagates correctly)
- #5 SPARK-39749: cast Decimal to string (Velox isScientific() reads
  sparkAnsiEnabled from session config)
- #4 out-of-range to long (Velox Conversions.h float→int64 boundary
  fix, committed separately in Velox upstream)

Velox C++ changes (not in this commit, require Velox upstream PR):
- SparkCastHooks::isScientific() returns !config_.sparkAnsiEnabled()
- Conversions.h: fix float/double→int64 boundary detection UB by
  comparing in double precision against exact 2^63 boundary

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 10, 2026
- #1 SPARK-36286: Fixed in Velox C++ (TimestampConversion.cpp) -
  "2015-03-18T" now correctly returns null in kSparkCast mode
- #2 cast from timestamp II: testGluten rewrite skipping Long.MinValue
  which overflows in collect() path rebaseGregorianToJulianMicros
- #3/#4 timezone: testGluten rewrite syncing session timezone with
  per-expression timezone via withSQLConf(SESSION_LOCAL_TIMEZONE)

Excludes original failing tests in VeloxTestSettings and adds
testGluten overrides in GlutenCastWithAnsiOffSuite for both
spark40 and spark41.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 10, 2026
- #1/#2 (TZ workaround): testGluten rewrite for "data type casting" and
  "cast string to timestamp" with withSQLConf to sync session timezone
- #3: Add PRESERVE_CHAR_VARCHAR_TYPE_INFO=true in beforeAll
- Exclude #4 (out-of-range to long) and #5 (Decimal to string) pending
  Velox C++ fixes

Verified: spark41 95 passed/5 ignored, spark40 79 passed/5 ignored

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 10, 2026
Enable ANSI mode session config and disable ANSI fallback in
GlutenCastWithAnsiOnSuite so Velox receives the correct ANSI flag.
This fixes:
- #3 Casting to char/varchar (now ANSI config propagates correctly)
- #5 SPARK-39749: cast Decimal to string (Velox isScientific() reads
  sparkAnsiEnabled from session config)
- #4 out-of-range to long (Velox Conversions.h float→int64 boundary
  fix, committed separately in Velox upstream)

Velox C++ changes (not in this commit, require Velox upstream PR):
- SparkCastHooks::isScientific() returns !config_.sparkAnsiEnabled()
- Conversions.h: fix float/double→int64 boundary detection UB by
  comparing in double precision against exact 2^63 boundary

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 11, 2026
- #1 SPARK-36286: Fixed in Velox C++ (TimestampConversion.cpp) -
  "2015-03-18T" now correctly returns null in kSparkCast mode
- #2 cast from timestamp II: testGluten rewrite skipping Long.MinValue
  which overflows in collect() path rebaseGregorianToJulianMicros
- #3/#4 timezone: testGluten rewrite syncing session timezone with
  per-expression timezone via withSQLConf(SESSION_LOCAL_TIMEZONE)

Excludes original failing tests in VeloxTestSettings and adds
testGluten overrides in GlutenCastWithAnsiOffSuite for both
spark40 and spark41.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 11, 2026
- #1/#2 (TZ workaround): testGluten rewrite for "data type casting" and
  "cast string to timestamp" with withSQLConf to sync session timezone
- #3: Add PRESERVE_CHAR_VARCHAR_TYPE_INFO=true in beforeAll
- Exclude #4 (out-of-range to long) and #5 (Decimal to string) pending
  Velox C++ fixes

Verified: spark41 95 passed/5 ignored, spark40 79 passed/5 ignored

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 11, 2026
Enable ANSI mode session config and disable ANSI fallback in
GlutenCastWithAnsiOnSuite so Velox receives the correct ANSI flag.
This fixes:
- #3 Casting to char/varchar (now ANSI config propagates correctly)
- #5 SPARK-39749: cast Decimal to string (Velox isScientific() reads
  sparkAnsiEnabled from session config)
- #4 out-of-range to long (Velox Conversions.h float→int64 boundary
  fix, committed separately in Velox upstream)

Velox C++ changes (not in this commit, require Velox upstream PR):
- SparkCastHooks::isScientific() returns !config_.sparkAnsiEnabled()
- Conversions.h: fix float/double→int64 boundary detection UB by
  comparing in double precision against exact 2^63 boundary

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 12, 2026
- #1 SPARK-36286: Fixed in Velox C++ (TimestampConversion.cpp) -
  "2015-03-18T" now correctly returns null in kSparkCast mode
- #2 cast from timestamp II: testGluten rewrite skipping Long.MinValue
  which overflows in collect() path rebaseGregorianToJulianMicros
- #3/#4 timezone: testGluten rewrite syncing session timezone with
  per-expression timezone via withSQLConf(SESSION_LOCAL_TIMEZONE)

Excludes original failing tests in VeloxTestSettings and adds
testGluten overrides in GlutenCastWithAnsiOffSuite for both
spark40 and spark41.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 12, 2026
- #1/#2 (TZ workaround): testGluten rewrite for "data type casting" and
  "cast string to timestamp" with withSQLConf to sync session timezone
- #3: Add PRESERVE_CHAR_VARCHAR_TYPE_INFO=true in beforeAll
- Exclude #4 (out-of-range to long) and #5 (Decimal to string) pending
  Velox C++ fixes

Verified: spark41 95 passed/5 ignored, spark40 79 passed/5 ignored

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 12, 2026
Enable ANSI mode session config and disable ANSI fallback in
GlutenCastWithAnsiOnSuite so Velox receives the correct ANSI flag.
This fixes:
- #3 Casting to char/varchar (now ANSI config propagates correctly)
- #5 SPARK-39749: cast Decimal to string (Velox isScientific() reads
  sparkAnsiEnabled from session config)
- #4 out-of-range to long (Velox Conversions.h float→int64 boundary
  fix, committed separately in Velox upstream)

Velox C++ changes (not in this commit, require Velox upstream PR):
- SparkCastHooks::isScientific() returns !config_.sparkAnsiEnabled()
- Conversions.h: fix float/double→int64 boundary detection UB by
  comparing in double precision against exact 2^63 boundary

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 18, 2026
- #1 SPARK-36286: Fixed in Velox C++ (TimestampConversion.cpp) -
  "2015-03-18T" now correctly returns null in kSparkCast mode
- #2 cast from timestamp II: testGluten rewrite skipping Long.MinValue
  which overflows in collect() path rebaseGregorianToJulianMicros
- #3/#4 timezone: testGluten rewrite syncing session timezone with
  per-expression timezone via withSQLConf(SESSION_LOCAL_TIMEZONE)

Excludes original failing tests in VeloxTestSettings and adds
testGluten overrides in GlutenCastWithAnsiOffSuite for both
spark40 and spark41.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 18, 2026
- #1/#2 (TZ workaround): testGluten rewrite for "data type casting" and
  "cast string to timestamp" with withSQLConf to sync session timezone
- #3: Add PRESERVE_CHAR_VARCHAR_TYPE_INFO=true in beforeAll
- Exclude #4 (out-of-range to long) and #5 (Decimal to string) pending
  Velox C++ fixes

Verified: spark41 95 passed/5 ignored, spark40 79 passed/5 ignored

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
baibaichen referenced this pull request in baibaichen/gluten Apr 18, 2026
Enable ANSI mode session config and disable ANSI fallback in
GlutenCastWithAnsiOnSuite so Velox receives the correct ANSI flag.
This fixes:
- #3 Casting to char/varchar (now ANSI config propagates correctly)
- #5 SPARK-39749: cast Decimal to string (Velox isScientific() reads
  sparkAnsiEnabled from session config)
- #4 out-of-range to long (Velox Conversions.h float→int64 boundary
  fix, committed separately in Velox upstream)

Velox C++ changes (not in this commit, require Velox upstream PR):
- SparkCastHooks::isScientific() returns !config_.sparkAnsiEnabled()
- Conversions.h: fix float/double→int64 boundary detection UB by
  comparing in double precision against exact 2^63 boundary

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
yaooqinn added a commit to yaooqinn/gluten that referenced this pull request May 13, 2026
Layer 2 mid-review apache#3 BLOCKER (cross-persona consensus: PMC#5 + skeptic SB-1).

Velox/Timestamp.h:183 toMicros() = seconds * 1e6 + nanos / 1000 (integer
floor). Naive use in PA-6.2.E floored BOTH lo and hi:
  - floor(lo) is OK: widens prune interval downward, conservative
  - floor(hi) SHRINKS prune interval -> false-negative drops rows whose
    true timestamp has nanos%1000 != 0 (Iceberg/Parquet/Arrow source
    data is naturally ns precision)

Fix: floor(lo), ceil(hi). ceil = floor + 1 us when nanos%1000 != 0.

RED:
  CB_1_testTimestampNanosCeilUpperFloorLower — Timestamp(s, 999500),
    failure: hiMicros=999 want 1000 (toMicros floors away the residue)
  CB_1_2_testTimestampExactMicrosNoCeil — regression sentinel,
    Timestamp(s, 1000) exact 1us must NOT over-ceil

GREEN: framedSerialize TIMESTAMP arm, +1 us when getNanos() % 1000 != 0.

Tests: cpp gtest 15/15 (was 13), JVM 9-suite 38/38 + E2ESuite 11/11
unchanged (JVM 0 changes, wire shape unchanged).

refs: todos/features/gluten-inmemory-cache-stats/reviews/batch-pa9-synthesis.md (CB-1)
yaooqinn added a commit to yaooqinn/gluten that referenced this pull request May 13, 2026
Layer 2 mid-review apache#3 BLOCKER (cross-persona consensus PMC#2 + skeptic
SB-5 + perf#5).

Before: cpp emitted full string bytes; JVM truncated to 256B + carry +
demote on read. Two sources of truth, JNI wire bloat (6.4MB/partition
for 64KB strings), and vanilla parity gap not declared.

After: cpp truncates at SOURCE (kStatsStringTruncateLen=256). JVM keeps
encodeStringBounds as defense-in-depth for legacy V2 fallback; JVM
carry path never triggers in production cpp+JVM path. Wire bytes
capped at 256B per bound regardless of source string size.

RED:
  CB_2_testVarcharCppTruncates256B — 300B 'a'/'m', expect lo=256B 'a',
    hi=255B 'm'+'n' (carry); failure: cpp emitted full 300B + wire
    layout offset shifted.
  CB_2_2_testVarcharCppCarryOverflowDemote — 300B 0xFF, expect
    supported=0; failure: cpp emitted supported=1 + raw bytes.

GREEN: VeloxColumnarBatchSerializer.cc computeStats VARCHAR case adds
  truncate + carry + demote. variant(std::move(...)) avoids redundant
  string copy. Mirrors JVM PA-9 encodeStringBounds semantics.

CB_2.3 regression sentinel: short string ('apple'/'banana') round-trip
  unchanged (no accidental rewrite of <= 256B inputs).

Tests: cpp gtest 18/18 PASS (was 13). JVM 9-suite 38/38 + E2ESuite 11/11
unchanged (JVM 0 changes; defense-in-depth carry path still tested by
ColumnarCacheShipBlockerMarshalSuite PA-9 cases).

refs: todos/features/gluten-inmemory-cache-stats/reviews/batch-pa9-synthesis.md (CB-2)
      todos/features/gluten-inmemory-cache-stats/docs/0008-layerA-fulltype-extension.md sec 3.1.a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant