Permalink
Commits on Jun 28, 2018
  1. [SPARK-24408][SQL][DOC] Move abs function to math_funcs group

    jaceklaskowski authored and holdenk committed Jun 28, 2018
    ## What changes were proposed in this pull request?
    
    A few math functions (`abs` , `bitwiseNOT`, `isnan`, `nanvl`) are not in **math_funcs** group. They should really be.
    
    ## How was this patch tested?
    
    Awaiting Jenkins
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #21448 from jaceklaskowski/SPARK-24408-math-funcs-doc.
Commits on Jun 15, 2018
  1. [SPARK-24490][WEBUI] Use WebUI.addStaticHandler in web UIs

    jaceklaskowski authored and vanzin committed Jun 15, 2018
    `WebUI` defines `addStaticHandler` that web UIs don't use (and simply introduce duplication). Let's clean them up and remove duplications.
    
    Local build and waiting for Jenkins
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #21510 from jaceklaskowski/SPARK-24490-Use-WebUI.addStaticHandler.
Commits on Feb 10, 2018
  1. [MINOR][HIVE] Typo fixes

    jaceklaskowski authored and Sean Owen committed Feb 10, 2018
    ## What changes were proposed in this pull request?
    
    Typo fixes (with expanding a Hive property)
    
    ## How was this patch tested?
    
    local build. Awaiting Jenkins
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #20550 from jaceklaskowski/hiveutils-typos.
Commits on Jan 22, 2018
  1. [MINOR] Typo fixes

    jaceklaskowski authored and Sean Owen committed Jan 22, 2018
    ## What changes were proposed in this pull request?
    
    Typo fixes
    
    ## How was this patch tested?
    
    Local build / Doc-only changes
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #20344 from jaceklaskowski/typo-fixes.
Commits on Sep 6, 2017
  1. [SPARK-21901][SS] Define toString for StateOperatorProgress

    jaceklaskowski authored and zsxwing committed Sep 6, 2017
    ## What changes were proposed in this pull request?
    
    Just `StateOperatorProgress.toString` + few formatting fixes
    
    ## How was this patch tested?
    
    Local build. Waiting for OK from Jenkins.
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #19112 from jaceklaskowski/SPARK-21901-StateOperatorProgress-toString.
Commits on Aug 31, 2017
  1. [SPARK-21886][SQL] Use SparkSession.internalCreateDataFrame to create…

    jaceklaskowski authored and gatorsmile committed Aug 31, 2017
    … Dataset with LogicalRDD logical operator
    
    ## What changes were proposed in this pull request?
    
    Reusing `SparkSession.internalCreateDataFrame` wherever possible (to cut dups)
    
    ## How was this patch tested?
    
    Local build and waiting for Jenkins
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #19095 from jaceklaskowski/SPARK-21886-internalCreateDataFrame.
Commits on Jul 7, 2017
  1. [SPARK-21313][SS] ConsoleSink's string representation

    jaceklaskowski authored and Sean Owen committed Jul 7, 2017
    ## What changes were proposed in this pull request?
    
    Add `toString` with options for `ConsoleSink` so it shows nicely in query progress.
    
    **BEFORE**
    
    ```
      "sink" : {
        "description" : "org.apache.spark.sql.execution.streaming.ConsoleSink4b340441"
      }
    ```
    
    **AFTER**
    
    ```
      "sink" : {
        "description" : "ConsoleSink[numRows=10, truncate=false]"
      }
    ```
    
    /cc zsxwing tdas
    
    ## How was this patch tested?
    
    Local build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #18539 from jaceklaskowski/SPARK-21313-ConsoleSink-toString.
  2. [SPARK-21329][SS] Make EventTimeWatermarkExec explicitly UnaryExecNode

    jaceklaskowski authored and zsxwing committed Jul 7, 2017
    ## What changes were proposed in this pull request?
    
    Making EventTimeWatermarkExec explicitly UnaryExecNode
    
    /cc tdas zsxwing
    
    ## How was this patch tested?
    
    Local build.
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #18509 from jaceklaskowski/EventTimeWatermarkExec-UnaryExecNode.
Commits on May 31, 2017
  1. [DOCS][MINOR] Scaladoc fixes (aka typo hunting)

    jaceklaskowski authored and Sean Owen committed May 31, 2017
    ## What changes were proposed in this pull request?
    
    Minor changes to scaladoc
    
    ## How was this patch tested?
    
    Local build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #18074 from jaceklaskowski/scaladoc-fixes.
Commits on May 25, 2017
  1. [SPARK-16202][SQL][DOC] Follow-up to Correct The Description of Creat…

    jaceklaskowski authored and gatorsmile committed May 25, 2017
    …ableRelationProvider's createRelation
    
    ## What changes were proposed in this pull request?
    
    Follow-up to SPARK-16202:
    
    1. Remove the duplication of the meaning of `SaveMode` (as one was in fact missing that had proven that the duplication may be incomplete in the future again)
    
    2. Use standard scaladoc tags
    
    /cc gatorsmile rxin yhuai (as they were involved previously)
    
    ## How was this patch tested?
    
    local build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #18026 from jaceklaskowski/CreatableRelationProvider-SPARK-16202.
Commits on May 11, 2017
  1. [SPARK-20600][SS] KafkaRelation should be pretty printed in web UI

    jaceklaskowski authored and zsxwing committed May 11, 2017
    ## What changes were proposed in this pull request?
    
    User-friendly name of `KafkaRelation` in web UI (under Details for Query).
    
    ### Before
    
    <img width="516" alt="spark-20600-before" src="https://cloud.githubusercontent.com/assets/62313/25841955/74479ac6-34a2-11e7-87fb-d9f62a1356a7.png">
    
    ### After
    
    <img width="439" alt="spark-20600-after" src="https://cloud.githubusercontent.com/assets/62313/25841829/f5335630-34a1-11e7-85a4-afe9b66d73c8.png">
    
    ## How was this patch tested?
    
    Local build
    
    ```
    ./bin/spark-shell --jars ~/.m2/repository/org/apache/spark/spark-sql-kafka-0-10_2.11/2.3.0-SNAPSHOT/spark-sql-kafka-0-10_2.11-2.3.0-SNAPSHOT.jar --packages org.apache.kafka:kafka-clients:0.10.0.1
    ```
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #17917 from jaceklaskowski/SPARK-20600-KafkaRelation-webUI.
Commits on May 7, 2017
  1. [MINOR][SQL][DOCS] Improve unix_timestamp's scaladoc (and typo hunting)

    jaceklaskowski authored and gatorsmile committed May 7, 2017
    ## What changes were proposed in this pull request?
    
    * Docs are consistent (across different `unix_timestamp` variants and their internal expressions)
    * typo hunting
    
    ## How was this patch tested?
    
    local build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #17801 from jaceklaskowski/unix_timestamp.
Commits on Apr 18, 2017
  1. [TEST][MINOR] Replace repartitionBy with distribute in CollapseRepart…

    jaceklaskowski authored and rxin committed Apr 18, 2017
    …itionSuite
    
    ## What changes were proposed in this pull request?
    
    Replace non-existent `repartitionBy` with `distribute` in `CollapseRepartitionSuite`.
    
    ## How was this patch tested?
    
    local build and `catalyst/testOnly *CollapseRepartitionSuite`
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #17657 from jaceklaskowski/CollapseRepartitionSuite.
Commits on Mar 30, 2017
  1. [DOCS] Docs-only improvements

    jaceklaskowski authored and Sean Owen committed Mar 30, 2017
    …adoc
    
    ## What changes were proposed in this pull request?
    
    Use recommended values for row boundaries in Window's scaladoc, i.e. `Window.unboundedPreceding`, `Window.unboundedFollowing`, and `Window.currentRow` (that were introduced in 2.1.0).
    
    ## How was this patch tested?
    
    Local build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #17417 from jaceklaskowski/window-expression-scaladoc.
Commits on Mar 24, 2017
  1. [SQL][MINOR] Fix for typo in Analyzer

    jaceklaskowski authored and gatorsmile committed Mar 24, 2017
    ## What changes were proposed in this pull request?
    
    Fix for typo in Analyzer
    
    ## How was this patch tested?
    
    local build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #17409 from jaceklaskowski/analyzer-typo.
Commits on Mar 18, 2017
  1. [SQL][MINOR] Fix scaladoc for UDFRegistration

    jaceklaskowski authored and rxin committed Mar 18, 2017
    ## What changes were proposed in this pull request?
    
    Fix scaladoc for UDFRegistration
    
    ## How was this patch tested?
    
    local build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #17337 from jaceklaskowski/udfregistration-scaladoc.
Commits on Dec 13, 2016
  1. [MINOR][CORE][SQL] Remove explicit RDD and Partition overrides

    jaceklaskowski authored and Sean Owen committed Dec 13, 2016
    ## What changes were proposed in this pull request?
    
    I **believe** that I _only_ removed duplicated code (that adds nothing but noise). I'm gonna remove the comment after Jenkins has built the changes with no issues and Spark devs has agreed to include the changes.
    
    Remove explicit `RDD` and `Partition` overrides (that turn out code duplication)
    
    ## How was this patch tested?
    
    Local build. Awaiting Jenkins.
    
    …cation)
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #16145 from jaceklaskowski/rdd-overrides-removed.
Commits on Dec 9, 2016
  1. [MINOR][CORE][SQL][DOCS] Typo fixes

    jaceklaskowski authored and Sean Owen committed Dec 9, 2016
    ## What changes were proposed in this pull request?
    
    Typo fixes
    
    ## How was this patch tested?
    
    Local build. Awaiting the official build.
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #16144 from jaceklaskowski/typo-fixes.
Commits on Nov 2, 2016
  1. [SPARK-18204][WEBUI] Remove SparkUI.appUIAddress

    jaceklaskowski authored and Sean Owen committed Nov 2, 2016
    ## What changes were proposed in this pull request?
    
    Removing `appUIAddress` attribute since it is no longer in use.
    ## How was this patch tested?
    
    Local build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #15603 from jaceklaskowski/sparkui-fixes.
Commits on Sep 2, 2016
  1. [SQL][DOC][MINOR] Add (Scala-specific) and (Java-specific)

    jaceklaskowski authored and Sean Owen committed Sep 2, 2016
    ## What changes were proposed in this pull request?
    
    Adds (Scala-specific) and (Java-specific) to Scaladoc.
    
    ## How was this patch tested?
    
    local build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #14891 from jaceklaskowski/scala-specifics.
Commits on Aug 23, 2016
  1. [SPARK-17199] Use CatalystConf.resolver for case-sensitivity comparison

    jaceklaskowski authored and hvanhovell committed Aug 23, 2016
    ## What changes were proposed in this pull request?
    
    Use `CatalystConf.resolver` consistently for case-sensitivity comparison (removed dups).
    
    ## How was this patch tested?
    
    Local build. Waiting for Jenkins to ensure clean build and test.
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #14771 from jaceklaskowski/17199-catalystconf-resolver.
Commits on Jul 22, 2016
  1. [SPARK-16287][HOTFIX][BUILD][SQL] Fix annotation argument needs to be…

    jaceklaskowski authored and Sean Owen committed Jul 22, 2016
    … a constant
    
    ## What changes were proposed in this pull request?
    
    Build fix for [SPARK-16287][SQL] Implement str_to_map SQL function that has introduced this compilation error:
    
    ```
    /Users/jacek/dev/oss/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala:402: error: annotation argument needs to be a constant; found: "_FUNC_(text[, pairDelim, keyValueDelim]) - Creates a map after splitting the text ".+("into key/value pairs using delimiters. ").+("Default delimiters are \',\' for pairDelim and \':\' for keyValueDelim.")
        "into key/value pairs using delimiters. " +
                                                  ^
    ```
    
    ## How was this patch tested?
    
    Local build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #14315 from jaceklaskowski/build-fix-complexTypeCreator.
Commits on Jun 1, 2016
  1. [CORE][DOC][MINOR] typos + links

    jaceklaskowski authored and Andrew Or committed Jun 1, 2016
    ## What changes were proposed in this pull request?
    
    A very tiny change to javadoc (which I don't mind if gets merged with a bigger change). I've just found it annoying and couldn't resist proposing a pull request. Sorry srowen and rxin.
    
    ## How was this patch tested?
    
    Manual build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #13383 from jaceklaskowski/memory-consumer.
  2. [CORE][MINOR][DOC] Removing incorrect scaladoc

    jaceklaskowski authored and Sean Owen committed Jun 1, 2016
    ## What changes were proposed in this pull request?
    
    I don't think the method will ever throw an exception so removing a false comment. Sorry srowen and rxin again -- I simply couldn't resist.
    
    I wholeheartedly support merging the change with a bigger one (and trashing this PR).
    
    ## How was this patch tested?
    
    Manual build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #13384 from jaceklaskowski/blockinfomanager.
Commits on May 5, 2016
  1. [SPARK-15152][DOC][MINOR] Scaladoc and Code style Improvements

    jaceklaskowski authored and Andrew Or committed May 5, 2016
    ## What changes were proposed in this pull request?
    
    Minor doc and code style fixes
    
    ## How was this patch tested?
    
    local build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #12928 from jaceklaskowski/SPARK-15152.
Commits on Apr 26, 2016
  1. [MINOR][DOCS] Minor typo fixes

    jaceklaskowski authored and Sean Owen committed Apr 26, 2016
    ## What changes were proposed in this pull request?
    
    Minor typo fixes (too minor to deserve separate a JIRA)
    
    ## How was this patch tested?
    
    local build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #12469 from jaceklaskowski/minor-typo-fixes.
Commits on Apr 24, 2016
  1. [DOCS][MINOR] Screenshot + minor fixes to improve reading for accumul…

    jaceklaskowski authored and Sean Owen committed Apr 24, 2016
    …ators
    
    ## What changes were proposed in this pull request?
    
    Added screenshot + minor fixes to improve reading
    
    ## How was this patch tested?
    
    Manual
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #12569 from jaceklaskowski/docs-accumulators.
Commits on Apr 8, 2016
  1. [SPARK-14402][HOTFIX] Fix ExpressionDescription annotation

    jaceklaskowski authored and Sean Owen committed Apr 8, 2016
    ## What changes were proposed in this pull request?
    
    Fix for the error introduced in c59abad:
    
    ```
    /Users/jacek/dev/oss/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala:626: error: annotation argument needs to be a constant; found: "_FUNC_(str) - ".+("Returns str, with the first letter of each word in uppercase, all other letters in ").+("lowercase. Words are delimited by white space.")
        "Returns str, with the first letter of each word in uppercase, all other letters in " +
                                                                                              ^
    ```
    
    ## How was this patch tested?
    
    Local build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #12192 from jaceklaskowski/SPARK-14402-HOTFIX.
Commits on Apr 2, 2016
  1. [MINOR] Typo fixes

    jaceklaskowski authored and Sean Owen committed Apr 2, 2016
    ## What changes were proposed in this pull request?
    
    Typo fixes. No functional changes.
    
    ## How was this patch tested?
    
    Built the sources and ran with samples.
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #11802 from jaceklaskowski/typo-fixes.
Commits on Apr 1, 2016
  1. [SPARK-13825][CORE] Upgrade to Scala 2.11.8

    jaceklaskowski authored and Sean Owen committed Apr 1, 2016
    ## What changes were proposed in this pull request?
    
    Upgrade to 2.11.8 (from the current 2.11.7)
    
    ## How was this patch tested?
    
    A manual build
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #11681 from jaceklaskowski/SPARK-13825-scala-2_11_8.
Commits on Feb 1, 2016
  1. Fix for [SPARK-12854][SQL] Implement complex types support in Columna…

    jaceklaskowski authored and rxin committed Feb 1, 2016
    …rBatch
    
    Fixes build for Scala 2.11.
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #10946 from jaceklaskowski/SPARK-12854-fix.
Commits on Jan 23, 2016
  1. [STREAMING][MINOR] Scaladoc + logs

    jaceklaskowski authored and rxin committed Jan 23, 2016
    Found while doing code review
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #10878 from jaceklaskowski/streaming-scaladoc-logs-tiny-fixes.
Commits on Jan 11, 2016
  1. [STREAMING][MINOR] Typo fixes

    jaceklaskowski authored and zsxwing committed Jan 11, 2016
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #10698 from jaceklaskowski/streaming-kafka-typo-fixes.
Commits on Jan 10, 2016
  1. [SPARK-12736][CORE][DEPLOY] Standalone Master cannot be started due t…

    jaceklaskowski authored and Sean Owen committed Jan 10, 2016
    …o NoClassDefFoundError: org/spark-project/guava/collect/Maps
    
    /cc srowen rxin
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #10674 from jaceklaskowski/SPARK-12736.
Commits on Jan 7, 2016
  1. [STREAMING][MINOR] More contextual information in logs + minor code i…

    jaceklaskowski authored and Sean Owen committed Jan 7, 2016
    …mprovements
    
    Please review and merge at your convenience. Thanks!
    
    Author: Jacek Laskowski <jacek@japila.pl>
    
    Closes #10595 from jaceklaskowski/streaming-minor-fixes.