Mark Hamstra
markhamstra

  • San Francisco, CA
  • Joined on Oct 2, 2010
Aug 21, 2016
markhamstra commented on pull request apache/spark#14737
@markhamstra

NODES, both here and in spark.ui.retainedNodes if far too ambiguous and non-specific for this configuration value -- "node" is already overloaded t…

Aug 21, 2016
markhamstra commented on pull request apache/spark#14737
@markhamstra

This isn't just a question of whether { } is necessary, but also whether using them to delimit closures with map, filter, etc. has become the defac…

Aug 16, 2016
@markhamstra
Aug 15, 2016
@markhamstra
Aug 15, 2016
@markhamstra
Aug 15, 2016
@markhamstra
SKIPME: Clear just the bits that are necessary
1 commit with 48 additions and 2 deletions
Aug 15, 2016
@markhamstra
Aug 15, 2016
markhamstra commented on pull request clearstorydata/spark#173
@markhamstra

I'll take a look at the tests -- essentially you want to run them locally and ignore a couple of known, benign test failures.

Aug 15, 2016
markhamstra commented on pull request apache/spark#14557
@markhamstra

There are multiple issues with this PR. Some are at a more stylistic level, but some include deeper issues -- e.g. see SPARK-17064. Most fundamenta…

Aug 12, 2016
@markhamstra
Aug 12, 2016
@markhamstra
Aug 12, 2016
@markhamstra
bumped snappy
1 commit with 1 addition and 1 deletion
Aug 12, 2016
@markhamstra
bumped snappy
1 commit with 1 addition and 1 deletion
Aug 12, 2016
Aug 11, 2016
@markhamstra
  • @petermaxlee cf93678
    [SPARK-17018][SQL] literals.sql for testing literal parsing
  • @cloud-fan acaf2a8
    [SPARK-17021][SQL] simplify the constructor parameters of QuantileSum…
  • 13 more commits »
Aug 10, 2016
@markhamstra
  • @gatorsmile 0992573
    [SPARK-15901][SQL][TEST] Verification of CONVERT_METASTORE_ORC and CO…
  • @nezihyigitbasi 4df8df5
    [SPARK-15782][YARN] Set spark.jars system property in client mode
  • 643 more commits »
Aug 8, 2016
markhamstra commented on pull request apache/spark#14533
@markhamstra

PR title typo? Intended "misleading"?

Aug 8, 2016
markhamstra commented on pull request apache/spark#14534
@markhamstra

Correct; SynchronizedMap has been deprecated since Scala 2.11.0 with this comment in the API docs: "Synchronization via traits is deprecated as it …

markhamstra created branch slf4j at markhamstra/spark
Aug 2, 2016
Jul 25, 2016
markhamstra commented on pull request apache/spark#14332
@markhamstra

The Scala API docs clearly specify that the most idiomatic usage of Option is to treat it as a collection or monad and use map, flatMap, filter, or…

Jul 25, 2016
markhamstra commented on pull request apache/spark#14332
@markhamstra

It appears that you are reading what you want to see instead of what is really there. All of the examples in the Scala API doc involve side-effects…

Jul 25, 2016
markhamstra commented on pull request apache/spark#14332
@markhamstra

The argument from consistency says to treat Option as a collection or monad, not as something special, and to treat Unit as just another type, not …

Jul 24, 2016
markhamstra commented on pull request apache/spark#14332
@markhamstra

Oh, and the "less obvious" argument is why my opinion on fold with Option was rejected -- even though it is a perfectly logical and obvious thing t…

Jul 24, 2016
markhamstra commented on pull request apache/spark#14332
@markhamstra

There is nothing incorrect about generating an expression of type Unit; what I am talking about is only pattern matching on Options, not the view i…

Jul 24, 2016
markhamstra commented on pull request apache/spark#14332
@markhamstra

I wouldn't change these kinds of constructs. It's almost purely a stylistic issue (for which I don't believe we have an officially declared prefere…

Jul 23, 2016
markhamstra commented on pull request apache/spark#14330
@markhamstra

See JIRA comment.

Jul 13, 2016
@markhamstra
  • @csd-jenkins 0e4dcf6
    [maven-release-plugin] prepare for next development iteration
  • @csd-jenkins 64f5435
    [maven-release-plugin] prepare release spark-parent_2.11-1.6.2-csd-1
  • 2 more commits »
Jul 13, 2016
@markhamstra
Jul 13, 2016
@markhamstra
Default build to Scala 2.11
1 commit with 71 additions and 71 deletions