Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Fetching contributors…

Cannot retrieve contributors at this time

file 571 lines (365 sloc) 19.006 kb
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571
Giraph Change Log

Release 0.2.0 - unreleased

  GIRAPH-341: Improved log messages (timing) and upgraded junit to 4.8
  for better tests. (aching)
  
  GIRAPH-343: Use published hcatalog jars. (nitayj via aching)

  GIRAPH-338: More Rat Ignores (Nitay Joffe via ereisman)

  GIRAPH-347: GiraphConfiguration broke hcatalog build
  (Nitay Joffe via ereisman)

  GIRAPH-340: Added client/server ExecutionHandlers to Netty to avoid
  and added WrappedAdaptiveReceiveBufferSizePredictorFactory to
  debug/predict the size of the incoming messages. (aching)

  GIRAPH-274: Jobs still failing due to tasks timeout during
  INPUT_SUPERSTEP. (nitayj via aching)

  GIRAPH-337: Make a specific Giraph configuration for Class caching
  and specific Giraph configuration. (aching)

  GIRAPH-334: Bugfix HCatalog Hive profile. (nitayj via aching)

  GIRAPH-93: Hive input / output format. (nitayj via aching)

  GIRAPH-277: Text Vertex Input/Output Format base classes overhaul.
  (nitayj via aching)

  GIRAPH-331: ReviewBoard post-review config. (nitayj via aching)

  GIRAPH-332: Duplicate unnecessary info in giraph-formats-contrib
  compile.xml. (nitay via aching)

  GIRAPH-330: Ignores file for Git. (nitay via aching)

  GIRAPH-327: Timesout values in BspServiceMaster.barrierOnWorkerList
  (majakabiljo via ereisman)

  GIRAPH-323: Check if requests are done before calling wait (majakabiljo
  via ereisman)

  GIRAPH-298: Reduce timeout for TestAutoCheckpoint. (majakabiljo via
  aching)

  GIRAPH-324: Add option to use combiner in benchmarks. (apresta via
  aching)

  GIRAPH-191: Random walks on graphs (Gianmarco De Francisci Morales
  via ereisman)

  GIRAPH-320: Provide a runtime configuration for choosing the
  log level (aching via ereisman)

  GIRAPH-321: Divide by 0 exception. (ereisman via aching)

  GIRAPH-316: Add test-patch.sh for precommit test using Jenkins.
  (hyunsik via ereisman)

  GIRAPH-319: Receiving two responses for a request causes an
  exception. (apresta via aching)

  GIRAPH-291: PredicateLock should have a constructor to take in a
  custom waiting time and additional testing (aching via ereisman)

  GIRAPH-318: New Iterator in LocalityInfoSorter is not working.
  (Eli Reisman via apresta)

  GIRAPH-317: Add subpackages to comm (Maja Kabiljo via ereisman)

  GIRAPH-301: InputSplit Reservations are clumping, leaving many workers
  asleep while other process too many splits and get overloaded.
  (Eli Reisman via apresta)

  GIRAPH-313: Open Netty client and server on master. (majakabiljo via
  aching)

  GIRAPH-249: Move part of the graph out-of-core when memory is low
  (apresta via aching).

  GIRAPH-306: Netty requests should be reliable and implement exactly
  once semantics. (aching)

  GIRAPH-309: Message count is wrong. (aching via apresta)

  GIRAPH-246: Periodic worker calls to context.progress() will prevent
  timeout on some Hadoop clusters during barrier waits. (Eli Reisman
  via aching)

  GIRAPH-295: Additional Example Algorithm to compute Outdegree and
  Indegree. (Sean Choi via aching)

  GIRAPH-305: Adding an argument to GiraphRunner for Master Compute
  classes. (Sean Choi via aching)

  GIRAPH-302: Thread safety issue with sending partitions around.
  (aching via apresta)

  GIRAPH-303: Regression: cleanup phase happens earlier than it
  should. (majakabiljo via apresta)

  GIRAPH-278: Website still tries to load incubator logo (ekoontz)

  GIRAPH-300) Improve netty reliability with retrying failed
  connections, tracking requests, thread-safe hash partitioning
  (aching via apresta).

  GIRAPH-296: TotalNumVertices and TotalNumEdges are not saved in
  checkpoint. (majakabiljo via apresta)

  GIRAPH-297: Checkpointing on master is done one superstep later
  (majakabiljo via aching).

  GIRAPH-275: Restore data locality to workers reading InputSplits
  where possible without querying NameNode, ZooKeeper. (Eli Reisman
  via jghoman)
  
  GIRAPH-258: Check type compatibility before submitting job.
  (Eli Reisman via jghoman)

  GIRAPH-218: Consolidate all I/O Format classes under one roof in
  lib/ directory. (Eli Reisman via jghoman)

  GIRAPH-259: TestBspBasic.testBspPageRank is broken (majakabiljo via
  apresta)

  GIRAPH-256: Partitioning outgoing graph data during INPUT_SUPERSTEP
  by # of vertices results in wide variance in RPC message sizes. (Eli
  Reisman via jghoman)
  
  GIRAPH-290: Add committer information for Alessandro Presta to pom.xml
  (apresta)

  GIRAPH-286. Remove DISCLAIMER from source tree. (jghoman)

  GIRAPH-287: Add option to limit the number of open requests.
  (Maja Kabiljo via jghoman)

  GIRAPH-262: Netty optimization to handle requests locally whenever
  possible. (aching)

  GIRAPH-288: Bandwidth tracking - subset of GIRAPH-262. (aching)

  GIRAPH-289: Add thread and channel pooling to NettyClient and
  NettyServer. (ekoontz via aching)

  GIRAPH-276: Fix broken tests in pseudo-distributed mode.
  (Alessandro Presta via jghoman)

  GIRAPH-281: Add options to control Netty's per-channel receive and
  send buffer sizes (ekoontz via aching).

  GIRAPH-228: SimpleTriangleClosingVertex should not use ArrayWritable
  for a vertex value. (Eli Reisman via jghoman)

  GIRAPH-209: Include munge version in artifact name.
  (Eli Reisman via jghoman)
  
  GIRAPH-280: Add IntelliJ-generated *.iml and *.ipr files to Apache
  Rat's <exclude> list. (ekoontz via aching).

  GIRAPH-45: Improve the way to keep outgoing messages (majakabiljo
  via aching).
 
  GIRAPH-271: Regression in imports in CommunicationsInterface (netj
  via aching).

  GIRAPH-267: Jobs can get killed for not reporting status during
  INPUT SUPERSTEP (netj via aching).

  GIRAPH-266: Average aggregators don't calculate real average
  (majakabiljo via aching).

  GIRAPH-244: Vertex API redesign (apresta via aching).

  GIRAPH-236: Add FindBugs to maven build (Jan van der Lugt via
  aching).

  GIRAPH-224: Netty server-side combiner (apresta via aching).

  GIRAPH-251: Allow to access the distributed cache from Vertexes and
  WorkerContext (Gianmarco De Francisci Morales via aching).

  GIRAPH-261: Rename isQuiet variable. (Gianmarco De Francisci Morales
  via jghoman).

  GIRAPH-248: Generic IdentityVertex for IO testing (Sean Choi via
  aching).

  GIRAPH-222: GIRAPH-222 giraph-formats-contrib needs a README (bfem
  via aching).

  GIRAPH-257: TestBspBasic.testBspMasterCompute is broken (majakabiljo
  via aching).

  GIRAPH-81: Create annotations on provided algorithms for cli
  (majakabiljo via aching).

  GIRAPH-242: HashMapVertex stores neighbor ids twice.
  (Alessandro Presta via hyunsik)

  GIRAPH-241: Small typos in var names in NettyWorkerClient.java
  (Eli Reisman via hyunsik)

  GIRAPH-239: IntIntNullIntVertex doesn't save halted state (apresta
  via aching)

  GIRAPH-238: BasicVertex should have default Writable implementation
  (apresta via aching)

  GIRAPH-233: Small errors found by FindBugs (Jan van der Lugt via hyunsik)
  
  GIRAPH-216: NullWritable as VertexData, EdgeData or MessageData
  should be allowed. (Jan van der Lugt via jghoman)

  GIRAPH-221: Make iteration over edges more explicit (apresta via aching).

  GIRAPH-225: Guava version in POM.XML is really old. Updated to version 12.0.
  (Eli Reisman via hyunsik)

  GIRAPH-223: Need to put Giraph jar on classpath, post-GIRAPH-205.
  (Eli Reisman via jghoman)

  GIRAPH-213: NettyClient.stop() could deadlock according to netty.io docs.
  (Eli Reisman via jghoman)

  GIRAPH-127: Extending the API with a master.compute() function.
  (Jan van der Lugt via jghoman)

  GIRAPH-220: Default implementation of BasicVertex#sendMsgToAllEdges().
  (Alessandro Presta via jghoman)

  GIRAPH-217: Add SimpleTriangleClosingVertex to Giraph examples.
  (Eli Reisman via jghoman)

  GIRAPH-219: pom in giraph-formats-contrib should have groupId
  'org.apache.giraph'. (Brian Femiano via jghoman)

  GIRAPH-215: Update site to use Giraph logo and remove 'incubator'
  text (ekoontz)

  GIRAPH-205: Move Giraph jar to root level of tar.gz.
  (Roman Shaposhnik via jghoman)
  
  GIRAPH-206: Break out SimpleShortestPathVertex. (Eli Reisman via jghoman)

  GIRAPH-210: Hadoop 1.0 profile has no activation. (jghoman)
 
  GIRAPH-192: Move aggregators to a separate sub-package.
  (Jan van der Lugt via jghoman)
  
  GIRAPH-208: LocalTestMode's zookeeper directory is not being cleaned up
  after job runs (ekoontz)

  GIRAPH-194: Fix up URLs in the pom. (omalley)

  GIRAPH-153: HBase/Accumulo Input and Output formats. (bfem via aching)

  GIRAPH-187: SequenceFileVertexInputFormat has WritableComparable<I>
  as a bounded type for I. (roman4asf via aching)

  GIRAPH-20: Move temporary test files from the project directory. (ssc)

  GIRAPH-37: Implement Netty-backed IPC. (aching)

  GIRAPH-184: Upgrade to junit4. (Devaraj K via jghoman)

  GIRAPH-176: BasicRPCCommunications has unnecessary cast of Vertex.
  (Devaraj K via jghoman)

  GIRAPH-175: Replace manual array copy to utility method call.
  (Devaraj K via jghoman)

  GIRAPH-181: Add Hadoop 1.0 profile to pom.xml. (ekoontz via aching)

  GIRAPH-183: Add Claudio's FOSDEM presentation (slides and video)
  to the site. (claudio)

  GIRAPH-179: BspServiceMaster's PathFilter can be simplified.
  (Devaraj K via jghoman)

  GIRAPH-177: SimplePageRankVertex has two redundant casts.
  (Devaraj K via jghoman)

  GIRAPH-168: Simplify munge directive usage with new munge flag
  HADOOP_SECURE (rather than HADOOP_FACEBOOK) and remove usage of
  HADOOP (ekoontz via aching).

  GIRAPH-85: Simplify return expression in
  RPCCommunications::getRPCProxy (Eli Reisman via jghoman)

  GIRAPH-171: Total time in MasterThread.run() is calculated
  incorrectly (ekoontz via aching).

  GIRAPH-144: GiraphJob should not extend Job (users should not be
  able to call Job methods like waitForCompletion or setMapper..etc)
  (aching).

  GIRAPH-159: Case insensitive file/directory name matching will
  produce errors on M/R jar unpack (bfem via aching).

  GIRAPH-166: add '*.patch' to list of files that Apache Rat ignores
  (ekoontz via aching).

  GIRAPH-167: mvn -Phadoop_non_secure clean verify fails (ekoontz via
  aching).

  GIRAPH-163: bin/giraph script overwrites CLASSPATH if "dev
  environment" detected (this also removes USER_JAR from CLASSPATH)
  (metaman via aching).

  GIRAPH-164: fix 5 "Line is longer than 80 characters" style errors
  in GiraphRunner (ekoontz via aching).

  GIRAPH-162: BspCase.setup() should catch FileNotFoundException
  thrown from org.apache.hadoop.fs.FileSystem.listStatus() (ekoontz
  via aching).

  GIRAPH-161: Handling null messages and edges when initializing
  IntIntNullIntVertex (dlogothetis via aching).

  GIRAPH-156: Users should be able to set simple 'custom arguments'
  via org.apache.giraph.GiraphRunner (ssc)

  GIRAPH-154: Worker ports are not synched properly with its peers
  (Zhiwei Gu via aching).

  GIRAPH-87: Simplify boolean expression in
  BspService::checkpointFrequencyMet (Eli Reisman via aching).

  GIRAPH-150: PageRankBenchmark accesses wrong conf after GiraphJob is
  created (aching).
 
  GIRAPH-40: Added checkstyle for enforcement of code conventions.
  All Giraph source files now pass checkstyle. (aching)

  GIRAPH-148: giraph-site.xml needs Apache header. (jghoman)
  
  GIRAPH-139: Change PageRankBenchmark to be accessible via bin/giraph.
  (jghoman)
  
  GIRAPH-143: Add support for giraph to have a conf file. (jghoman)

  GIRAPH-142: _hadoopBsp should be prefixable via configuration. (jghoman)

  GIRAPH-145. Change partition request log level to debug rather than info.
  (jghoman)

  GIRAPH-130: Fix Javadoc warnings. (Harsh J. Chouraria via jghoman)

  GIRAPH-137: De-duplicate pagerank implementation in PageRankBenchmark.
  (Harsh J. Chouraria via jghoman)
 
  GIRAPH-133: Typo in JavaDoc in BspCase::remove().
  (Harsh J. Chouraria via jghoman)
  
  GIRAPH-136: Error message for bin/graph could be improved. (jghoman)

Release 0.1.0 - 2012-01-31

  GIRAPH-120: Add Sebastian Schelter to site. (ssc)

  GIRAPH-117: DefaultWorkerContext should preserve the method signatures
  of WorkerContext. (ssc)

  GIRAPH-135: Need DISCLAIMER for incubator. (jghoman)

  GIRAPH-134: Fix NOTICE file for release. (jghoman)

  GIRAPH-128: RPC port from BasicRPCCommunications should be only a
  starting port, and retried. (aching)

  GIRAPH-131: Enable creation of test-jars to simplify testing in
  downstream projects. (André Kelpe via jghoman)
  
  GIRAPH-129: Enable creation of javadoc and sources jars.
  (André Kelpe via jghoman)
  
  GIRAPH-124: Combiner should return Iterable<M> instead of M or
  null. (claudio)

  GIRAPH-125: Bug in LongDoubleFloatDoubleVertex.sendMsgToAllEdges().
  (humming80 via aching)

  GIRAPH-122: Roll version back to 0.1. (jghoman)

  GIRAPH-118: Clarify messages behavior in BasicVertex. (claudio)

  GIRAPH-119: VertexCombiner should work on Iterable<M> instead of
  List<M>. (claudio)

  GIRAPH-116: Make EdgeListVertex the default vertex implementation,
  fix bugs related to EdgeListVertex. (aching)

  GIRAPH-115: Port of the HCC algorithm for identifying all connected
  components of a graph. (ssc via aching)

  GIRAPH-112: Use elements() properly in LongDoubleFloatDoubleVertex.
  (aching)

  GIRAPH-114: Inconsistent message map handling in
  BasicRPCCommunications.LargeMessageFlushExecutor. (ssc via aching)

  GIRAPH-109: GiraphRunner should provide support for combiners.
  (ssc via claudio)

  GIRAPH-113: Change cast to Vertex used in prepareSuperstep() to
  BasicVertex. (humming80 via aching)

  GIRAPH-110: Add guide to setup the enviroment for running the
  unittests in a pseudo-distributed hadoop instance. (ssc via aching)

  GIRAPH-73: A little refactoring. (ssc via aching)

  GIRAPH-106: Change prepareSuperstep() to make
  setMessages(Iterable<M> messages) package-private. (aching)
 
  GIRAPH-105: BspServiceMaster.checkWorkers() should return empty
  lists instead of null. (ssc via aching)

  GIRAPH-80: Don't expose the list holding the messages in
  BasicVertex. (ssc via aching)

  GIRAPH-103: Added properties for commonly used package version to
  pom.xml. (aching)

  GIRAPH-57: Add new RPC call (putVertexIdMessagesList) to batch
  putMsgList RPCs together. (aching)

  GIRAPH-104: Save half of maximum memory used from messaging. (aching)

  GIRAPH-10: Aggregators are not exported. (claudio)

  GIRAPH-100: GIRAPH-100 - Data input sampling and testing
  improvements. (aching)

  GIRAPH-51: Provide unit testing tool for Giraph algorithms.
  (Sebastian Schelter via jghoman)

  GIRAPH-89: Simplify boolean expressions in BspRecordReader.
  (shaunak via claudio)

  GIRAPH-90: LongDoubleFloatDoubleVertex has possibily the iterator()
  implementation broken (claudio)

  GIRAPH-99: Make AdjacencyListVertexReader and its constructor public.
  (Kohei Ozaki via jghoman)

  GIRAPH-98: Add Claudio Martella to site. (claudio)

  GIRAPH-97: TestIdWithValueTextOutputFormat.java and
  IdWithValueTextOutputFormat.java missing license header (claudio)

  GIRAPH-92: Need outputformat for just vertex ID and value. (jghoman)

  GIRAPH-86: Simplify boolean expressions in ZooKeeperExt::createExt.
  (attilacsordas via jghoman)

  GIRAPH-91: Large-memory improvements (Memory reduced vertex
  implementation, fast failure, added settings). (aching)

  GIRAPH-89: Remove debugging system.out from LongDoubleFloatDoubleVertex.
  (shaunak via aching)

  GIRAPH-88: Message count not updated properly after GIRAPH-11. (aching)

  GIRAPH-70: Misspellings in PseudoRandomVertexInputFormat configuration
  parameters. (attilacsordas via jghoman)

  GIRAPH-58: Update site with Arun's id (asuresh)
  
  GIRAPH-11: Improve the graph distribution of Giraph. (aching)
  
  GIRAPH-64: Create VertexRunner to make it easier to run users'
  computations. (jghoman)
 
  GIRAPH-79: Change the menu layout of the site. (hyunsik via jghoman)

  GIRAPH-75: Create sections on how to get involved and how
  to generate patches on website. (jghoman)

  GIRAPH-63: Typo in PageRankBenchmark. (shaunak via jghoman)

  GIRAPH-47: Export Worker's Context/State to vertices through
  pre/post/Application/Superstep. (cmartella via aching)

  GIRAPH-71: SequenceFileVertexInputFormat missing license header;
  rat fails. (jghoman)

  GIRAPH-36: Ensure that subclassing BasicVertex is possible by user
  apps. (jmannix via aching)

  GIRAPH-50: Require Maven 3 in order to work with munging plugin.
  (jghoman)
  
  GIRAPH-67: Provide AdjacencyList InputFormat for Ids of Strings and
  double values. (jghoman)

  GIRAPH-56: Create a CSV TextOutputFormat. (jghoman)

  GIRAPH-66: Add presentations section to website. (jghoman)

  GIRAPH-62: Provide input format for reading graphs stored as adjacency
  lists. (jghoman)

  GIRAPH-59: Missing some test if debug enabled before LOG.debug() and
  LOG.info(). (guzhiwei via aching)

  GIRAPH-48: numFlushThreads is 0 when doing a single worker
  unittest. Changing the minimum to 1. (aching)

  GIRAPH-44: Add documentation about counter limits in Hadoop 0.203+.
  (mtiwari via jghoman)

  GIRAPH-12: Investigate communication improvements. (hyunsik)

  GIRAPH-46: Race condition on superstep 1 with RPC servers not
  started by the time that requests are sent. (aching)

  GIRAPH-21: Revise CODE_CONVENTIONS. (aching via jghoman)

  GIRAPH-39: mvn rat doesn't like .git or .idea. (jghoman)

  GIRAPH-32: Implement benchmarks to evaluate the performance of message
  passing. (hyunsik)

  GIRAPH-34: Failure of Vertex reflection for putVertexList from
  GIRAPH-27. (aching)

  GIRAPH-35: Modifying the site to indicate that Jake Mannix and
  Dmitriy Ryaboy are now Giraph committers. (aching)

  GIRAPH-33: Missing license header of GraphState.java (Claudio
  Martella via hyunsik)

  GIRAPH-31: Hide the SortedMap<I, Edge<I,E>> in Vertex from client
  visibility (impl. detail), replace with appropriate accessor
  methods. (jake.mannix via aching)

  GIRAPH-30: NPE in ZooKeeperManager if base directory cannot be
  created. apurtell via aching.

  GIRAPH-27: Mutable static global state in Vertex.java should be
  refactored. jake.mannix via aching.
 
  GIRAPH-25: NPE in BspServiceMaster when failing a job. dvryaboy via
  aching.

  GIRAPH-24: Job-level statistics reports one superstep greater than
  workers. (jghoman)
  
  GIRAPH-18: Refactor BspServiceWorker::loadVertices(). (jghoman)
  
  GIRAPH-14: Support for the Facebook Hadoop branch. (aching)

  GIRAPH-16: Add Apache RAT to the verify build step. (omalley)

  GIRAPH-17: Giraph doesn't give up properly after the maximum connect
  attempts to ZooKeeper. (aching)

  GIRAPH-2: Make the project homepage. (jghoman)

  GIRAPH-9: Change Yahoo License Header to Apache License Header (hyunsik)

  GIRAPH-6: Remove Yahoo-specific code from pom.xml. (jghoman)

  GIRAPH-5: Remove Yahoo directories after svn import from Yahoo! (aching)

  GIRAPH-3: Vertex:sentMsgToAllEdges should be sendMsg. (jghoman)
Something went wrong with that request. Please try again.