Permalink
Browse files

trunk

git-svn-id: https://svn.apache.org/repos/asf/incubator/giraph/trunk@1161805 13f79535-47bb-0310-9956-ffa450edef68
  • Loading branch information...
1 parent 3802e1d commit 879ae5407254418614d4b0410357951257e9dea9 @aching aching committed Oct 29, 2010
Showing with 0 additions and 17,426 deletions.
  1. +0 −31 CODE_CONVENTIONS
  2. +0 −202 LICENSE.txt
  3. +0 −8 NOTICE
  4. +0 −67 README
  5. +0 −264 pom.xml
  6. +0 −175 src/main/java/org/apache/giraph/benchmark/PageRankBenchmark.java
  7. +0 −186 src/main/java/org/apache/giraph/benchmark/PseudoRandomVertexInputFormat.java
  8. +0 −29 src/main/java/org/apache/giraph/bsp/ApplicationState.java
  9. +0 −90 src/main/java/org/apache/giraph/bsp/BspInputFormat.java
  10. +0 −82 src/main/java/org/apache/giraph/bsp/BspInputSplit.java
  11. +0 −71 src/main/java/org/apache/giraph/bsp/BspOutputFormat.java
  12. +0 −75 src/main/java/org/apache/giraph/bsp/BspRecordReader.java
  13. +0 −29 src/main/java/org/apache/giraph/bsp/BspRecordWriter.java
  14. +0 −79 src/main/java/org/apache/giraph/bsp/CentralizedService.java
  15. +0 −89 src/main/java/org/apache/giraph/bsp/CentralizedServiceMaster.java
  16. +0 −154 src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java
  17. +0 −64 src/main/java/org/apache/giraph/bsp/ImmutableOutputCommitter.java
  18. +0 −29 src/main/java/org/apache/giraph/bsp/SuperstepState.java
  19. +0 −109 src/main/java/org/apache/giraph/comm/ArrayListWritable.java
  20. +0 −1,084 src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java
  21. +0 −128 src/main/java/org/apache/giraph/comm/CommunicationsInterface.java
  22. +0 −44 src/main/java/org/apache/giraph/comm/MsgList.java
  23. +0 −164 src/main/java/org/apache/giraph/comm/RPCCommunications.java
  24. +0 −66 src/main/java/org/apache/giraph/comm/ServerInterface.java
  25. +0 −56 src/main/java/org/apache/giraph/comm/VertexList.java
  26. +0 −112 src/main/java/org/apache/giraph/comm/WorkerCommunications.java
  27. +0 −55 src/main/java/org/apache/giraph/examples/GeneratedVertexInputFormat.java
  28. +0 −73 src/main/java/org/apache/giraph/examples/GeneratedVertexReader.java
  29. +0 −52 src/main/java/org/apache/giraph/examples/LongSumAggregator.java
  30. +0 −53 src/main/java/org/apache/giraph/examples/MaxAggregator.java
  31. +0 −53 src/main/java/org/apache/giraph/examples/MinAggregator.java
  32. +0 −228 src/main/java/org/apache/giraph/examples/SimpleCheckpointVertex.java
  33. +0 −65 src/main/java/org/apache/giraph/examples/SimpleCombinerVertex.java
  34. +0 −68 src/main/java/org/apache/giraph/examples/SimpleFailVertex.java
  35. +0 −63 src/main/java/org/apache/giraph/examples/SimpleMsgVertex.java
  36. +0 −154 src/main/java/org/apache/giraph/examples/SimpleMutateGraphVertex.java
  37. +0 −218 src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java
  38. +0 −269 src/main/java/org/apache/giraph/examples/SimpleShortestPathsVertex.java
  39. +0 −45 src/main/java/org/apache/giraph/examples/SimpleSumCombiner.java
  40. +0 −139 src/main/java/org/apache/giraph/examples/SimpleSuperstepVertex.java
  41. +0 −71 src/main/java/org/apache/giraph/examples/SimpleTextVertexOutputFormat.java
  42. +0 −53 src/main/java/org/apache/giraph/examples/SumAggregator.java
  43. +0 −80 src/main/java/org/apache/giraph/examples/SuperstepBalancer.java
  44. +0 −62 src/main/java/org/apache/giraph/graph/Aggregator.java
  45. +0 −57 src/main/java/org/apache/giraph/graph/AggregatorUsage.java
  46. +0 −302 src/main/java/org/apache/giraph/graph/AutoBalancer.java
  47. +0 −161 src/main/java/org/apache/giraph/graph/BasicVertex.java
  48. +0 −76 src/main/java/org/apache/giraph/graph/BasicVertexRangeBalancer.java
  49. +0 −59 src/main/java/org/apache/giraph/graph/BasicVertexResolver.java
  50. +0 −1,101 src/main/java/org/apache/giraph/graph/BspService.java
  51. +0 −1,991 src/main/java/org/apache/giraph/graph/BspServiceMaster.java
  52. +0 −1,334 src/main/java/org/apache/giraph/graph/BspServiceWorker.java
  53. +0 −362 src/main/java/org/apache/giraph/graph/BspUtils.java
  54. +0 −119 src/main/java/org/apache/giraph/graph/Edge.java
  55. +0 −496 src/main/java/org/apache/giraph/graph/GiraphJob.java
  56. +0 −655 src/main/java/org/apache/giraph/graph/GraphMapper.java
  57. +0 −170 src/main/java/org/apache/giraph/graph/MasterThread.java
  58. +0 −92 src/main/java/org/apache/giraph/graph/MutableVertex.java
  59. +0 −42 src/main/java/org/apache/giraph/graph/StaticBalancer.java
  60. +0 −346 src/main/java/org/apache/giraph/graph/Vertex.java
  61. +0 −72 src/main/java/org/apache/giraph/graph/VertexChanges.java
  62. +0 −49 src/main/java/org/apache/giraph/graph/VertexCombiner.java
  63. +0 −79 src/main/java/org/apache/giraph/graph/VertexInputFormat.java
  64. +0 −122 src/main/java/org/apache/giraph/graph/VertexMutations.java
  65. +0 −82 src/main/java/org/apache/giraph/graph/VertexOutputFormat.java
  66. +0 −351 src/main/java/org/apache/giraph/graph/VertexRange.java
  67. +0 −185 src/main/java/org/apache/giraph/graph/VertexRangeBalancer.java
  68. +0 −71 src/main/java/org/apache/giraph/graph/VertexReader.java
  69. +0 −122 src/main/java/org/apache/giraph/graph/VertexResolver.java
  70. +0 −67 src/main/java/org/apache/giraph/graph/VertexWriter.java
  71. +0 −40 src/main/java/org/apache/giraph/hadoop/BspPolicyProvider.java
  72. +0 −51 src/main/java/org/apache/giraph/hadoop/BspTokenSelector.java
  73. +0 −31 src/main/java/org/apache/giraph/lib/JsonBase64VertexFormat.java
  74. +0 −154 src/main/java/org/apache/giraph/lib/JsonBase64VertexInputFormat.java
  75. +0 −124 src/main/java/org/apache/giraph/lib/JsonBase64VertexOutputFormat.java
  76. +0 −123 src/main/java/org/apache/giraph/lib/TextVertexInputFormat.java
  77. +0 −117 src/main/java/org/apache/giraph/lib/TextVertexOutputFormat.java
  78. +0 −125 src/main/java/org/apache/giraph/utils/ReflectionUtils.java
  79. +0 −49 src/main/java/org/apache/giraph/zk/BspEvent.java
  80. +0 −52 src/main/java/org/apache/giraph/zk/ContextLock.java
  81. +0 −117 src/main/java/org/apache/giraph/zk/PredicateLock.java
  82. +0 −278 src/main/java/org/apache/giraph/zk/ZooKeeperExt.java
  83. +0 −801 src/main/java/org/apache/giraph/zk/ZooKeeperManager.java
  84. +0 −237 src/test/java/org/apache/giraph/BspCase.java
  85. +0 −88 src/test/java/org/apache/giraph/TestAutoCheckpoint.java
  86. +0 −318 src/test/java/org/apache/giraph/TestBspBasic.java
  87. +0 −115 src/test/java/org/apache/giraph/TestJsonBase64Format.java
  88. +0 −114 src/test/java/org/apache/giraph/TestManualCheckpoint.java
  89. +0 −69 src/test/java/org/apache/giraph/TestMutateGraphVertex.java
  90. +0 −84 src/test/java/org/apache/giraph/TestPredicateLock.java
  91. +0 −126 src/test/java/org/apache/giraph/TestVertexRangeBalancer.java
  92. +0 −205 src/test/java/org/apache/giraph/TestVertexTypes.java
  93. +0 −157 src/test/java/org/apache/giraph/TestZooKeeperExt.java
View
@@ -1,31 +0,0 @@
-This codebase follows the Oracle "Code Conventions for the Java
-Programming Language". See the following link:
-
-http://www.oracle.com/technetwork/java/codeconvtoc-136057.html
-
-In addition, this project has a couple of more specific rules:
-
-* No tabs
-* All indents should be 4 spaces (or 8 when there is confusion)
-* All while/if/else must have brackets, even if there there is only a one
- line statement following. 'else' and 'else if' are expected to line up
- with the '}'. For example:
-
-if (condition) {
- statement;
-}
-
-if (condition) {
- statement;
-} else {
- statement;
-}
-
-* Any use of LOG should be enclosed with an is*Enabled() method. For example:
-
-if (LOG.isInfoEnabled()) {
- LOG.info("something happened");
-}
-
-* Class members should not begin with 'm_' or '_'.
-* No warnings allowed, but be as specific as possible with warning suppression
View
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
View
8 NOTICE
@@ -1,8 +0,0 @@
-=========================================================================
-NOTICE file for use with, and corresponding to Section 4 of, the
-Apache License, Version 2.0, in this case for the Giraph project.
-=========================================================================
-
-This product includes software developed by
-Yahoo! Inc. (www.yahoo.com)
-Copyright (c) 2011 Yahoo! Inc. All rights reserved.
View
67 README
@@ -1,67 +0,0 @@
-Giraph : Large-scale graph processing on Hadoop
-
-Web and online social graphs have been rapidly growing in size and
-scale during the past decade. In 2008, Google estimated that the
-number of web pages reached over a trillion. Online social networking
-and email sites, including Yahoo!, Google, Microsoft, Facebook,
-LinkedIn, and Twitter, have hundreds of millions of users and are
-expected to grow much more in the future. Processing these graphs
-plays a big role in relevant and personalized information for users,
-such as results from a search engine or news in an online social
-networking site.
-
-Graph processing platforms to run large-scale algorithms (such as page
-rank, shared connections, personalization-based popularity, etc.) have
-become quite popular. Some recent examples include Pregel and HaLoop.
-For general-purpose big data computation, the map-reduce computing
-model has been well adopted and the most deployed map-reduce
-infrastructure is Apache Hadoop. We have implemented a
-graph-processing framework that is launched as a typical Hadoop job to
-leverage existing Hadoop infrastructure, such as Amazon’s EC2. Giraph
-builds upon the graph-oriented nature of Pregel but additionally adds
-fault-tolerance to the coordinator process with the use of ZooKeeper
-as its centralized coordination service.
-
-Giraph follows the bulk-synchronous parallel model relative to graphs
-where vertices can send messages to other vertices during a given
-superstep. Checkpoints are initiated by the Giraph infrastructure at
-user-defined intervals and are used for automatic application restarts
-when any worker in the application fails. Any worker in the
-application can act as the application coordinator and one will
-automatically take over if the current application coordinator fails.
-
--------------------------------
-
-Hadoop versions for use with Giraph:
-
-Secure Hadoop versions:
-- Apache Hadoop 0.20.203, 0.20.204, other secure versions may work as well
--- Other versions reported working include:
---- Cloudera CDH3u0, CDH3u1
-
-Unsecure Hadoop versions:
-- Apache Hadoop 0.20.1, 0.20.2, 0.20.3
-
-While we provide support for unsecure Hadoop with the maven profile
-'hadoop_non_secure', we have been primarily focusing on secure Hadoop
-releases at this time.
-
--------------------------------
-
-Building and testing:
-
-You will need the following:
-- Java 1.6
-- Maven 2 or higher
-
-Use the maven commands with secure Hadoop to:
-- compile (i.e mvn compile)
-- package (i.e. mvn package)
-- test (i.e. mvn test)
--- For testing, one can submit the test to a running Hadoop instance
- (i.e. mvn test -Dprop.mapred.job.tracker=localhost:50300)
-
-For the non-secure versions of Hadoop, run the maven commands with the
-additional argument '-Dhadoop=non_secure' to enable the maven profile
- 'hadoop_non_secure'. An example compilation command is
-'mvn -Dhadoop=non_secure compile'.
Oops, something went wrong.

0 comments on commit 879ae54

Please sign in to comment.