Skip to content
Alluxio, data orchestration for analytics and machine learning in the cloud
Java TypeScript Shell Go Ruby Python Other
Branch: master
Clone or download
gpang and alluxio-bot Fix flakiness of migrate test
This test became flaky when the default write type became async-through.
This is because when the migrate is configured to delete the source,
sometimes, that inode is NOT persisted, so it is not removed from the
UFS. Then, on the `exists` call, it will loadMetadata from UFS, and it
will load the file from UFS and the deleted file will "return".

pr-link: #9884
change-id: cid-d9c66bd59b9e4180756dc13a8ac243f6951aafbe
Latest commit 714d0a8 Sep 13, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github/ISSUE_TEMPLATE Add a new issue type on docs defect Aug 27, 2019
assembly Update poms and other locations to 2.1.0-SNAPSHOT Jun 27, 2019
bin Update launch-process contains function so the script does not exit Aug 19, 2019
build Add non-generated gRPC classes under checkstyle/findbugs coverage Apr 26, 2019
conf Update alluxio-env templete to contain proxy opts Aug 6, 2019
core Increase copycat transport max message size Sep 11, 2019
dev Update generate-tarball script to support hadoop-3.2 Sep 13, 2019
docs Add helm templates for Alluxio HA tests Sep 12, 2019
examples Bump up retry count to tolerate eventual consistency Aug 20, 2019
integration Set workdir and path for docker image Sep 12, 2019
job Remove useless dependencies Sep 9, 2019
libexec Update poms and other locations to 2.1.0-SNAPSHOT Jun 27, 2019
logs Add logs/README.md Feb 26, 2016
logserver Update poms and other locations to 2.1.0-SNAPSHOT Jun 27, 2019
minicluster Add integration test for growing embedded journal cluster Aug 8, 2019
shaded Update poms and other locations to 2.1.0-SNAPSHOT Jun 27, 2019
shell Show progress for taking/applying backups Aug 30, 2019
templates Update template script url to https Jun 5, 2019
tests Fix flakiness of migrate test Sep 13, 2019
underfs Update generate-tarball script to support hadoop-3.2 Sep 13, 2019
webui Add 'newer alluxio version available' message to WEB UI Sep 3, 2019
.gitignore Update vagrant integration and documentation Jun 23, 2019
LICENSE Migrate to ForkJoinPool for master RPCs Apr 17, 2019
NOTICE Jira 2868 Rename file tests/src/test/java/alluxio/shell/command/Leade… Jun 25, 2017
README.md Switch to console mode from bash for code block Jul 29, 2019
pom.xml Remove useless dependencies Sep 9, 2019

README.md

logo

Slack Release Docker Pulls Documentation Twitter Follow License

What is Alluxio

Alluxio (formerly known as Tachyon) is a virtual distributed storage system. It bridges the gap between computation frameworks and storage systems, enabling computation applications to connect to numerous storage systems through a common interface. Read more about Alluxio Overview.

The Alluxio project originated from a research project called Tachyon at AMPLab, UC Berkeley, which was the data layer of the Berkeley Data Analytics Stack (BDAS). For more details, please refer to Haoyuan Li's PhD dissertation Alluxio: A Virtual Distributed File System.

Who Uses Alluxio

Alluxio is used in production to manage Petabytes of data in many leading companies, with the largest deployment exceeding 1300 nodes. Find more use cases at Powered by Alluxio.

Community and Events

Please use the following to reach members of the community:

Download Alluxio

Binary download

Prebuilt binaries are available to download at https://www.alluxio.io/download .

Docker

Download and start an Alluxio master and a worker. More details can be found in documentation.

# Create a network for connecting Alluxio containers
$ docker network create alluxio_nw
# Create a volume for storing ufs data
$ docker volume create ufs
# Launch the Alluxio master
$ docker run -d --net=alluxio_nw \
    -p 19999:19999 \
    --name=alluxio-master \
    -v ufs:/opt/alluxio/underFSStorage \
    alluxio/alluxio master
# Launch the Alluxio worker
$ export ALLUXIO_WORKER_MEMORY_SIZE=1G
$ docker run -d --net=alluxio_nw \
    --shm-size=${ALLUXIO_WORKER_MEMORY_SIZE} \
    --name=alluxio-worker \
    -v ufs:/opt/alluxio/underFSStorage \
    -e ALLUXIO_JAVA_OPTS="-Dalluxio.worker.memory.size=${ALLUXIO_WORKER_MEMORY_SIZE} -Dalluxio.master.hostname=alluxio-master" \
    alluxio/alluxio worker

MacOS Homebrew

$ brew install alluxio

Quick Start

Please follow the Guide to Get Started to run a simple example with Alluxio.

Report a Bug

To report bugs, suggest improvements, or create new feature requests, please open a Github Issue. Our previous Alluxio JIRA system has been deprecated since December 2018.

Depend on Alluxio

For Alluxio versions 1.4 or earlier, use the alluxio-core-client artifact.

For Alluxio versions 1.5 or later, Alluxio provides several different client artifacts. The Alluxio file system interface provided by the alluxio-core-client-fs artifact is recommended for the best performance and access to Alluxio-specific functionality. If you want to use other interfaces, include the appropriate client artifact. For example, alluxio-core-client-hdfs provides a client implementing HDFS's file system API.

Apache Maven

<dependency>
  <groupId>org.alluxio</groupId>
  <artifactId>alluxio-core-client-fs</artifactId>
  <version>2.0.0</version>
</dependency>

SBT

libraryDependencies += "org.alluxio" % "alluxio-core-client-fs" % "2.0.0"

Contributing

Contributions via GitHub pull requests are gladly accepted from their original author. Along with any pull requests, please state that the contribution is your original work and that you license the work to the project under the project's open source license. Whether or not you state this explicitly, by submitting any copyrighted material via pull request, email, or other means you agree to license the material under the project's open source license and warrant that you have the legal authority to do so. For a more detailed step-by-step guide, please read how to contribute to Alluxio. For new contributor, please take 2 new contributor tasks.

Useful Links

You can’t perform that action at this time.