sbt is installed on your machine, and run
Configuring the Migrator
config.yaml for your migration using the template
config.yaml in the repository root. Read the comments throughout carefully.
Running on a live Spark cluster
The Scylla Migrator is built against Spark 2.3.1, so you'll need to run that version on your cluster.
build.sh, copy the jar from
./target/scala-2.11/scylla-migrator-assembly-0.0.1.jar and the
config.yaml you've created to the Spark master server.
Then, run this command on the Spark master server:
spark-submit --class com.scylladb.migrator.Migrator \ --master spark://<spark-master-hostname>:7077 \ --conf spark.scylla.config=<path to config.yaml> <path to scylla-migrator-assembly-0.0.1.jar>
To run in the local Docker-based setup:
- First start the environment:
docker-compose up -d
cqlshin Cassandra's container and create a keyspace and a table with some data:
docker-compose exec cassandra cqlsh <create stuff>
cqlshin Scylla's container and create the destination keyspace and table with the same schema as the source table:
docker-compose exec scylla cqlsh <create stuff>
config.yamlfile; note the comments throughout.
spark-submitin the master's container to run the job:
docker-compose exec spark-master spark-submit --class com.scylladb.migrator.Migrator \ --master spark://spark-master:7077 \ --conf spark.driver.host=spark-master \ --conf spark.scylla.config=/app/config.yaml \ /jars/scylla-migrator-assembly-0.0.1.jar
spark-master container mounts the
./target/scala-2.11 dir on
/jars and the repository root on
/app. To update the jar with new code, just run
build.sh and then run