diff --git a/README.md b/README.md
index ce024f3..3e702b8 100644
--- a/README.md
+++ b/README.md
@@ -1,5 +1,6 @@
-
-# DataSqueeze
+
+
+
## Overview
DataSqueeze performs compaction of files from source directory to target directory maintaining the directory structure of the source.
@@ -7,6 +8,14 @@ DataSqueeze performs compaction of files from source directory to target directo
## Documentation
This README is intended to provide detailed technical documentation for advanced users.
+## Changes since last release
+
+* Edited the pom file for publishing this project's artifacts to the Maven Central
+* Changed namespace to com.expedia.dsp
+* Renamed dataSqueeze to datasqueeze
+* Refactored code to remove the dataSqueeze-manager layer
+* Added DataSqueeze logo
+
## General operation
DataSqueeze supports two types of compaction
@@ -18,19 +27,19 @@ DataSqueeze supports two types of compaction
a. Fetch the source file paths to be compacted from the source path provided.
b. Perform mapreduce job using the following configuration
1. Mapper maps records together based on same parent directory and emits parent directory as key.
- 2. Reducer reduces records based on same key but writes data to the target directory provided by the user, retaining
- the directory structure.
+ 2. Reducer reduces records based on same key but writes data to the target directory provided by the user,
+ retaining the directory structure.
-2. Inplace Compaction - Performs compaction on the source path. This is not recommended on AWS-S3, since the performance will be terrible.
+2. In-Place Compaction - Performs compaction on the source path. This is not recommended on AWS-S3, since the performance will be terrible.
- Below is a high level summary of the steps that Compaction Utility performs during the course of a typical run for inplace compaction.
+ Below is a high level summary of the steps that Compaction Utility performs during the course of a typical run for in-place compaction.
a. Fetch the file paths to be compacted from the source path provided.
b. Perform mapreduce job using the following configuration
1. Mapper maps records together based on same parent directory and emits parent directory as key.
- 2. Reducer reduces records based on same key but writes data to the target directory provided by the user, retaining
- the directory structure.
+ 2. Reducer reduces records based on same key but writes data to the target directory provided by the user,
+ retaining the directory structure.
c. Store the compacted files on temp-compacted path.
d. Move files from source to temp location.
e. Move files from temp-compacted location to source location specified by the user.
@@ -49,7 +58,7 @@ DataSqueeze is a standard Maven project. Run the following in the project root f
mvn clean package
-The compiled JAR can be found at `dataSqueeze-manager/target/dataSqueeze-manager-{VERSION}.jar`.
+The compiled JAR can be found at `datasqueeze/target/datasqueeze.jar`.
To build an RPM, use the optional Maven profile `-P rpm`:
@@ -64,14 +73,14 @@ There are two different ways of running DataSqueeze:
1. CLI -
a. For TEXT/ORC/SEQ
```java
- hadoop jar dataSqueeze-manager-1.0-SNAPSHOT.jar com.expedia.edw.data.squeeze.Utility
+ hadoop jar datasqueeze.jar com.expedia.dsp.data.squeeze.Utility
-sp s3a://edwprod/user/ysontakke/compactiontest1/ -tp s3a://edwprod/user/ysontakke/compactionoutput_text_yash_1/
-threshold 12345
```
b. For AVRO
```java
- hadoop jar dataSqueeze-manager-1.0-SNAPSHOT.jar com.expedia.edw.data.squeeze.Utility
+ hadoop jar datasqueeze.jar com.expedia.dsp.data.squeeze.Utility
-sp s3a://edwprod/user/ysontakke/compactiontest1/ -tp s3a://edwprod/user/ysontakke/compactionoutput_text_yash_1/
-threshold 12345 -fileType AVRO -schemaPath s3a://edwprod/user/ysontakke/compactionschema_text_yash_1/schema.avsc
```
@@ -87,7 +96,7 @@ There are two different ways of running DataSqueeze:
* fileType - Type of file to be compacted (AVRO / TEXT / SEQ / ORC). It is mandatory for AVRO
* schemaPath - schema used for compaction (mandatory for AVRO)
-2. API - [CompactionManager](dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/CompactionManager.java)
+2. API - [CompactionManager](datasqueeze/src/main/java/com/expedia/dsp/data/squeeze/CompactionManager.java)
```java
CompactionResponse compact() throws Exception;
diff --git a/dataSqueeze-logo.png b/dataSqueeze-logo.png
new file mode 100644
index 0000000..74ea46f
Binary files /dev/null and b/dataSqueeze-logo.png differ
diff --git a/dataSqueeze-manager/pom.xml b/dataSqueeze-manager/pom.xml
deleted file mode 100644
index 6f0f4ad..0000000
--- a/dataSqueeze-manager/pom.xml
+++ /dev/null
@@ -1,212 +0,0 @@
-
-
- 4.0.0
-
-
- dataSqueeze-parent
- com.expedia.edw.data.squeeze
- 1.0-SNAPSHOT
-
- dataSqueeze-manager
-
- Hadoop utility to compact small files
- https://expediaincecommerceplatform.github.io/dataSqueeze/
-
-
-
- Apache 2.0
- https://www.apache.org/licenses/LICENSE-2.0
-
-
-
-
- /opt/dataSqueeze
- ${project.basedir}/target
- hadoop
- hadoop
- UTF-8
-
-
-
-
-
- org.apache.hadoop
- hadoop-mapreduce-client-common
- provided
-
-
- org.apache.avro
- avro
-
-
-
-
- org.projectlombok
- lombok
- provided
-
-
-
-
- commons-cli
- commons-cli
- compile
-
-
- org.apache.commons
- commons-lang3
- compile
-
-
- net.sf.jmimemagic
- jmimemagic
- compile
-
-
- org.json
- json
- compile
-
-
- org.apache.httpcomponents
- httpcore
- compile
-
-
- org.apache.orc
- orc-core
- compile
-
-
- org.apache.orc
- orc-mapreduce
- compile
-
-
- xerces
- xercesImpl
- compile
-
-
- org.apache.avro
- avro-mapred
- 1.7.4
- compile
- hadoop2
-
-
-
-
- com.hotels
- avro-compatibility
- 2.1.1
-
-
-
-
- junit
- junit
- test
-
-
- org.powermock
- powermock-module-junit4
- test
-
-
- org.powermock
- powermock-api-mockito
- test
-
-
- org.apache.mrunit
- mrunit
- test
- hadoop2
-
-
- org.mockito
- mockito-all
- test
-
-
-
-
-
-
- org.apache.maven.plugins
- maven-shade-plugin
- 3.0.0
-
-
- package
-
- shade
-
-
-
-
- *:*
-
- META-INF/*.SF
- META-INF/*.DSA
- META-INF/*.RSA
-
-
-
-
-
-
-
-
-
-
-
-
- rpm
-
-
-
- org.codehaus.mojo
- rpm-maven-plugin
- 2.1.5
-
-
- attach-rpm
- package
-
- attached-rpm
-
-
- ${project.description}
- Applications/Engineering
- Linux
- redhat
- ${rpm.install.user}
- ${rpm.install.group}
- 775
- 644
-
-
- ${rpm.install.dir}/lib
-
-
-
- ${project.build.directory}/dataSqueeze-manager-${project.version}.jar
-
- dataSqueeze-manager.jar
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/models/CompactionCriteriaTest.java b/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/models/CompactionCriteriaTest.java
deleted file mode 100644
index 339e20f..0000000
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/models/CompactionCriteriaTest.java
+++ /dev/null
@@ -1,130 +0,0 @@
-//package com.expedia.edw.data.squeeze.models;
-//
-//import org.junit.Assert;
-//import org.junit.Test;
-//
-//import java.util.HashMap;
-//import java.util.Map;
-//
-//import static org.junit.Assert.assertEquals;
-//import static org.junit.Assert.assertNull;
-//
-//import com.expedia.edw.data.squeeze.impl.CompactionManagerImpl;
-//
-///**
-// * Tests for {@link CompactionCriteria}
-// *
-// * @author Yashraj R. Sontakke
-// */
-//public class CompactionCriteriaTest {
-//
-// @Test
-// public void testNullThreshold() throws Exception {
-// final Map options = retrieveOptions(null, "1000");
-// CompactionCriteria criteria = new CompactionCriteria(options);
-// assertNull(criteria.getThresholdInBytes());
-// }
-//
-// @Test(expected = NumberFormatException.class)
-// public void testWrongThreshold() throws Exception {
-// final Map options = retrieveOptions("1234a", "1000");
-// CompactionCriteria criteria = new CompactionCriteria(options);
-// assertNull(criteria.getThresholdInBytes());
-// }
-//
-// @Test
-// public void testNullMaxReducers() throws Exception {
-// final Map options = retrieveOptions("1234", null);
-// CompactionCriteria criteria = new CompactionCriteria(options);
-// assertNull(criteria.getMaxReducers());
-// }
-//
-// @Test(expected = NumberFormatException.class)
-// public void testWrongMaxReducers() throws Exception {
-// final Map options = retrieveOptions("1234", "1000a");
-// CompactionCriteria criteria = new CompactionCriteria(options);
-// assertNull(criteria.getMaxReducers());
-// }
-//
-// @Test(expected = IllegalArgumentException.class)
-// public void testNoSchemaPath() throws Exception {
-// final Map options = retrieveOptions("1234", "1000");
-// options.put("fileType", "AVRO");
-// CompactionCriteria criteria = new CompactionCriteria(options);
-// }
-//
-// @Test
-// public void testCriteria() throws Exception {
-//<<<<<<< HEAD
-// CompactionCriteria criteria = new CompactionCriteria("source", "target", 12345L, "temp", false, 1000L);
-// assertEquals("source", criteria.getSourcePath());
-// assertEquals("target", criteria.getTargetPath());
-// assertEquals(12345L, criteria.getThresholdInBytes(), 0);
-// assertEquals("temp", criteria.getTempPath());
-// assertFalse(criteria.isRetainTempFiles());
-// assertEquals(1000L, criteria.getMaxReducers(), 0);
-//
-// criteria = new CompactionCriteria("source", "target", 12345L, "temp", true, 1000L);
-// assertTrue(criteria.isRetainTempFiles());
-//
-// Map options = retrieveOptions("1234", "1000");
-// options.put("retainTempFiles", "true");
-//
-// criteria = new CompactionCriteria(options);
-// assertEquals("sourcePath", criteria.getSourcePath());
-// assertEquals("targetPath", criteria.getTargetPath());
-// assertEquals(1234L, criteria.getThresholdInBytes(), 0);
-// assertEquals("tempPath", criteria.getTempPath());
-// assertTrue(criteria.isRetainTempFiles());
-// assertEquals(1000L, criteria.getMaxReducers(), 0);
-//
-// options.put("retainTempFiles", "false");
-// criteria = new CompactionCriteria(options);
-// assertFalse(criteria.isRetainTempFiles());
-//
-// options.put("retainTempFiles", "abcd");
-// criteria = new CompactionCriteria(options);
-// assertFalse(criteria.isRetainTempFiles());
-//
-// assertNull(criteria.getFileType());
-// assertNull(criteria.getSchemaPath());
-//
-// options.put("fileType", "AVRO");
-// options.put("schemaPath", "schemaPath");
-// criteria = new CompactionCriteria(options);
-//
-// assertEquals(FileType.AVRO.getValue(), criteria.getFileType());
-// assertEquals("schemaPath", criteria.getSchemaPath());
-//
-// options.put("fileType", "ORC");
-// options.remove("schemaPath");
-// criteria = new CompactionCriteria(options);
-//
-// assertEquals(FileType.ORC.getValue(), criteria.getFileType());
-//
-//=======
-// CompactionCriteria criteria = new CompactionCriteria("source", "target", 12345L);
-// assertEquals("source", criteria.getSourcePath());
-// assertEquals("target", criteria.getTargetPath());
-// assertEquals(12345L, criteria.getThresholdInBytes(), 0);
-//>>>>>>> develop
-// }
-//
-// @Test(expected = IllegalArgumentException.class)
-// public void testUnsupportedFileFormat() {
-// Map options = retrieveOptions("1234", "1000");
-// options.put("fileType", "AVROO");
-// new CompactionCriteria(options);
-// }
-//
-// private Map retrieveOptions(final String threshold, final String maxReducers) {
-// final Map options = new HashMap();
-// options.put("sourcePath", "sourcePath");
-// options.put("targetPath", "targetPath");
-// options.put("thresholdInBytes", threshold);
-// options.put("tempPath", "tempPath");
-// options.put("maxReducers", maxReducers);
-// return options;
-// }
-//
-//}
diff --git a/pom.xml b/pom.xml
index f28362a..1bfc37d 100644
--- a/pom.xml
+++ b/pom.xml
@@ -4,14 +4,13 @@
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
4.0.0
- com.expedia.edw.data.squeeze
- dataSqueeze-parent
- dataSqueeze-parent
- Hadoop utility to compact small files
- https://expediaincecommerceplatform.github.io/dataSqueeze/
+ com.expedia.dsp
+ datasqueeze
+ 2.0-SNAPSHOT
- pom
- 1.0-SNAPSHOT
+ datasqueeze
+ Hadoop utility to compact small files
+ https://expediaincecommerceplatform.github.io/datasqueeze/
@@ -20,15 +19,17 @@
+
+ scm:git:git://github.com/ExpediaInceCommercePlatform/datasqueeze.git
+ scm:git:ssh://github.com:ExpediaInceCommercePlatform/datasqueeze.git
+ https://github.com/ExpediaInceCommercePlatform/datasqueeze/tree/master
+
+
github
- https://github.com/ExpediaInceCommercePlatform/dataSqueeze/issues
+ https://github.com/ExpediaInceCommercePlatform/datasqueeze/issues
-
- dataSqueeze-manager
-
-
Yashraj R. Sontakke
@@ -38,6 +39,8 @@
+ UTF-8
+
1.3.2
1.11.98
@@ -45,6 +48,7 @@
2.4
3.0
+
2.7.4
1.3.2
@@ -74,265 +78,282 @@
3.3.9
1.10.19
1.3.2
- 1.6.1
1.7.25
2.8.1
+ 1.7.4
+
+ /opt/dataSqueeze
+ ${project.basedir}/target
+ hadoop
+ hadoop
+
+
+
+
+ commons-cli
+ commons-cli
+ ${commons-cli.version}
+ compile
+
+
+ org.apache.orc
+ orc-mapreduce
+ ${orc.version}
+ compile
+
+
+ org.apache.orc
+ orc-core
+ ${orc.version}
+ compile
+
+
+ org.apache.avro
+ avro-mapred
+ ${avro.version}
+ compile
+ hadoop2
+
+
+ org.apache.commons
+ commons-lang3
+ ${commons-lang3.version}
+ compile
+
+
+ org.apache.httpcomponents
+ httpcore
+ ${httpcore.version}
+ compile
+
+
+ org.json
+ json
+ ${json.version}
+ compile
+
+
+ xerces
+ xercesImpl
+ ${xerces.version}
+ compile
+
+
+ net.sf.jmimemagic
+ jmimemagic
+ ${jmimemagic.version}
+ compile
+
+
+ commons-logging
+ commons-logging
+
+
+ commons-io
+ commons-io
+
+
+ log4j
+ log4j
+
+
+ org.codehaus.jackson
+ jackson-xc
+
+
+ org.codehaus.jackson
+ jackson-jaxrs
+
+
+ xerces
+ xercesImpl
+
+
+
+
+ commons-io
+ commons-io
+ ${commons.io.version}
+
-
-
-
- com.amazonaws
- aws-java-sdk-s3
- ${aws.java.sdk.version}
-
-
- commons-cli
- commons-cli
- ${commons-cli.version}
-
-
- org.apache.commons
- commons-io
- ${apache.commons.io.version}
-
-
- commons-io
- commons-io
- ${commons.io.version}
-
-
- org.apache.commons
- commons-lang3
- ${commons.lang3.version}
-
-
- org.apache.hadoop
- hadoop-aws
- ${hadoop.version}
-
-
- org.apache.hadoop
- hadoop-common
- ${hadoop.version}
-
-
- org.apache.httpcomponents
- httpcore
-
-
-
-
- org.apache.hadoop
- hadoop-hdfs
- ${hadoop.version}
-
-
- org.apache.hadoop
- hadoop-mapreduce-client-jobclient
- ${hadoop.version}
-
-
- org.apache.hadoop
- hadoop-mapreduce-client-common
- ${hadoop.version}
-
-
- org.apache.hive
- hive-jdbc
- ${hive.jdbc.version}
-
-
- org.apache.hive
- hive-metastore
- ${hive.metastore.version}
-
-
- org.apache.orc
- orc-mapreduce
- ${orc.version}
-
-
- org.apache.orc
- orc-core
- ${orc.version}
-
-
- org.projectlombok
- lombok
- ${lombok.version}
-
-
- commons-cli
- commons-cli
- ${commons-cli.version}
-
-
- commons-io
- commons-io
- ${commons-io.version}
-
-
- org.apache.commons
- commons-io
- ${commons-io-1x.version}
-
-
- org.apache.commons
- commons-lang3
- ${commons-lang3.version}
-
-
- org.apache.httpcomponents
- httpclient
- ${httpclient.version}
-
-
- org.apache.httpcomponents
- httpcore
- ${httpcore.version}
-
-
- net.sf.jmimemagic
- jmimemagic
- ${jmimemagic.version}
-
-
- commons-logging
- commons-logging
-
-
- commons-io
- commons-io
-
-
- log4j
- log4j
-
-
- org.codehaus.jackson
- jackson-xc
-
-
- org.codehaus.jackson
- jackson-jaxrs
-
-
- xerces
- xercesImpl
-
-
-
-
- org.json
- json
- ${json.version}
-
-
- xerces
- xercesImpl
- ${xerces.version}
-
-
- org.apache.maven
- maven-model
- ${maven.model.version}
-
+
+ org.apache.hadoop
+ hadoop-mapreduce-client-common
+ ${hadoop.version}
+ provided
+
+
+ org.apache.avro
+ avro
+
+
+
+
+ com.hotels
+ avro-compatibility
+ 2.1.1
+
+
+ org.projectlombok
+ lombok
+ ${lombok.version}
+ provided
+
-
-
- junit
- junit
- ${junit.version}
- test
-
-
- org.powermock
- powermock-api-mockito
- ${powermock.version}
- test
-
-
- org.powermock
- powermock-core
- ${powermock.version}
- test
-
-
- org.powermock
- powermock-module-junit4
- ${powermock.version}
- test
-
-
- org.slf4j
- slf4j-log4j12
- ${slf4j.version}
-
-
- org.apache.mrunit
- mrunit
- 1.1.0
- hadoop2
- test
-
-
- org.mockito
- mockito-all
- ${mockito.version}
- test
-
-
-
+
+ org.apache.maven
+ maven-model
+ ${maven.model.version}
+
-
-
- central
- http://repo1.maven.org/maven2/
-
-
+
+
+ junit
+ junit
+ ${junit.version}
+ test
+
+
+ org.powermock
+ powermock-api-mockito
+ ${powermock.version}
+ test
+
+
+ org.apache.mrunit
+ mrunit
+ ${mrunit.version}
+ hadoop2
+ test
+
+
+ org.mockito
+ mockito-all
+ ${mockito.version}
+ test
+
+
+ org.powermock
+ powermock-module-junit4
+ ${powermock.version}
+ test
+
+
+ org.powermock
+ powermock-core
+ ${powermock.version}
+ test
+
+
+ org.slf4j
+ slf4j-log4j12
+ ${slf4j.version}
+
+
-
-
-
- org.apache.maven.plugins
- maven-surefire-plugin
- 2.14.1
-
-
- org.apache.maven.plugins
- maven-failsafe-plugin
- 2.15
-
-
-
- integration-tests
- verify
-
-
-
-
-
-
+ ${project.artifactId}
+
+
+ org.apache.maven.plugins
+ maven-compiler-plugin
+ 3.6.1
+
+ 1.6
+ 1.6
+
+
+
+ org.apache.maven.plugins
+ maven-shade-plugin
+ 3.0.0
+
+
+ package
+
+ shade
+
+
+
+
+ *:*
+
+ META-INF/*.SF
+ META-INF/*.DSA
+ META-INF/*.RSA
+
+
+
+
+
+
+
+
- integration-tests
+ rpm
- org.apache.maven.plugins
- maven-surefire-plugin
-
- true
-
+ org.codehaus.mojo
+ rpm-maven-plugin
+ 2.1.5
+
+
+ attach-rpm
+ package
+
+ attached-rpm
+
+
+ ${project.description}
+ Applications/Engineering
+ Linux
+ redhat
+ ${rpm.install.user}
+ ${rpm.install.group}
+ 775
+ 644
+
+
+ ${rpm.install.dir}/lib
+
+
+ ${project.build.directory}/datasqueeze.jar
+ datasqueeze.jar
+
+
+
+
+
+
+
+
+
+
+ central
+ http://repo1.maven.org/maven2/
+
+
+
+
+
+ ossrh
+ https://oss.sonatype.org/content/repositories/snapshots
+
+
+ ossrh
+ https://oss.sonatype.org/service/local/staging/deploy/maven2/
+
+
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/CompactionManager.java b/src/main/java/com/expedia/dsp/data/squeeze/CompactionManager.java
similarity index 74%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/CompactionManager.java
rename to src/main/java/com/expedia/dsp/data/squeeze/CompactionManager.java
index cae8855..96c0a6d 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/CompactionManager.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/CompactionManager.java
@@ -1,7 +1,7 @@
-package com.expedia.edw.data.squeeze;
+package com.expedia.dsp.data.squeeze;
-import com.expedia.edw.data.squeeze.models.CompactionCriteria;
-import com.expedia.edw.data.squeeze.models.CompactionResponse;
+import com.expedia.dsp.data.squeeze.models.CompactionCriteria;
+import com.expedia.dsp.data.squeeze.models.CompactionResponse;
/**
* Manager that performs compaction.
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/CompactionManagerFactory.java b/src/main/java/com/expedia/dsp/data/squeeze/CompactionManagerFactory.java
similarity index 90%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/CompactionManagerFactory.java
rename to src/main/java/com/expedia/dsp/data/squeeze/CompactionManagerFactory.java
index b87ab29..291b23e 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/CompactionManagerFactory.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/CompactionManagerFactory.java
@@ -1,8 +1,8 @@
-package com.expedia.edw.data.squeeze;
+package com.expedia.dsp.data.squeeze;
-import com.expedia.edw.data.squeeze.impl.CompactionManagerImpl;
-import com.expedia.edw.data.squeeze.impl.CompactionManagerInPlaceImpl;
-import com.expedia.edw.data.squeeze.models.CompactionCriteria;
+import com.expedia.dsp.data.squeeze.impl.CompactionManagerImpl;
+import com.expedia.dsp.data.squeeze.impl.CompactionManagerInPlaceImpl;
+import com.expedia.dsp.data.squeeze.models.CompactionCriteria;
import org.apache.commons.configuration.PropertiesConfiguration;
import org.apache.commons.lang3.StringUtils;
import org.apache.hadoop.conf.Configuration;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/SchemaSelector.java b/src/main/java/com/expedia/dsp/data/squeeze/SchemaSelector.java
similarity index 65%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/SchemaSelector.java
rename to src/main/java/com/expedia/dsp/data/squeeze/SchemaSelector.java
index b3ae25f..1d9b7e0 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/SchemaSelector.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/SchemaSelector.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze;
+package com.expedia.dsp.data.squeeze;
public interface SchemaSelector {
public String getSchemaJSON();
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/Utility.java b/src/main/java/com/expedia/dsp/data/squeeze/Utility.java
similarity index 96%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/Utility.java
rename to src/main/java/com/expedia/dsp/data/squeeze/Utility.java
index ce0d8be..1953c5c 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/Utility.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/Utility.java
@@ -1,6 +1,6 @@
-package com.expedia.edw.data.squeeze;
+package com.expedia.dsp.data.squeeze;
-import com.expedia.edw.data.squeeze.models.CompactionResponse;
+import com.expedia.dsp.data.squeeze.models.CompactionResponse;
import lombok.extern.slf4j.Slf4j;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/BaseCompactionManagerImpl.java b/src/main/java/com/expedia/dsp/data/squeeze/impl/BaseCompactionManagerImpl.java
similarity index 85%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/BaseCompactionManagerImpl.java
rename to src/main/java/com/expedia/dsp/data/squeeze/impl/BaseCompactionManagerImpl.java
index b96b637..c335c24 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/BaseCompactionManagerImpl.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/impl/BaseCompactionManagerImpl.java
@@ -1,7 +1,8 @@
-package com.expedia.edw.data.squeeze.impl;
+package com.expedia.dsp.data.squeeze.impl;
+
+import com.expedia.dsp.data.squeeze.CompactionManager;
+import com.expedia.dsp.data.squeeze.models.CompactionCriteria;
-import com.expedia.edw.data.squeeze.CompactionManager;
-import com.expedia.edw.data.squeeze.models.CompactionCriteria;
import org.apache.commons.lang3.Validate;
import org.apache.hadoop.conf.Configuration;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/CombineFileWritable.java b/src/main/java/com/expedia/dsp/data/squeeze/impl/CombineFileWritable.java
similarity index 97%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/CombineFileWritable.java
rename to src/main/java/com/expedia/dsp/data/squeeze/impl/CombineFileWritable.java
index 07c852f..84f8b1e 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/CombineFileWritable.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/impl/CombineFileWritable.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.impl;
+package com.expedia.dsp.data.squeeze.impl;
import lombok.Getter;
import org.apache.hadoop.io.Text;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/CompactionManagerImpl.java b/src/main/java/com/expedia/dsp/data/squeeze/impl/CompactionManagerImpl.java
similarity index 90%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/CompactionManagerImpl.java
rename to src/main/java/com/expedia/dsp/data/squeeze/impl/CompactionManagerImpl.java
index 2cc8865..a0f442e 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/CompactionManagerImpl.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/impl/CompactionManagerImpl.java
@@ -1,23 +1,23 @@
-package com.expedia.edw.data.squeeze.impl;
+package com.expedia.dsp.data.squeeze.impl;
-import com.expedia.edw.data.squeeze.CompactionManager;
-import com.expedia.edw.data.squeeze.CompactionManagerFactory;
-import com.expedia.edw.data.squeeze.SchemaSelector;
-import com.expedia.edw.data.squeeze.impl.orc.OrcCombineFileInputFormat;
-import com.expedia.edw.data.squeeze.impl.text.TextCombineFileInputFormat;
-import com.expedia.edw.data.squeeze.mappers.AvroCompactionMapper;
-import com.expedia.edw.data.squeeze.mappers.BytesWritableCompactionMapper;
-import com.expedia.edw.data.squeeze.mappers.OrcCompactionMapper;
-import com.expedia.edw.data.squeeze.mappers.SeqCompactionMapper;
-import com.expedia.edw.data.squeeze.mappers.TextCompactionMapper;
-import com.expedia.edw.data.squeeze.models.CompactionCriteria;
-import com.expedia.edw.data.squeeze.models.CompactionResponse;
-import com.expedia.edw.data.squeeze.models.FilePaths;
-import com.expedia.edw.data.squeeze.models.FileType;
-import com.expedia.edw.data.squeeze.reducers.AvroCompactionReducer;
-import com.expedia.edw.data.squeeze.reducers.BytesWritableCompactionReducer;
-import com.expedia.edw.data.squeeze.reducers.OrcCompactionReducer;
-import com.expedia.edw.data.squeeze.reducers.TextCompactionReducer;
+import com.expedia.dsp.data.squeeze.CompactionManager;
+import com.expedia.dsp.data.squeeze.CompactionManagerFactory;
+import com.expedia.dsp.data.squeeze.impl.orc.OrcCombineFileInputFormat;
+import com.expedia.dsp.data.squeeze.mappers.AvroCompactionMapper;
+import com.expedia.dsp.data.squeeze.mappers.OrcCompactionMapper;
+import com.expedia.dsp.data.squeeze.mappers.SeqCompactionMapper;
+import com.expedia.dsp.data.squeeze.mappers.TextCompactionMapper;
+import com.expedia.dsp.data.squeeze.models.CompactionCriteria;
+import com.expedia.dsp.data.squeeze.models.FilePaths;
+import com.expedia.dsp.data.squeeze.models.FileType;
+import com.expedia.dsp.data.squeeze.reducers.OrcCompactionReducer;
+import com.expedia.dsp.data.squeeze.SchemaSelector;
+import com.expedia.dsp.data.squeeze.impl.text.TextCombineFileInputFormat;
+import com.expedia.dsp.data.squeeze.mappers.BytesWritableCompactionMapper;
+import com.expedia.dsp.data.squeeze.models.CompactionResponse;
+import com.expedia.dsp.data.squeeze.reducers.AvroCompactionReducer;
+import com.expedia.dsp.data.squeeze.reducers.BytesWritableCompactionReducer;
+import com.expedia.dsp.data.squeeze.reducers.TextCompactionReducer;
import lombok.extern.slf4j.Slf4j;
@@ -56,8 +56,9 @@
import java.util.List;
import java.util.UUID;
-import static com.expedia.edw.data.squeeze.models.FileType.AVRO;
-import static com.expedia.edw.data.squeeze.models.FileType.ORC;
+import static com.expedia.dsp.data.squeeze.models.FileType.AVRO;
+import static com.expedia.dsp.data.squeeze.models.FileType.ORC;
+
/**
* Implementation for {@link CompactionManager}.
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/CompactionManagerInPlaceImpl.java b/src/main/java/com/expedia/dsp/data/squeeze/impl/CompactionManagerInPlaceImpl.java
similarity index 92%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/CompactionManagerInPlaceImpl.java
rename to src/main/java/com/expedia/dsp/data/squeeze/impl/CompactionManagerInPlaceImpl.java
index e1fa8e9..c5ce0aa 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/CompactionManagerInPlaceImpl.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/impl/CompactionManagerInPlaceImpl.java
@@ -1,8 +1,8 @@
-package com.expedia.edw.data.squeeze.impl;
+package com.expedia.dsp.data.squeeze.impl;
-import com.expedia.edw.data.squeeze.CompactionManager;
-import com.expedia.edw.data.squeeze.models.CompactionCriteria;
-import com.expedia.edw.data.squeeze.models.CompactionResponse;
+import com.expedia.dsp.data.squeeze.CompactionManager;
+import com.expedia.dsp.data.squeeze.models.CompactionCriteria;
+import com.expedia.dsp.data.squeeze.models.CompactionResponse;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.Validate;
import org.apache.hadoop.conf.Configuration;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/DataSkew.java b/src/main/java/com/expedia/dsp/data/squeeze/impl/DataSkew.java
similarity index 96%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/DataSkew.java
rename to src/main/java/com/expedia/dsp/data/squeeze/impl/DataSkew.java
index 00e2132..af6f549 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/DataSkew.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/impl/DataSkew.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.impl;
+package com.expedia.dsp.data.squeeze.impl;
import lombok.extern.slf4j.Slf4j;
import org.apache.hadoop.io.Text;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/FileManager.java b/src/main/java/com/expedia/dsp/data/squeeze/impl/FileManager.java
similarity index 96%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/FileManager.java
rename to src/main/java/com/expedia/dsp/data/squeeze/impl/FileManager.java
index 1809ec9..209e6b0 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/FileManager.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/impl/FileManager.java
@@ -1,8 +1,9 @@
-package com.expedia.edw.data.squeeze.impl;
+package com.expedia.dsp.data.squeeze.impl;
+
+import com.expedia.dsp.data.squeeze.CompactionManagerFactory;
+import com.expedia.dsp.data.squeeze.models.FilePaths;
+import com.expedia.dsp.data.squeeze.models.FileType;
-import com.expedia.edw.data.squeeze.CompactionManagerFactory;
-import com.expedia.edw.data.squeeze.models.FilePaths;
-import com.expedia.edw.data.squeeze.models.FileType;
import lombok.extern.slf4j.Slf4j;
import net.sf.jmimemagic.Magic;
import net.sf.jmimemagic.MagicMatchNotFoundException;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/JobRunner.java b/src/main/java/com/expedia/dsp/data/squeeze/impl/JobRunner.java
similarity index 93%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/JobRunner.java
rename to src/main/java/com/expedia/dsp/data/squeeze/impl/JobRunner.java
index 031c499..8825b95 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/JobRunner.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/impl/JobRunner.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.impl;
+package com.expedia.dsp.data.squeeze.impl;
import lombok.extern.slf4j.Slf4j;
import org.apache.hadoop.conf.Configured;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/SchemaSelectorImpl.java b/src/main/java/com/expedia/dsp/data/squeeze/impl/SchemaSelectorImpl.java
similarity index 87%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/SchemaSelectorImpl.java
rename to src/main/java/com/expedia/dsp/data/squeeze/impl/SchemaSelectorImpl.java
index 7ae6281..0386cc9 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/SchemaSelectorImpl.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/impl/SchemaSelectorImpl.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.impl;
+package com.expedia.dsp.data.squeeze.impl;
import java.io.IOException;
@@ -7,9 +7,9 @@
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
-import com.expedia.edw.data.squeeze.SchemaSelector;
-import com.expedia.edw.data.squeeze.models.CompactionCriteria;
-import com.expedia.edw.data.squeeze.models.FileType;
+import com.expedia.dsp.data.squeeze.SchemaSelector;
+import com.expedia.dsp.data.squeeze.models.CompactionCriteria;
+import com.expedia.dsp.data.squeeze.models.FileType;
import lombok.extern.slf4j.Slf4j;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/orc/OrcCombineFileInputFormat.java b/src/main/java/com/expedia/dsp/data/squeeze/impl/orc/OrcCombineFileInputFormat.java
similarity index 91%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/orc/OrcCombineFileInputFormat.java
rename to src/main/java/com/expedia/dsp/data/squeeze/impl/orc/OrcCombineFileInputFormat.java
index b671728..8f763d2 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/orc/OrcCombineFileInputFormat.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/impl/orc/OrcCombineFileInputFormat.java
@@ -1,6 +1,6 @@
-package com.expedia.edw.data.squeeze.impl.orc;
+package com.expedia.dsp.data.squeeze.impl.orc;
-import com.expedia.edw.data.squeeze.impl.CombineFileWritable;
+import com.expedia.dsp.data.squeeze.impl.CombineFileWritable;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.mapreduce.InputSplit;
import org.apache.hadoop.mapreduce.JobContext;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/orc/OrcRecordReader.java b/src/main/java/com/expedia/dsp/data/squeeze/impl/orc/OrcRecordReader.java
similarity index 97%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/orc/OrcRecordReader.java
rename to src/main/java/com/expedia/dsp/data/squeeze/impl/orc/OrcRecordReader.java
index 0c0769e..7f8cca4 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/orc/OrcRecordReader.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/impl/orc/OrcRecordReader.java
@@ -1,6 +1,7 @@
-package com.expedia.edw.data.squeeze.impl.orc;
+package com.expedia.dsp.data.squeeze.impl.orc;
+
+import com.expedia.dsp.data.squeeze.impl.CombineFileWritable;
-import com.expedia.edw.data.squeeze.impl.CombineFileWritable;
import lombok.extern.slf4j.Slf4j;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/seq/SeqCombineFileInputFormat.java b/src/main/java/com/expedia/dsp/data/squeeze/impl/seq/SeqCombineFileInputFormat.java
similarity index 91%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/seq/SeqCombineFileInputFormat.java
rename to src/main/java/com/expedia/dsp/data/squeeze/impl/seq/SeqCombineFileInputFormat.java
index efd4269..f135b5f 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/seq/SeqCombineFileInputFormat.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/impl/seq/SeqCombineFileInputFormat.java
@@ -1,6 +1,7 @@
-package com.expedia.edw.data.squeeze.impl.seq;
+package com.expedia.dsp.data.squeeze.impl.seq;
+
+import com.expedia.dsp.data.squeeze.impl.CombineFileWritable;
-import com.expedia.edw.data.squeeze.impl.CombineFileWritable;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.InputSplit;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/seq/SeqRecordReader.java b/src/main/java/com/expedia/dsp/data/squeeze/impl/seq/SeqRecordReader.java
similarity index 97%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/seq/SeqRecordReader.java
rename to src/main/java/com/expedia/dsp/data/squeeze/impl/seq/SeqRecordReader.java
index 6b33eb8..cfb613b 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/seq/SeqRecordReader.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/impl/seq/SeqRecordReader.java
@@ -1,6 +1,6 @@
-package com.expedia.edw.data.squeeze.impl.seq;
+package com.expedia.dsp.data.squeeze.impl.seq;
-import com.expedia.edw.data.squeeze.impl.CombineFileWritable;
+import com.expedia.dsp.data.squeeze.impl.CombineFileWritable;
import lombok.extern.slf4j.Slf4j;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/text/TextCombineFileInputFormat.java b/src/main/java/com/expedia/dsp/data/squeeze/impl/text/TextCombineFileInputFormat.java
similarity index 91%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/text/TextCombineFileInputFormat.java
rename to src/main/java/com/expedia/dsp/data/squeeze/impl/text/TextCombineFileInputFormat.java
index c3c52ef..8dc3401 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/text/TextCombineFileInputFormat.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/impl/text/TextCombineFileInputFormat.java
@@ -1,6 +1,7 @@
-package com.expedia.edw.data.squeeze.impl.text;
+package com.expedia.dsp.data.squeeze.impl.text;
+
+import com.expedia.dsp.data.squeeze.impl.CombineFileWritable;
-import com.expedia.edw.data.squeeze.impl.CombineFileWritable;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.InputSplit;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/text/TextRecordReader.java b/src/main/java/com/expedia/dsp/data/squeeze/impl/text/TextRecordReader.java
similarity index 97%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/text/TextRecordReader.java
rename to src/main/java/com/expedia/dsp/data/squeeze/impl/text/TextRecordReader.java
index 02b976e..ac2c805 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/impl/text/TextRecordReader.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/impl/text/TextRecordReader.java
@@ -1,6 +1,7 @@
-package com.expedia.edw.data.squeeze.impl.text;
+package com.expedia.dsp.data.squeeze.impl.text;
+
+import com.expedia.dsp.data.squeeze.impl.CombineFileWritable;
-import com.expedia.edw.data.squeeze.impl.CombineFileWritable;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hadoop.fs.FileSystem;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/AvroCompactionMapper.java b/src/main/java/com/expedia/dsp/data/squeeze/mappers/AvroCompactionMapper.java
similarity index 98%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/AvroCompactionMapper.java
rename to src/main/java/com/expedia/dsp/data/squeeze/mappers/AvroCompactionMapper.java
index 041083d..4e1a142 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/AvroCompactionMapper.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/mappers/AvroCompactionMapper.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.mappers;
+package com.expedia.dsp.data.squeeze.mappers;
import java.io.IOException;
import java.util.HashMap;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/BaseMapper.java b/src/main/java/com/expedia/dsp/data/squeeze/mappers/BaseMapper.java
similarity index 97%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/BaseMapper.java
rename to src/main/java/com/expedia/dsp/data/squeeze/mappers/BaseMapper.java
index 7496ef9..6bf3146 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/BaseMapper.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/mappers/BaseMapper.java
@@ -1,6 +1,7 @@
-package com.expedia.edw.data.squeeze.mappers;
+package com.expedia.dsp.data.squeeze.mappers;
+
+import com.expedia.dsp.data.squeeze.impl.DataSkew;
-import com.expedia.edw.data.squeeze.impl.DataSkew;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang.StringUtils;
import org.apache.hadoop.conf.Configuration;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/BytesWritableCompactionMapper.java b/src/main/java/com/expedia/dsp/data/squeeze/mappers/BytesWritableCompactionMapper.java
similarity index 98%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/BytesWritableCompactionMapper.java
rename to src/main/java/com/expedia/dsp/data/squeeze/mappers/BytesWritableCompactionMapper.java
index 500a16e..d69c5f3 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/BytesWritableCompactionMapper.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/mappers/BytesWritableCompactionMapper.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.mappers;
+package com.expedia.dsp.data.squeeze.mappers;
import lombok.extern.slf4j.Slf4j;
import org.apache.hadoop.conf.Configuration;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/OrcCompactionMapper.java b/src/main/java/com/expedia/dsp/data/squeeze/mappers/OrcCompactionMapper.java
similarity index 91%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/OrcCompactionMapper.java
rename to src/main/java/com/expedia/dsp/data/squeeze/mappers/OrcCompactionMapper.java
index 68e43f9..895709d 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/OrcCompactionMapper.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/mappers/OrcCompactionMapper.java
@@ -1,6 +1,7 @@
-package com.expedia.edw.data.squeeze.mappers;
+package com.expedia.dsp.data.squeeze.mappers;
+
+import com.expedia.dsp.data.squeeze.impl.CombineFileWritable;
-import com.expedia.edw.data.squeeze.impl.CombineFileWritable;
import lombok.extern.slf4j.Slf4j;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/SeqCompactionMapper.java b/src/main/java/com/expedia/dsp/data/squeeze/mappers/SeqCompactionMapper.java
similarity index 95%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/SeqCompactionMapper.java
rename to src/main/java/com/expedia/dsp/data/squeeze/mappers/SeqCompactionMapper.java
index 36326ce..f90afd0 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/SeqCompactionMapper.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/mappers/SeqCompactionMapper.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.mappers;
+package com.expedia.dsp.data.squeeze.mappers;
import lombok.extern.slf4j.Slf4j;
import org.apache.hadoop.io.Text;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/TextCompactionMapper.java b/src/main/java/com/expedia/dsp/data/squeeze/mappers/TextCompactionMapper.java
similarity index 89%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/TextCompactionMapper.java
rename to src/main/java/com/expedia/dsp/data/squeeze/mappers/TextCompactionMapper.java
index 2f4cb1b..7800659 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/mappers/TextCompactionMapper.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/mappers/TextCompactionMapper.java
@@ -1,6 +1,7 @@
-package com.expedia.edw.data.squeeze.mappers;
+package com.expedia.dsp.data.squeeze.mappers;
+
+import com.expedia.dsp.data.squeeze.impl.CombineFileWritable;
-import com.expedia.edw.data.squeeze.impl.CombineFileWritable;
import lombok.extern.slf4j.Slf4j;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/models/CompactionCriteria.java b/src/main/java/com/expedia/dsp/data/squeeze/models/CompactionCriteria.java
similarity index 99%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/models/CompactionCriteria.java
rename to src/main/java/com/expedia/dsp/data/squeeze/models/CompactionCriteria.java
index 22ee1e0..b1904f4 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/models/CompactionCriteria.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/models/CompactionCriteria.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.models;
+package com.expedia.dsp.data.squeeze.models;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/models/CompactionResponse.java b/src/main/java/com/expedia/dsp/data/squeeze/models/CompactionResponse.java
similarity index 95%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/models/CompactionResponse.java
rename to src/main/java/com/expedia/dsp/data/squeeze/models/CompactionResponse.java
index 6f6d1c2..51c62f7 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/models/CompactionResponse.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/models/CompactionResponse.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.models;
+package com.expedia.dsp.data.squeeze.models;
import lombok.Getter;
import lombok.Setter;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/models/FilePaths.java b/src/main/java/com/expedia/dsp/data/squeeze/models/FilePaths.java
similarity index 95%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/models/FilePaths.java
rename to src/main/java/com/expedia/dsp/data/squeeze/models/FilePaths.java
index 98ad172..5c66d7f 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/models/FilePaths.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/models/FilePaths.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.models;
+package com.expedia.dsp.data.squeeze.models;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/models/FileType.java b/src/main/java/com/expedia/dsp/data/squeeze/models/FileType.java
similarity index 87%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/models/FileType.java
rename to src/main/java/com/expedia/dsp/data/squeeze/models/FileType.java
index 9dc590a..bf91b19 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/models/FileType.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/models/FileType.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.models;
+package com.expedia.dsp.data.squeeze.models;
import lombok.Getter;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/AvroCompactionReducer.java b/src/main/java/com/expedia/dsp/data/squeeze/reducers/AvroCompactionReducer.java
similarity index 96%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/AvroCompactionReducer.java
rename to src/main/java/com/expedia/dsp/data/squeeze/reducers/AvroCompactionReducer.java
index 6be1fc6..84b9a7e 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/AvroCompactionReducer.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/reducers/AvroCompactionReducer.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.reducers;
+package com.expedia.dsp.data.squeeze.reducers;
import java.io.IOException;
import java.net.URI;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/BaseReducer.java b/src/main/java/com/expedia/dsp/data/squeeze/reducers/BaseReducer.java
similarity index 97%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/BaseReducer.java
rename to src/main/java/com/expedia/dsp/data/squeeze/reducers/BaseReducer.java
index 4d20dba..2ae6381 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/BaseReducer.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/reducers/BaseReducer.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.reducers;
+package com.expedia.dsp.data.squeeze.reducers;
import lombok.extern.slf4j.Slf4j;
import org.apache.hadoop.conf.Configuration;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/BytesWritableCompactionReducer.java b/src/main/java/com/expedia/dsp/data/squeeze/reducers/BytesWritableCompactionReducer.java
similarity index 97%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/BytesWritableCompactionReducer.java
rename to src/main/java/com/expedia/dsp/data/squeeze/reducers/BytesWritableCompactionReducer.java
index 31054d5..20505d9 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/BytesWritableCompactionReducer.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/reducers/BytesWritableCompactionReducer.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.reducers;
+package com.expedia.dsp.data.squeeze.reducers;
import lombok.extern.slf4j.Slf4j;
import org.apache.hadoop.conf.Configuration;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/OrcCompactionReducer.java b/src/main/java/com/expedia/dsp/data/squeeze/reducers/OrcCompactionReducer.java
similarity index 94%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/OrcCompactionReducer.java
rename to src/main/java/com/expedia/dsp/data/squeeze/reducers/OrcCompactionReducer.java
index 5121efb..74bfa06 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/OrcCompactionReducer.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/reducers/OrcCompactionReducer.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.reducers;
+package com.expedia.dsp.data.squeeze.reducers;
import lombok.extern.slf4j.Slf4j;
import org.apache.hadoop.io.NullWritable;
diff --git a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/TextCompactionReducer.java b/src/main/java/com/expedia/dsp/data/squeeze/reducers/TextCompactionReducer.java
similarity index 94%
rename from dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/TextCompactionReducer.java
rename to src/main/java/com/expedia/dsp/data/squeeze/reducers/TextCompactionReducer.java
index bf6e985..a134b7d 100644
--- a/dataSqueeze-manager/src/main/java/com/expedia/edw/data/squeeze/reducers/TextCompactionReducer.java
+++ b/src/main/java/com/expedia/dsp/data/squeeze/reducers/TextCompactionReducer.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.reducers;
+package com.expedia.dsp.data.squeeze.reducers;
import lombok.extern.slf4j.Slf4j;
import org.apache.hadoop.io.NullWritable;
diff --git a/dataSqueeze-manager/src/main/resources/compaction.properties b/src/main/resources/compaction.properties
similarity index 100%
rename from dataSqueeze-manager/src/main/resources/compaction.properties
rename to src/main/resources/compaction.properties
diff --git a/dataSqueeze-manager/src/main/resources/log4j.properties b/src/main/resources/log4j.properties
similarity index 100%
rename from dataSqueeze-manager/src/main/resources/log4j.properties
rename to src/main/resources/log4j.properties
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/CompactionManagerFactoryTest.java b/src/test/java/com/expedia/dsp/data/squeeze/CompactionManagerFactoryTest.java
similarity index 94%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/CompactionManagerFactoryTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/CompactionManagerFactoryTest.java
index 54df008..5d02312 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/CompactionManagerFactoryTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/CompactionManagerFactoryTest.java
@@ -1,7 +1,8 @@
-package com.expedia.edw.data.squeeze;
+package com.expedia.dsp.data.squeeze;
+
+import com.expedia.dsp.data.squeeze.impl.CompactionManagerImpl;
+import com.expedia.dsp.data.squeeze.impl.CompactionManagerInPlaceImpl;
-import com.expedia.edw.data.squeeze.impl.CompactionManagerImpl;
-import com.expedia.edw.data.squeeze.impl.CompactionManagerInPlaceImpl;
import org.junit.Test;
import java.util.HashMap;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/DataSkewTest.java b/src/test/java/com/expedia/dsp/data/squeeze/DataSkewTest.java
similarity index 90%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/DataSkewTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/DataSkewTest.java
index 3b05d09..06742d5 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/DataSkewTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/DataSkewTest.java
@@ -1,6 +1,7 @@
-package com.expedia.edw.data.squeeze;
+package com.expedia.dsp.data.squeeze;
+
+import com.expedia.dsp.data.squeeze.impl.DataSkew;
-import com.expedia.edw.data.squeeze.impl.DataSkew;
import org.apache.hadoop.io.Text;
import org.json.JSONObject;
import org.junit.Test;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/UtilityTest.java b/src/test/java/com/expedia/dsp/data/squeeze/UtilityTest.java
similarity index 93%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/UtilityTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/UtilityTest.java
index 59ea72e..67745fb 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/UtilityTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/UtilityTest.java
@@ -1,7 +1,7 @@
-package com.expedia.edw.data.squeeze;
+package com.expedia.dsp.data.squeeze;
-import com.expedia.edw.data.squeeze.models.CompactionResponse;
-import com.expedia.edw.data.squeeze.models.FileType;
+import com.expedia.dsp.data.squeeze.models.CompactionResponse;
+import com.expedia.dsp.data.squeeze.models.FileType;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/CompactionManagerImplTest.java b/src/test/java/com/expedia/dsp/data/squeeze/impl/CompactionManagerImplTest.java
similarity index 98%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/CompactionManagerImplTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/impl/CompactionManagerImplTest.java
index 2b9da76..7b82350 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/CompactionManagerImplTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/impl/CompactionManagerImplTest.java
@@ -1,10 +1,10 @@
-package com.expedia.edw.data.squeeze.impl;
+package com.expedia.dsp.data.squeeze.impl;
-import com.expedia.edw.data.squeeze.CompactionManagerFactory;
-import com.expedia.edw.data.squeeze.models.CompactionCriteria;
-import com.expedia.edw.data.squeeze.models.CompactionResponse;
-import com.expedia.edw.data.squeeze.models.FilePaths;
-import com.expedia.edw.data.squeeze.models.FileType;
+import com.expedia.dsp.data.squeeze.CompactionManagerFactory;
+import com.expedia.dsp.data.squeeze.models.CompactionCriteria;
+import com.expedia.dsp.data.squeeze.models.CompactionResponse;
+import com.expedia.dsp.data.squeeze.models.FilePaths;
+import com.expedia.dsp.data.squeeze.models.FileType;
import net.sf.jmimemagic.Magic;
import net.sf.jmimemagic.MagicMatch;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/CompactionManagerInPlaceImplTest.java b/src/test/java/com/expedia/dsp/data/squeeze/impl/CompactionManagerInPlaceImplTest.java
similarity index 97%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/CompactionManagerInPlaceImplTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/impl/CompactionManagerInPlaceImplTest.java
index 943c79b..3014e04 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/CompactionManagerInPlaceImplTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/impl/CompactionManagerInPlaceImplTest.java
@@ -1,10 +1,10 @@
-package com.expedia.edw.data.squeeze.impl;
+package com.expedia.dsp.data.squeeze.impl;
-import com.expedia.edw.data.squeeze.CompactionManagerFactory;
-import com.expedia.edw.data.squeeze.models.CompactionCriteria;
-import com.expedia.edw.data.squeeze.models.CompactionResponse;
-import com.expedia.edw.data.squeeze.models.FilePaths;
-import com.expedia.edw.data.squeeze.models.FileType;
+import com.expedia.dsp.data.squeeze.CompactionManagerFactory;
+import com.expedia.dsp.data.squeeze.models.CompactionCriteria;
+import com.expedia.dsp.data.squeeze.models.CompactionResponse;
+import com.expedia.dsp.data.squeeze.models.FilePaths;
+import com.expedia.dsp.data.squeeze.models.FileType;
import net.sf.jmimemagic.Magic;
import net.sf.jmimemagic.MagicMatch;
import org.apache.commons.math3.util.Pair;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/FileManagerTest.java b/src/test/java/com/expedia/dsp/data/squeeze/impl/FileManagerTest.java
similarity index 98%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/FileManagerTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/impl/FileManagerTest.java
index 7b543b5..f4c55d5 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/FileManagerTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/impl/FileManagerTest.java
@@ -1,8 +1,8 @@
-package com.expedia.edw.data.squeeze.impl;
+package com.expedia.dsp.data.squeeze.impl;
-import com.expedia.edw.data.squeeze.CompactionManagerFactory;
-import com.expedia.edw.data.squeeze.models.FilePaths;
-import com.expedia.edw.data.squeeze.models.FileType;
+import com.expedia.dsp.data.squeeze.CompactionManagerFactory;
+import com.expedia.dsp.data.squeeze.models.FilePaths;
+import com.expedia.dsp.data.squeeze.models.FileType;
import com.google.common.collect.ImmutableList;
import net.sf.jmimemagic.Magic;
import net.sf.jmimemagic.MagicMatch;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/JobRunnerTest.java b/src/test/java/com/expedia/dsp/data/squeeze/impl/JobRunnerTest.java
similarity index 95%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/JobRunnerTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/impl/JobRunnerTest.java
index 968d43d..0168e19 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/JobRunnerTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/impl/JobRunnerTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.impl;
+package com.expedia.dsp.data.squeeze.impl;
import org.apache.hadoop.mapreduce.Job;
import org.junit.Test;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/SchemaSelectorImplTest.java b/src/test/java/com/expedia/dsp/data/squeeze/impl/SchemaSelectorImplTest.java
similarity index 97%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/SchemaSelectorImplTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/impl/SchemaSelectorImplTest.java
index e8dc9ba..2d78189 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/SchemaSelectorImplTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/impl/SchemaSelectorImplTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.impl;
+package com.expedia.dsp.data.squeeze.impl;
import static org.mockito.Matchers.any;
import static org.powermock.api.mockito.PowerMockito.mock;
@@ -22,7 +22,7 @@
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.PowerMockRunner;
-import com.expedia.edw.data.squeeze.models.CompactionCriteria;
+import com.expedia.dsp.data.squeeze.models.CompactionCriteria;
/**
* Tests for {@link SchemaSelectorImpl}
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/orc/OrcCombineFileInputFormatTest.java b/src/test/java/com/expedia/dsp/data/squeeze/impl/orc/OrcCombineFileInputFormatTest.java
similarity index 97%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/orc/OrcCombineFileInputFormatTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/impl/orc/OrcCombineFileInputFormatTest.java
index 07663f7..91e0eec 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/orc/OrcCombineFileInputFormatTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/impl/orc/OrcCombineFileInputFormatTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.impl.orc;
+package com.expedia.dsp.data.squeeze.impl.orc;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.mapreduce.JobContext;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/orc/OrcRecordReaderTest.java b/src/test/java/com/expedia/dsp/data/squeeze/impl/orc/OrcRecordReaderTest.java
similarity index 97%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/orc/OrcRecordReaderTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/impl/orc/OrcRecordReaderTest.java
index 53630fc..663f8b0 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/orc/OrcRecordReaderTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/impl/orc/OrcRecordReaderTest.java
@@ -1,6 +1,7 @@
-package com.expedia.edw.data.squeeze.impl.orc;
+package com.expedia.dsp.data.squeeze.impl.orc;
+
+import com.expedia.dsp.data.squeeze.impl.CombineFileWritable;
-import com.expedia.edw.data.squeeze.impl.CombineFileWritable;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/seq/SeqCombineFileInputFormatTest.java b/src/test/java/com/expedia/dsp/data/squeeze/impl/seq/SeqCombineFileInputFormatTest.java
similarity index 97%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/seq/SeqCombineFileInputFormatTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/impl/seq/SeqCombineFileInputFormatTest.java
index b1aaef7..7a29618 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/seq/SeqCombineFileInputFormatTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/impl/seq/SeqCombineFileInputFormatTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.impl.seq;
+package com.expedia.dsp.data.squeeze.impl.seq;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.mapreduce.JobContext;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/text/TextCombineFileInputFormatTest.java b/src/test/java/com/expedia/dsp/data/squeeze/impl/text/TextCombineFileInputFormatTest.java
similarity index 97%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/text/TextCombineFileInputFormatTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/impl/text/TextCombineFileInputFormatTest.java
index 3d9f0ee..6f5104c 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/text/TextCombineFileInputFormatTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/impl/text/TextCombineFileInputFormatTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.impl.text;
+package com.expedia.dsp.data.squeeze.impl.text;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.mapreduce.JobContext;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/text/TextRecordReaderTest.java b/src/test/java/com/expedia/dsp/data/squeeze/impl/text/TextRecordReaderTest.java
similarity index 98%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/text/TextRecordReaderTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/impl/text/TextRecordReaderTest.java
index 6603543..c634141 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/impl/text/TextRecordReaderTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/impl/text/TextRecordReaderTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.impl.text;
+package com.expedia.dsp.data.squeeze.impl.text;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FSDataInputStream;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/AvroCompactionMapperTest.java b/src/test/java/com/expedia/dsp/data/squeeze/mappers/AvroCompactionMapperTest.java
similarity index 99%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/AvroCompactionMapperTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/mappers/AvroCompactionMapperTest.java
index c1a8458..d92dbb5 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/AvroCompactionMapperTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/mappers/AvroCompactionMapperTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.mappers;
+package com.expedia.dsp.data.squeeze.mappers;
import static org.mockito.Matchers.any;
import static org.powermock.api.mockito.PowerMockito.mock;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/BaseMapperTest.java b/src/test/java/com/expedia/dsp/data/squeeze/mappers/BaseMapperTest.java
similarity index 98%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/BaseMapperTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/mappers/BaseMapperTest.java
index 040e23b..01e541e 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/BaseMapperTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/mappers/BaseMapperTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.mappers;
+package com.expedia.dsp.data.squeeze.mappers;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileStatus;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/BytesWritableCompactionMapperTest.java b/src/test/java/com/expedia/dsp/data/squeeze/mappers/BytesWritableCompactionMapperTest.java
similarity index 98%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/BytesWritableCompactionMapperTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/mappers/BytesWritableCompactionMapperTest.java
index 2bd16ed..2304fa8 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/BytesWritableCompactionMapperTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/mappers/BytesWritableCompactionMapperTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.mappers;
+package com.expedia.dsp.data.squeeze.mappers;
import org.apache.hadoop.conf.Configuration;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/OrcCompactionMapperTest.java b/src/test/java/com/expedia/dsp/data/squeeze/mappers/OrcCompactionMapperTest.java
similarity index 97%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/OrcCompactionMapperTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/mappers/OrcCompactionMapperTest.java
index 612e75d..8a82bb0 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/OrcCompactionMapperTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/mappers/OrcCompactionMapperTest.java
@@ -1,7 +1,8 @@
-package com.expedia.edw.data.squeeze.mappers;
+package com.expedia.dsp.data.squeeze.mappers;
-import com.expedia.edw.data.squeeze.impl.CombineFileWritable;
+import com.expedia.dsp.data.squeeze.impl.CombineFileWritable;
+
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileStatus;
import org.apache.hadoop.fs.FileSystem;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/SeqCompactionMapperTest.java b/src/test/java/com/expedia/dsp/data/squeeze/mappers/SeqCompactionMapperTest.java
similarity index 98%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/SeqCompactionMapperTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/mappers/SeqCompactionMapperTest.java
index 788a0f8..3df22ce 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/SeqCompactionMapperTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/mappers/SeqCompactionMapperTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.mappers;
+package com.expedia.dsp.data.squeeze.mappers;
import static org.mockito.Matchers.any;
import static org.powermock.api.mockito.PowerMockito.mock;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/TextCompactionMapperTest.java b/src/test/java/com/expedia/dsp/data/squeeze/mappers/TextCompactionMapperTest.java
similarity index 97%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/TextCompactionMapperTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/mappers/TextCompactionMapperTest.java
index 986f038..a834013 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/mappers/TextCompactionMapperTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/mappers/TextCompactionMapperTest.java
@@ -1,7 +1,7 @@
-package com.expedia.edw.data.squeeze.mappers;
+package com.expedia.dsp.data.squeeze.mappers;
-import com.expedia.edw.data.squeeze.impl.CombineFileWritable;
+import com.expedia.dsp.data.squeeze.impl.CombineFileWritable;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileStatus;
import org.apache.hadoop.fs.FileSystem;
diff --git a/src/test/java/com/expedia/dsp/data/squeeze/models/CompactionCriteriaTest.java b/src/test/java/com/expedia/dsp/data/squeeze/models/CompactionCriteriaTest.java
new file mode 100644
index 0000000..24ea35e
--- /dev/null
+++ b/src/test/java/com/expedia/dsp/data/squeeze/models/CompactionCriteriaTest.java
@@ -0,0 +1,101 @@
+package com.expedia.dsp.data.squeeze.models;
+
+import org.junit.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNull;
+
+/**
+ * Tests for {@link CompactionCriteria}
+ *
+ * @author Yashraj R. Sontakke
+ */
+public class CompactionCriteriaTest {
+
+ @Test
+ public void testNullThreshold() throws Exception {
+ final Map options = retrieveOptions(null, "1000");
+ CompactionCriteria criteria = new CompactionCriteria(options);
+ assertNull(criteria.getThresholdInBytes());
+ }
+
+ @Test(expected = NumberFormatException.class)
+ public void testWrongThreshold() throws Exception {
+ final Map options = retrieveOptions("1234a", "1000");
+ CompactionCriteria criteria = new CompactionCriteria(options);
+ assertNull(criteria.getThresholdInBytes());
+ }
+
+ @Test
+ public void testNullMaxReducers() throws Exception {
+ final Map options = retrieveOptions("1234", null);
+ CompactionCriteria criteria = new CompactionCriteria(options);
+ assertNull(criteria.getMaxReducers());
+ }
+
+ @Test(expected = NumberFormatException.class)
+ public void testWrongMaxReducers() throws Exception {
+ final Map options = retrieveOptions("1234", "1000a");
+ CompactionCriteria criteria = new CompactionCriteria(options);
+ assertNull(criteria.getMaxReducers());
+ }
+
+ @Test(expected = IllegalArgumentException.class)
+ public void testNoSchemaPath() throws Exception {
+ final Map options = retrieveOptions("1234", "1000");
+ options.put("fileType", "AVRO");
+ options.remove("schemaPath");
+ CompactionCriteria criteria = new CompactionCriteria(options);
+ }
+
+ @Test
+ public void testCriteria() throws Exception {
+ Map options = retrieveOptions("1234", "1000");
+
+ CompactionCriteria criteria = new CompactionCriteria(options);
+ assertEquals("sourcePath", criteria.getSourcePath());
+ assertEquals("targetPath", criteria.getTargetPath());
+ assertEquals(1234L, criteria.getThresholdInBytes(), 0);
+ assertEquals("schemaPath", criteria.getSchemaPath());
+ assertEquals(1000L, criteria.getMaxReducers(), 0);
+ assertNull(criteria.getFileType());
+ options.put("fileType", "AVRO");
+
+ criteria = new CompactionCriteria(options);
+ assertEquals(FileType.AVRO.getValue(), criteria.getFileType());
+
+ criteria = new CompactionCriteria("source", "target", 12345L, 1000L);
+ assertEquals("source", criteria.getSourcePath());
+ assertEquals("target", criteria.getTargetPath());
+ assertEquals(12345L, criteria.getThresholdInBytes(), 0);
+ assertEquals(1000L, criteria.getMaxReducers(), 0);
+
+ criteria = new CompactionCriteria("source", "target", 12345L, 1000L, "ORC", "schemaPath");
+ assertEquals("source", criteria.getSourcePath());
+ assertEquals("target", criteria.getTargetPath());
+ assertEquals(12345L, criteria.getThresholdInBytes(), 0);
+ assertEquals(1000L, criteria.getMaxReducers(), 0);
+ assertEquals("ORC", criteria.getFileType());
+ assertEquals("schemaPath", criteria.getSchemaPath());
+ }
+
+ @Test(expected = IllegalArgumentException.class)
+ public void testUnsupportedFileFormat() {
+ Map options = retrieveOptions("1234", "1000");
+ options.put("fileType", "AVROO");
+ new CompactionCriteria(options);
+ }
+
+ private Map retrieveOptions(final String threshold, final String maxReducers) {
+ final Map options = new HashMap();
+ options.put("sourcePath", "sourcePath");
+ options.put("targetPath", "targetPath");
+ options.put("thresholdInBytes", threshold);
+ options.put("schemaPath", "schemaPath");
+ options.put("maxReducers", maxReducers);
+ return options;
+ }
+}
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/models/CompactionResponseTest.java b/src/test/java/com/expedia/dsp/data/squeeze/models/CompactionResponseTest.java
similarity index 92%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/models/CompactionResponseTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/models/CompactionResponseTest.java
index 0dc4e80..19ab1c4 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/models/CompactionResponseTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/models/CompactionResponseTest.java
@@ -1,8 +1,8 @@
-package com.expedia.edw.data.squeeze.models;
+package com.expedia.dsp.data.squeeze.models;
import org.junit.Test;
-import static com.expedia.edw.data.squeeze.models.FileType.*;
+import static com.expedia.dsp.data.squeeze.models.FileType.*;
import static org.junit.Assert.*;
/**
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/AvroCompactionReducerTest.java b/src/test/java/com/expedia/dsp/data/squeeze/reducers/AvroCompactionReducerTest.java
similarity index 98%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/AvroCompactionReducerTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/reducers/AvroCompactionReducerTest.java
index 0806fed..0b9f1b3 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/AvroCompactionReducerTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/reducers/AvroCompactionReducerTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.reducers;
+package com.expedia.dsp.data.squeeze.reducers;
import static org.powermock.api.mockito.PowerMockito.mock;
import static org.powermock.api.mockito.PowerMockito.when;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/BaseReducerTest.java b/src/test/java/com/expedia/dsp/data/squeeze/reducers/BaseReducerTest.java
similarity index 98%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/BaseReducerTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/reducers/BaseReducerTest.java
index 938a8c8..f065829 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/BaseReducerTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/reducers/BaseReducerTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.reducers;
+package com.expedia.dsp.data.squeeze.reducers;
import com.google.common.collect.ImmutableList;
import org.apache.hadoop.conf.Configuration;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/BytesWritableCompactionReducerTest.java b/src/test/java/com/expedia/dsp/data/squeeze/reducers/BytesWritableCompactionReducerTest.java
similarity index 98%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/BytesWritableCompactionReducerTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/reducers/BytesWritableCompactionReducerTest.java
index 8f8de08..98491e6 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/BytesWritableCompactionReducerTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/reducers/BytesWritableCompactionReducerTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.reducers;
+package com.expedia.dsp.data.squeeze.reducers;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.io.BytesWritable;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/OrcCompactionReducerTest.java b/src/test/java/com/expedia/dsp/data/squeeze/reducers/OrcCompactionReducerTest.java
similarity index 98%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/OrcCompactionReducerTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/reducers/OrcCompactionReducerTest.java
index 3e0d4f7..b2bd492 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/OrcCompactionReducerTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/reducers/OrcCompactionReducerTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.reducers;
+package com.expedia.dsp.data.squeeze.reducers;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Reducer;
diff --git a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/TextCompactionReducerTest.java b/src/test/java/com/expedia/dsp/data/squeeze/reducers/TextCompactionReducerTest.java
similarity index 98%
rename from dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/TextCompactionReducerTest.java
rename to src/test/java/com/expedia/dsp/data/squeeze/reducers/TextCompactionReducerTest.java
index ecb8b6e..7dd0921 100644
--- a/dataSqueeze-manager/src/test/java/com/expedia/edw/data/squeeze/reducers/TextCompactionReducerTest.java
+++ b/src/test/java/com/expedia/dsp/data/squeeze/reducers/TextCompactionReducerTest.java
@@ -1,4 +1,4 @@
-package com.expedia.edw.data.squeeze.reducers;
+package com.expedia.dsp.data.squeeze.reducers;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Reducer;
diff --git a/dataSqueeze-manager/src/test/resources/compaction.properties b/src/test/resources/compaction.properties
similarity index 100%
rename from dataSqueeze-manager/src/test/resources/compaction.properties
rename to src/test/resources/compaction.properties
diff --git a/dataSqueeze-manager/src/test/resources/log4j.properties b/src/test/resources/log4j.properties
similarity index 100%
rename from dataSqueeze-manager/src/test/resources/log4j.properties
rename to src/test/resources/log4j.properties
diff --git a/dataSqueeze-manager/src/test/resources/source/path/text/textfile.txt b/src/test/resources/source/path/text/textfile.txt
similarity index 100%
rename from dataSqueeze-manager/src/test/resources/source/path/text/textfile.txt
rename to src/test/resources/source/path/text/textfile.txt