JPEG processing fails on Amazon Linux

[ptah23]

I am trying to consume images for classification task and the code runs fine on OS X.
I am getting the below stacktrace on Amazon Linux though:

Error in pixReadMemJp2k: function not present
Error in pixReadMem: jp2k: no pix returned
java.io.IOException: Could not decode image from input stream
 at org.datavec.image.loader.NativeImageLoader.asMatrix(NativeImageLoader.java:194)
 at org.datavec.image.loader.NativeImageLoader.asMatrix(NativeImageLoader.java:183)
 at org.datavec.image.recordreader.BaseImageRecordReader.next(BaseImageRecordReader.java:208)
 at org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator.next(RecordReaderDataSetIterator.java:176)
 at org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator.next(RecordReaderDataSetIterator.java:372)
 at org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator.next(RecordReaderDataSetIterator.java:52)
 at org.deeplearning4j.datasets.iterator.AsyncDataSetIterator$IteratorRunnable.run(AsyncDataSetIterator.java:303)
Exception in thread "main" java.lang.RuntimeException: Exception thrown in base iterator
 at org.deeplearning4j.datasets.iterator.AsyncDataSetIterator.next(AsyncDataSetIterator.java:231)
 at org.deeplearning4j.datasets.iterator.AsyncDataSetIterator.next(AsyncDataSetIterator.java:36)
 at org.deeplearning4j.datasets.iterator.MultipleEpochsIterator.next(MultipleEpochsIterator.java:119)
 at org.deeplearning4j.datasets.iterator.MultipleEpochsIterator.next(MultipleEpochsIterator.java:142)
 at org.deeplearning4j.datasets.iterator.MultipleEpochsIterator.next(MultipleEpochsIterator.java:37)
 at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fit(MultiLayerNetwork.java:1012)

I am using Deep Learning AMI - Amazon Linux - 1.5 (MXNet with MKL Support) (ami-41570b32) on AWS EC2.

[saudet]

Could you provide some sample images that are failing to load?

[ptah23]

I have attached some on here

[ptah23]

any progress?

[saudet]

Thanks for the reminder! Was just getting to this. Unfortunately, I am unable to reproduce the error, even on AWS:

Linux ip-172-31-7-26 4.4.19-29.55.amzn1.x86_64 #1 SMP Mon Aug 29 23:29:40 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Is this what you are running?

This line works just fine for me:

INDArray arr = new NativeImageLoader().asMatrix(new File("/path/to/848fb24e-e48c-11e6-8593-7844516347e5.jpg"));
System.out.println(arr.shapeInfoToString());

This outputs as expected:

Rank: 4,Offset: 0
 Order: c Shape: [1,1,390,528],  stride: [205920,205920,528,1]

[ptah23]

No, I am using a later version:
Linux ip-172-31-47-242 4.4.30-32.54.amzn1.x86_64 #1 SMP Thu Nov 10 15:52:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

[ptah23]

Can you try it with the specific AMI Deep Learning AMI - Amazon Linux - 1.5 (MXNet with MKL Support) (ami-41570b32)
ami-41570b32 is the ID. or can you tell me which AMI you are using. does it have CUDA support?

[saudet]

I'll be doing that, but in the meantime could you show me your pom.xml file?

[ptah23]

4.0.0

<groupId>io.regularization</groupId>
<artifactId>leaf-classifier</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
    <!-- Change the nd4j.backend property to nd4j-cuda-7.5-platform or nd4j-cuda-8.0-platform to use CUDA GPUs -->
    <nd4j.backend>nd4j-native-platform</nd4j.backend>
    <!--nd4j.backend>nd4j-cuda-7.5-platform</nd4j.backend-->

    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <shadedClassifier>bin</shadedClassifier>
    <java.version>1.8</java.version>
    <nd4j.version>0.7.2</nd4j.version>
    <dl4j.version>0.7.2</dl4j.version>
    <datavec.version>0.7.2</datavec.version>
    <arbiter.version>0.7.2</arbiter.version>
    <rl4j.version>0.7.2</rl4j.version>
    <guava.version>19.0</guava.version>
    <logback.version>1.1.7</logback.version>
    <jfreechart.version>1.0.13</jfreechart.version>
    <jcommon.version>1.0.23</jcommon.version>
    <maven-shade-plugin.version>2.4.3</maven-shade-plugin.version>
    <exec-maven-plugin.version>1.4.0</exec-maven-plugin.version>
    <maven.minimum.version>3.3.1</maven.minimum.version>
</properties>
<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.nd4j</groupId>
            <artifactId>nd4j-native-platform</artifactId>
            <version>${nd4j.version}</version>
        </dependency>
        <dependency>
            <groupId>org.nd4j</groupId>
            <artifactId>nd4j-cuda-7.5-platform</artifactId>
            <version>${nd4j.version}</version>
        </dependency>
        <dependency>
            <groupId>org.nd4j</groupId>
            <artifactId>nd4j-cuda-8.0-platform</artifactId>
            <version>${nd4j.version}</version>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>


    <!-- ND4J backend. You need one in every DL4J project. Normally define artifactId as either "nd4j-native-platform" or "nd4j-cuda-7.5-platform" -->
    <dependency>
        <groupId>org.nd4j</groupId>
        <artifactId>${nd4j.backend}</artifactId>
    </dependency>

    <!-- Core DL4J functionality -->
    <dependency>
        <groupId>org.deeplearning4j</groupId>
        <artifactId>deeplearning4j-core</artifactId>
        <version>${dl4j.version}</version>
    </dependency>

    <dependency>
        <groupId>org.deeplearning4j</groupId>
        <artifactId>deeplearning4j-nlp</artifactId>
        <version>${dl4j.version}</version>
    </dependency>

    <!-- deeplearning4j-ui is used for HistogramIterationListener + visualization: see http://deeplearning4j.org/visualization -->
    <dependency>
        <groupId>org.deeplearning4j</groupId>
        <artifactId>deeplearning4j-ui_2.10</artifactId>
        <version>${dl4j.version}</version>
    </dependency>

    <!-- Force guava versions for using UI/HistogramIterationListener -->
    <dependency>
        <groupId>com.google.guava</groupId>
        <artifactId>guava</artifactId>
        <version>${guava.version}</version>
    </dependency>

    <!-- datavec-data-codec: used only in video example for loading video data -->
    <dependency>
        <artifactId>datavec-data-codec</artifactId>
        <groupId>org.datavec</groupId>
        <version>${datavec.version}</version>
    </dependency>


</dependencies>
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
        </plugin>
    </plugins>
</build>
<profiles>
    <profile>
        <id>train</id>
        <build>
            <finalName>${project.artifactId}-train</finalName>
            <plugins>
                <plugin>
                    <artifactId>maven-assembly-plugin</artifactId>
                    <version>3.0.0</version>
                    <configuration>
                        <archive>
                            <manifest>
                                <mainClass>io.regularization.leafclassifier.ResNetTrainer</mainClass>
                            </manifest>
                        </archive>
                        <descriptorRefs>
                            <descriptorRef>jar-with-dependencies</descriptorRef>
                        </descriptorRefs>
                    </configuration>
                    <executions>
                        <execution>
                            <id>make-assembly</id> <!-- this is used for inheritance merges -->
                            <phase>package</phase> <!-- bind to the packaging phase -->
                            <goals>
                                <goal>single</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
    <profile>
        <id>predict</id>
        <build>
            <finalName>${project.artifactId}-predict</finalName>
            <plugins>
                <plugin>
                    <artifactId>maven-assembly-plugin</artifactId>
                    <version>3.0.0</version>
                    <configuration>
                        <archive>
                            <manifest>
                                <mainClass>io.regularization.leafclassifier.ResNetPredictor</mainClass>
                            </manifest>
                        </archive>
                        <descriptorRefs>
                            <descriptorRef>jar-with-dependencies</descriptorRef>
                        </descriptorRefs>
                    </configuration>
                    <executions>
                        <execution>
                            <id>make-assembly</id> <!-- this is used for inheritance merges -->
                            <phase>package</phase> <!-- bind to the packaging phase -->
                            <goals>
                                <goal>single</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

[saudet]

Looks fine. So, could you show the content of /proc/<pid of the java process>/maps?

[ptah23]

it exits before I can check the contents of /proc//maps.
I have also tried Deep Learning AMI for Ubuntu. Please tell me which AMI you are using?

[ptah23]

I managed to check what is in /proc and there is noting for the process id. Which AMI are you using.

[ptah23]

from what i can see it looks like the jpeg library yo are using is different from what is available on amazon linux. can you show me a way to rectify this? maybe i should just use python instead as i an't get this to work at all

[ptah23]

@saudet
leaf-classifier.zip

[saudet]

File parentDir = new File(args[0] + "/images/train/");

Where is that directory? ↑

[ptah23]

@saudet It is quite big, where do you want me to upload to?

[ptah23]

@saudet I have put it on google drive: https://drive.google.com/open?id=0B3La2rDhbx8ySjNxQUlXNHdRVzA

[saudet]

Works perfectly fine here. Loads all images, and starts training.

[saudet]

Here is the exact command I used:

mvn exec:java -Dexec.mainClass=io.regularization.leafclassifier.ResNetTrainer -Dexec.args=/path/to/images/

[ptah23]

can you put link to AMI you are using please?

[saudet]

Just one from here: https://aws.amazon.com/amazon-linux-ami/

[ptah23]

That one involves installing CUDa by hand.
Do you know why it doesn't work on the Deep learning AMI that has CUDA installed already?

[saudet]

Could you give me a URL to that one?

[saudet]

So, it turns out that ami-41570b32 is available from https://aws.amazon.com/marketplace/pp/B01M0AXXQB if we choose the EU (Ireland) region. So I tried that out. Precisely I

1. Clicked the "Continue" button
2. Selected "EU (Ireland)" region
3. Chose "p2.xlarge" instance type
4. Clicked "Launch with 1-click"
5. Logged on to the instance, downloaded and extracted
   http://ftp.jaist.ac.jp/pub/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
6. Ran the following command in your project files:

/path/to/apache-maven-3.3.9/bin/mvn compile exec:java -Dexec.mainClass=io.regularization.leafclassifier.ResNetTrainer -Dexec.args=/path/to/images/

And everything went perfectly fine. So there is no problem at all with this instance as well.

This AWS image is interesting though. Maybe we should document the above steps somewhere @tomthetrainer ?

[agibsonccc]

Going to assume user error here. Closing this.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.