In this study of X-Ray image classification, we compare, analyze and contrast 3 machine learning techniques to classify Chest X-Ray images according to disorder categories. We used the ChestX-ray141 X-Ray image dataset, along with the corresponding research from the NIH to understand the dataset and its labels. We compared and analyzed Random Forests, Shallow CNN and Deep CNN classifiers to understand the advantages and disadvantages of each architecture on this problem. Through this study we aim to understand the effectiveness of each of the models, reporting the accuracy, f1- scores and choosing which model is best suited for X-Ray image classification, which has many interesting considerations such as explainability, the differences in importance of recall vs precision, and the complexity in image interpretation. The experimental code is hosted on GitHub.