Fraud Detection on Bank Payments.ipynb

{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"# Fraud Detection on Bank Payments\n\n## Fraud and detecting it\n\nFraudulent behavior can be seen across many different fields such as e-commerce, healthcare, payment and banking systems. Fraud is a billion-dollar business and it is increasing every year. The PwC global economic crime survey of 2018 [1] found that half (49 percent) of the 7,200 companies they surveyed had experienced fraud of some kind.\n\nEven if fraud seems to be scary for businesses it can be detected using intelligent systems such as rules engines or machine learning. Most people here in Kaggle are familier with machine learning but for rule engines here is a quick information. \n    A rules engine is a software system that executes one or more business rules in a runtime production environment. These rules are generally written by domain experts for transferring the knowledge of the problem to the rules engine and from there to production. Two rules examples for fraud detection would be limiting the number of transactions in a time period (velocity rules),  denying the transactions which come from previously known fraudulent IP's and/or domains.\n    \nRules are great for detecting some type of frauds but they can fire a lot of false positives or false negatives in some cases because they have predefined threshold values. For example let's think of a rule for denying a transaction which has an amount that is bigger than 10000 dollars for a specific user. If this user is an experienced fraudster, he/she may be aware of the fact that the system would have a threshold and he/she can just make a transaction just below the threshold value (9999 dollars).\n\nFor these type of problems ML comes for help and reduce the risk of frauds and the risk of business to lose money. With the combination of rules and machine learning, detection of the fraud would be more precise and confident.\n\n## Banksim dataset\n\nWe detect the fraudulent transactions from the Banksim dataset. This synthetically generated dataset consists of payments from various customers made in different time periods and with different amounts. For\nmore information on the dataset you can check the [Kaggle page](https://www.kaggle.com/ntnu-testimon/banksim1) for this dataset which also has the link to the original paper. \n\nHere what we'll do in this kernel:\n1. [Exploratory Data Analysis (EDA)](#Explaratory-Data-Analysis)\n2. [Data Preprocessing](#Data-Preprocessing)\n3. [Oversampling with SMOTE](#Oversampling-with-SMOTE)\n4. [K-Neighbours Classifier](#K-Neighbours-Classifier)\n5. [Random Forest Classifier](#Random-Forest-Classifier)\n6. [XGBoost Classifier](#XGBoost-Classifier)\n7. [Conclusion](#Conclusion)"},{"metadata":{},"cell_type":"markdown","source":"## Explaratory Data Analysis\n\nIn this chapter we will perform an EDA on the data and try to gain some insight from it."},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Necessary imports\n\n## Data loading, processing and for more\nimport pandas as pd\nimport numpy as np\nfrom imblearn.over_sampling import SMOTE\n\n## Visualization\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n# set seaborn style because it prettier\nsns.set()\n\n## Metrics\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import confusion_matrix, classification_report\nfrom sklearn.metrics import roc_curve, auc\n\n## Models\nimport xgboost as xgb\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.ensemble import VotingClassifier","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e1b7b9feedaed877bfee5bd871c627cdbcf2e31b"},"cell_type":"markdown","source":"**Data**\nAs we can see in the first rows below the dataset has 9 feature columns and a target column. \nThe feature columms are :\n* **Step**: This feature represents the day from the start of simulation. It has 180 steps so simulation ran for virtually 6 months.\n* **Customer**: This feature represents the customer id\n* **zipCodeOrigin**: The zip code of origin/source.\n* **Merchant**: The merchant's id\n* **zipMerchant**: The merchant's zip code\n* **Age**: Categorized age \n    * 0: <= 18, \n    * 1: 19-25, \n    * 2: 26-35, \n    * 3: 36-45,\n    * 4: 46:55,\n    * 5: 56:65,\n    * 6: > 65\n    * U: Unknown\n* **Gender**: Gender for customer\n     * E : Enterprise,\n     * F: Female,\n     * M: Male,\n     * U: Unknown\n* **Category**: Category of the purchase. I won't write all categories here, we'll see them later in the analysis.\n* **Amount**: Amount of the purchase\n* **Fraud**: Target variable which shows if the transaction fraudulent(1) or benign(0)"},{"metadata":{"trusted":true,"_uuid":"2bf619515c1a924cf041c419cb4a0514ba6ce820","_kg_hide-input":true},"cell_type":"code","source":"# read the data and show first 5 rows\ndata = pd.read_csv(\"../input/bs140513_032310.csv\")\ndata.head(5)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"29a786fcbe642321727b007bd015cec05a198238"},"cell_type":"markdown","source":"Let's look at column types and missing values in data.  Oh im sorry there is **no** missing values which means we don't have to perform an imputation."},{"metadata":{"trusted":true,"_uuid":"c285fad4bda30a3b5feec97b71e0a9d1e0b7ec8c"},"cell_type":"code","source":"data.info()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"36778347862db884536c7b89e817c959acddc514"},"cell_type":"markdown","source":"**Fraud data** will be imbalanced like you see in the plot below and from the count of instances. To balance the dataset one can perform oversample or undersample techniques. Oversampling is increasing the number of the minority class by generating instances from the minority class . Undersampling is reducing the number of instances in the majority class by selecting random points from it to where it is equal with the minority class. Both operations have some risks: Oversample will create copies or similar data points which sometimes would not be helpful for the case of fraud detection because fraudulent transactions may vary. Undersampling means that we lost data points thus information. We will perform an oversampled technique called SMOTE (Synthetic Minority Over-sampling Technique). SMOTE will create new data points from minority class using the neighbour instances so generated samples are not exact copies but they are similar to instances we have."},{"metadata":{"trusted":true,"_uuid":"ee9ed07328bb6ae710263c4e55066f3e757206a0","_kg_hide-input":true},"cell_type":"code","source":"# Create two dataframes with fraud and non-fraud data \ndf_fraud = data.loc[data.fraud == 1] \ndf_non_fraud = data.loc[data.fraud == 0]\n\nsns.countplot(x=\"fraud\",data=data)\nplt.title(\"Count of Fraudulent Payments\")\nplt.show()\nprint(\"Number of normal examples: \",df_non_fraud.fraud.count())\nprint(\"Number of fradulent examples: \",df_fraud.fraud.count())\n#print(data.fraud.value_counts()) # does the same thing above","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cf77a853d6b363b91512d3c3ce22717a5ed29645"},"cell_type":"markdown","source":"We can see the mean amount and fraud percent by category below. Looks like leisure and the travel is the most selected categories for fraudsters. Fraudsters chose the categories which people spend more on average. Let's confirm this hypothesis by checking the fraud and non-fraud amount transacted."},{"metadata":{"trusted":true,"_uuid":"44479e279e91d208d9108112327441e5d57d5fee"},"cell_type":"code","source":"print(\"Mean feature values per category\",data.groupby('category')['amount','fraud'].mean())","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"258b20b07d771a0fd9a7a22fda617e40ce967842"},"cell_type":"markdown","source":"Our hypothesis for fraudsters choosing the categories which people spend more is only partly correct, but as we can see in the table below we can say confidently say that a fraudulent transaction will be much more (about four times or more) than average for that category."},{"metadata":{"trusted":true,"_uuid":"b5a14927d2edfb847e809969fd534e9bae339a2a"},"cell_type":"code","source":"# Create two dataframes with fraud and non-fraud data \npd.concat([df_fraud.groupby('category')['amount'].mean(),df_non_fraud.groupby('category')['amount'].mean(),\\\n           data.groupby('category')['fraud'].mean()*100],keys=[\"Fraudulent\",\"Non-Fraudulent\",\"Percent(%)\"],axis=1,\\\n          sort=False).sort_values(by=['Non-Fraudulent'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e41aeb2f2c6b898ebbcb5b4222fd44ca8e87c8ae"},"cell_type":"markdown","source":"Average amount spend it categories are similar; between 0-500 discarding the outliers, except for the travel category which goes very high. "},{"metadata":{"trusted":true,"_uuid":"290e7163a62fb029fcbdf45a3d35c7c0d58c423d","_kg_hide-input":false},"cell_type":"code","source":"# Plot histograms of the amounts in fraud and non-fraud data \nplt.figure(figsize=(30,10))\nsns.boxplot(x=data.category,y=data.amount)\nplt.title(\"Boxplot for the Amount spend in category\")\nplt.ylim(0,4000)\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"69dbe47eb7f490030006aaa42bac0a7414b3c100"},"cell_type":"markdown","source":"Again we can see in the histogram below the fradulent transactions are less in count but more in amount."},{"metadata":{"trusted":true,"_uuid":"93272d1343f6783a0152a80b5a177541faed8e25"},"cell_type":"code","source":"# Plot histograms of the amounts in fraud and non-fraud data \nplt.hist(df_fraud.amount, alpha=0.5, label='fraud',bins=100)\nplt.hist(df_non_fraud.amount, alpha=0.5, label='nonfraud',bins=100)\nplt.title(\"Histogram for fraudulent and nonfraudulent payments\")\nplt.ylim(0,10000)\nplt.xlim(0,1000)\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"76c33765a55bb89e0fb8a5d1cd55aa6bd7727dfa"},"cell_type":"markdown","source":"Looks like fraud occurs more in ages equal and below 18(0th category). Can it be because of fraudsters thinking it would be less consequences if they show their age younger, or maybe they really are young."},{"metadata":{"trusted":true,"_uuid":"d5e387216fa347c7f437c7f187095edd3912f848"},"cell_type":"code","source":"print((data.groupby('age')['fraud'].mean()*100).reset_index().rename(columns={'age':'Age','fraud' : 'Fraud Percent'}).sort_values(by='Fraud Percent'))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"23001479b16bd3f01682dedd94b47a3f231a7c9e"},"cell_type":"markdown","source":"## Data Preprocessing\n\nIn this part we will preprocess the data and prepare for the training.\n\nThere are only one unique zipCode values so we will drop them."},{"metadata":{"trusted":true,"_uuid":"47246c26a027470f30ea0a5e4a149e6dbb951403"},"cell_type":"code","source":"print(\"Unique zipCodeOri values: \",data.zipcodeOri.nunique())\nprint(\"Unique zipMerchant values: \",data.zipMerchant.nunique())\n# dropping zipcodeori and zipMerchant since they have only one unique value\ndata_reduced = data.drop(['zipcodeOri','zipMerchant'],axis=1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7f589a60028130489740b7ceaa12ca7c4879c2e7"},"cell_type":"markdown","source":"Checking the data after dropping."},{"metadata":{"trusted":true,"_uuid":"4b5689fb2da6f4ab7d4ba8e707530ea10753e8bb"},"cell_type":"code","source":"data_reduced.columns","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5da150a203f2ab7e310af70365a4e90f7f522168"},"cell_type":"markdown","source":"Here we will transform categorical features into numerical values. It is usually better to turn these type of categorical values into dummies because they have no relation in size(i.e. customer1 is not greater than customer2) but since they are too many (over 500k customers and merchants) the features will grow 10^5 in size and it will take forever to train. I've put the code below for turning categorical features into dummies if you want to give it a try.\n> data_reduced.loc[:,['customer','merchant','category']].astype('category')\n> data_dum = pd.get_dummies(data_reduced.loc[:,['customer','merchant','category','gender']],drop_first=True) # dummies\n> print(data_dum.info())"},{"metadata":{"trusted":true,"_uuid":"4d7d4e448b2fa8c9313810f7c24b6e010af719e4"},"cell_type":"code","source":"# turning object columns type to categorical for easing the transformation process\ncol_categorical = data_reduced.select_dtypes(include= ['object']).columns\nfor col in col_categorical:\n    data_reduced[col] = data_reduced[col].astype('category')\n# categorical values ==> numeric values\ndata_reduced[col_categorical] = data_reduced[col_categorical].apply(lambda x: x.cat.codes)\ndata_reduced.head(5)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"37f7ed8c735c17e699e36cc34699df01a2fddda8"},"cell_type":"markdown","source":"Let's define our independent variable (X) and dependant/target variable y"},{"metadata":{"trusted":true,"_uuid":"bca1ece9de83b5f0db01cc4e3fbb9799bdcc3e22"},"cell_type":"code","source":"X = data_reduced.drop(['fraud'],axis=1)\ny = data['fraud']\nprint(X.head(),\"\\n\")\nprint(y.head())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"474518cb2bb397a51477e826541fc33c62d2d492"},"cell_type":"code","source":"y[y==1].count()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"37bca88900f65dd21a308beca5cc615fe4a40c31"},"cell_type":"markdown","source":"## Oversampling with SMOTE\n\nUsing SMOTE(Synthetic Minority Oversampling Technique) [2] for balancing the dataset. Resulted counts show that now we have exact number of class instances (1 and 0)."},{"metadata":{"_uuid":"784e528eb3ea2ceb20af530a516911279d5df8f4","trusted":true},"cell_type":"code","source":"sm = SMOTE(random_state=42)\nX_res, y_res = sm.fit_resample(X, y)\ny_res = pd.DataFrame(y_res)\nprint(y_res[0].value_counts())","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4cafd096d0bd1bbe528eb8d5f7d943d58fd277db"},"cell_type":"markdown","source":"I will do a train test split for measuring the performance. I haven't done cross validation since we have a lot of instances and i don't want to wait that much for training but it should be better to cross validate most of the times. "},{"metadata":{"trusted":true,"_uuid":"1c4f59c9cec206b7fa56e61a1202050f8966a94b"},"cell_type":"code","source":"# I won't do cross validation since we have a lot of instances\nX_train, X_test, y_train, y_test = train_test_split(X_res,y_res,test_size=0.3,random_state=42,shuffle=True,stratify=y_res)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"47b6d299e0a3c08509066a19b2b225bf37acf0ad"},"cell_type":"markdown","source":"I will define a function for plotting the ROC_AUC curve. It is a good visual way to see the classification performance."},{"metadata":{"trusted":true,"_uuid":"c15c0a19c3fe5402109c9e54e58d0402c40107dd"},"cell_type":"code","source":"# %% Function for plotting ROC_AUC curve\n\ndef plot_roc_auc(y_test, preds):\n    '''\n    Takes actual and predicted(probabilities) as input and plots the Receiver\n    Operating Characteristic (ROC) curve\n    '''\n    fpr, tpr, threshold = roc_curve(y_test, preds)\n    roc_auc = auc(fpr, tpr)\n    plt.title('Receiver Operating Characteristic')\n    plt.plot(fpr, tpr, 'b', label = 'AUC = %0.2f' % roc_auc)\n    plt.legend(loc = 'lower right')\n    plt.plot([0, 1], [0, 1],'r--')\n    plt.xlim([0, 1])\n    plt.ylim([0, 1])\n    plt.ylabel('True Positive Rate')\n    plt.xlabel('False Positive Rate')\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"589632daf9c07f7903870ecbf183c2057e10d4be"},"cell_type":"markdown","source":"As i talked about it before fraud datasets will be imbalanced and most of the instances will be non-fraudulent. Imagine that we have the dataset here and we are always predicting non-fraudulent. Our accuracy would be almost 99 % for this dataset and mostly for others as well since fraud percentage is very low. Our accuracy is very high but we are not detecting any frauds so it is a useless classifier. So the base accuracy score should be better at least than predicting always non-fraudulent for performing a detection."},{"metadata":{"trusted":true,"_uuid":"0bacd9ca1db82f754e402ce66b654df460d51f88"},"cell_type":"code","source":"# The base score should be better than predicting always non-fraduelent\nprint(\"Base accuracy score we must beat is: \", \n      df_non_fraud.fraud.count()/ np.add(df_non_fraud.fraud.count(),df_fraud.fraud.count()) * 100)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"03bf9d11a03798ff2411e93d3956d378719172d7"},"cell_type":"markdown","source":"## **K-Neighbours Classifier**"},{"metadata":{"trusted":true,"_uuid":"6d3a294403b1afbdbf18219ab0418a2e03e201e2"},"cell_type":"code","source":"# %% K-ello Neigbors\n\nknn = KNeighborsClassifier(n_neighbors=5,p=1)\n\nknn.fit(X_train,y_train)\ny_pred = knn.predict(X_test)\n\n\nprint(\"Classification Report for K-Nearest Neighbours: \\n\", classification_report(y_test, y_pred))\nprint(\"Confusion Matrix of K-Nearest Neigbours: \\n\", confusion_matrix(y_test,y_pred))\nplot_roc_auc(y_test, knn.predict_proba(X_test)[:,1])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7fed4c48fb1a0b6d73df873c8492847b03b069c4"},"cell_type":"markdown","source":"## **Random Forest Classifier**"},{"metadata":{"trusted":true,"_uuid":"b320aa8bea1fbf19ef198905ad7f7a365561ccdd"},"cell_type":"code","source":"# %% Random Forest Classifier\n\nrf_clf = RandomForestClassifier(n_estimators=100,max_depth=8,random_state=42,\n                                verbose=1,class_weight=\"balanced\")\n\nrf_clf.fit(X_train,y_train)\ny_pred = rf_clf.predict(X_test)\n\nprint(\"Classification Report for Random Forest Classifier: \\n\", classification_report(y_test, y_pred))\nprint(\"Confusion Matrix of Random Forest Classifier: \\n\", confusion_matrix(y_test,y_pred))\nplot_roc_auc(y_test, rf_clf.predict_proba(X_test)[:,1])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1d5898c4320b78015fd40d059e158f767399d354"},"cell_type":"markdown","source":"## XGBoost Classifier"},{"metadata":{"trusted":true,"_uuid":"1cbe2c476bcef6f4ed9523c891d0536077a6861a"},"cell_type":"code","source":"XGBoost_CLF = xgb.XGBClassifier(max_depth=6, learning_rate=0.05, n_estimators=400, \n                                objective=\"binary:hinge\", booster='gbtree', \n                                n_jobs=-1, nthread=None, gamma=0, min_child_weight=1, max_delta_step=0, \n                                subsample=1, colsample_bytree=1, colsample_bylevel=1, reg_alpha=0, reg_lambda=1, \n                                scale_pos_weight=1, base_score=0.5, random_state=42, verbosity=True)\n\nXGBoost_CLF.fit(X_train,y_train)\n\ny_pred = XGBoost_CLF.predict(X_test)\n\nprint(\"Classification Report for XGBoost: \\n\", classification_report(y_test, y_pred))\nprint(\"Confusion Matrix of XGBoost: \\n\", confusion_matrix(y_test,y_pred))\nplot_roc_auc(y_test, XGBoost_CLF.predict_proba(X_test)[:,1])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ef0e8b46a02a3ba1e3b7df37a4b9279428569adb"},"cell_type":"markdown","source":"## Conclusion\n\nIn this kernel we have tried to do fraud detection on a bank payment data and we have achieved remarkable results with our classifiers. Since fraud datasets have an imbalance class problem we performed an oversampling technique called SMOTE and generated new minority class examples. I haven't put the classification results without SMOTE here but i added them in my github repo before so if you are interested to compare both results you can also check [my github repo](https://github.com/atavci/fraud-detection-on-banksim-data). \n\nThanks for taking the time to read or just view the results from my first kernel i hope you enjoyed it. I would be grateful for any kind of critique, suggestion or comment and i wish you to have a great day with lots of beautiful data!"},{"metadata":{"_uuid":"3d840bf8b52b3514a2ae86f17f2bed7235f4a927"},"cell_type":"markdown","source":"## Resources\n\n\n[[1]](#-1). Lavion, Didier; et al. [\"PwC's Global Economic Crime and Fraud Survey 2018\"](https://www.pwc.com/gx/en/forensics/global-economic-crime-and-fraud-survey-2018.pdf) (PDF). PwC.com. Retrieved 28 August 2018. \n\n[[2]](#2). [SMOTE: Synthetic Minority Over-sampling Technique](https://jair.org/index.php/jair/article/view/10302)"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}