LLM-Code-Classifier

Project for DS-301, LLM-generated Code Classifier with Transformer-based Embeddings.

Shi Luo sl7213 Mentor: Ali Quidwai DS-UA 301, Professor Parijat Dube

Running The Demo

Model Input: labeled_functionlv_embeddings.csv.zip

This Model could be running on a Jupyter Notebook / Colab with GPU acceleration: Classifier_on_function_level_embeddings.ipynb

Project Overview

Performance Evaluation Results

Accuracy, AUC on certain Epochs:

Epoch 1/20 59/59 [==============================] - 2s 14ms/step - loss: 0.6483 - accuracy: 0.6305 - auc: 0.5950 - val_loss: 0.6168 - val_accuracy: 0.6352 - val_auc: 0.9293

Epoch 10/20 59/59 [==============================] - 0s 8ms/step - loss: 0.2119 - accuracy: 0.9398 - auc: 0.9806 - val_loss: 0.2611 - val_accuracy: 0.9013 - val_auc: 0.9612

Epoch 20/20 59/59 [==============================] - 0s 6ms/step - loss: 0.1201 - accuracy: 0.9667 - auc: 0.9921 - val_loss: 0.2027 - val_accuracy: 0.9313 - val_auc: 0.9674