Welcome to Tackling Tech Debt With Transformers! 🚀 Here, we're all about using fine-tuned transformer models to help open source repositories keep their technical debt in check.
- Feature 1: Diff Extraction
- Feature 2: Tech Debt Classifier
- Feature 3: Tech Debt Label Generator
"Implied cost of future reworking required when choosing an easy but limited solution instead of a better approach that could take more time." Wikipedia. Common application: Think of taking a shortcut when you are pushed for time. Later, you have to come back and rework code to "repay" the debt.
The Technical Debt Dataset!
- 78K commits from 33 Java projects
- Annotations from SZZ Algorithm for technical debt
- Labels from SonarQube for code smells
Read more from their paper: The Technical Debt Dataset and their repo: Technical Debt Dataset
Lenarduzzi, Valentina, Nyyti Saarimäki, and Davide Taibi. "The technical debt dataset." Proceedings of the fifteenth international conference on predictive models and data analytics in software engineering. 2019.
We use this dataset to create our own two datasets:
We will be fine-tuning a small CodeBERTa model for classification and a small codeT5 model for label generation.
This results in two models:
Visit this Huggingface space created by me! You can see how the models work together!
Here's a good example that works well:
@@ -0,0 +1,25 @@
+package org.apache.accumulo.server.test.randomwalk.shard;
+
+import java.util.Properties;
+import java.util.Random;
+import java.util.SortedSet;
+
+import org.apache.accumulo.server.test.randomwalk.State;
+import org.apache.accumulo.server.test.randomwalk.Test;
+import org.apache.hadoop.io.Text;
+
+
+public class Split extends Test {
+
+ @Override
+ public void visit(State state, Properties props) throws Exception {
+ String indexTableName = (String)state.get("indexTableName");
+ int numPartitions = (Integer)state.get("numPartitions");
+ Random rand = (Random) state.get("rand");
+
+ SortedSet<Text> splitSet = ShardFixture.genSplits(numPartitions, rand.nextInt(numPartitions)+1, "%06x");
+ log.debug("adding splits " + indexTableName);
+ state.getConnector().tableOperations().addSplits(indexTableName, splitSet);
+ }
+
+}- Impact: Now you can analyze the commits in any repo!
- You can see which commits might have tech debt!
- 95.4% accuracy on test set
- 0.949 F1 score on test set
- You can see what kind of tech debt it might be!
- You can see which commits might have tech debt!
- Next steps:
- We can improve the model by using better/more data.
- Label generation needs to be evaluated more.