Help a newbie train a model

[brian-o-mars]

I'm trying to train a tree model and I'm struggling to figure it out. The docs/video is quite outdated, most of provided code don't work anymore. I'd share all my code below and describe my issue please.

I start with first using the base model to run predictions:

pip install git+https://github.com/weecology/DeepForest.git
from deepforest import main
from deepforest import get_data
from deepforest import utilities
from deepforest.visualize import plot_results
from deepforest.preprocess import split_raster
import glob
import os

#Initialize the model class
model = main.deepforest()

#Load a pretrained tree detection model from Hugging Face
model.load_model(model_name="weecology/deepforest-tree", revision="main")

raster_path="/content/RGB_MOT23a_plot2.tif"
predicted_raster = model.predict_image(path=raster_path)
plot_results(predicted_raster, thickness = 30, results_color=[0, 0, 255])

I then import an annotation file

anno_df = utilities.shapefile_to_annotations(
    shapefile="/content/crowns/cr_metrics_plot12_forDL.shp",
    rgb="/content/RGB_MOT23a_plot2.tif",
    #convert_to_boxes=True,
    buffer_size=0.15
)

The annotation file looks something like this:

treeID	label	geometry	image_path
5.0	Tree	POLYGON ((1630.506 288.352, 1621.678 291, 1618...	RGB_MOT23a_plot2.tif
6.0	Tree	POLYGON ((2325.284 319, 2318.038 319, 2312.89 ...	RGB_MOT23a_plot2.tif

Now for the training, I'm doing this:

#Directories to save the training data
train_dir = "/content/deepforest_train"
os.makedirs(train_dir, exist_ok=True)
val_dir = "/content/deepforest_val"
os.makedirs(val_dir, exist_ok=True)

#Paths for your images and annotations
train_raster_path = "/content/RGB_MOT23a_plot2.tif" # The RGB for the anno_df you created

#Use split_raster to create training crops and a new CSV with their annotations
split_raster(
    path_to_raster=train_raster_path,
    annotations_file=anno_df,
    save_dir=train_dir,
    # Use a patch_size that is suitable for the model, 400x400 is common.
    patch_size=400,
    patch_overlap=0.10
)

#Newly created CSV files in the directories
train_csv = glob.glob(os.path.join(train_dir, "*.csv"))[0]

#Load training data
train_dataloader = model.load_dataset(
    csv_file=train_csv,
    #root_dir=train_dir,
    root_dir="/content/RGB_MOT23a_plot2.tif",
    #augment=True, 
    batch_size=5,  
    shuffle=True,
    #train=True  
)

#Model configurations before training
model.config["train"]["epochs"] = 10
model.config["train"]['csv_file'] = train_csv

#Training
model.trainer.fit(
    model,
    train_dataloader,
    #val_dataloader=val_dataloader
)

Here's the result for my initial prediction using the base model:

Here's the result using the trained model on a new stand:

For transparency, most of the code is initially AI generated with many tweaks from me to make it work. I'm suspecting the issue may be from the split_raster function. Should I even be splitting? I'd appreciate any help with this please. Also, the annotation file is just a shapefile of tree crowns over an image.

[henrykironde]

Make the annotation file into a csv and try again

[brian-o-mars]

Hi Henry, I was able to make it work well. About annotation, do we have to make the annotations square boxes, or can the annotations be irregular polygons that follow the shape of the crowns more closely? Would the irregular polygons make the fine-tuning better? I'd appreciate your advice

[ethanwhite]

Glad to hear that you go it working @brian-o-mars! At the moment boxes are the best way to go because our model is box-based. That said, we are working on adding support for polygon models, so you could do polygon annotations (so that you have them when the new polygon models role out), but for now convert them to boxes based on min/max of x/y for fine tuning the current models.