Use one model.pth file for multiple datasets. #1177
Comments
|
Datasets are not kept in memory during training. It would be helpful if you can share your |
|
This is my get_config function. 1 to 10 are the input images of size around 50 MB and 5000*5000 resolution. and the error is "unable to allocate 51380224 bytes in RAM, buy New RAM". def get_config(runner, raw_uri, processed_uri, root_uri, test=False):
train_ids = [
'1', '2', '3', '4', '5', '6', '7', '8', '9',
'10'
]
val_ids = ['11', '12', '13']
if test:
train_ids = train_ids[0:1]
val_ids = val_ids[0:1]
def make_scene(id):
raster_uri = join(raw_uri,
{}.tif'.format(id))
label_uri = join(processed_uri,{}.geojson'.format(id))
if test:
crop_uri = join(processed_uri, 'crops',
os.path.basename(raster_uri))
save_image_crop(
raster_uri,
crop_uri,
label_uri=label_uri,
size=1000,
min_features=5)
raster_uri = crop_uri
raster_source = RasterioSourceConfig(
uris=[raster_uri], channel_order=[0, 1, 2])
label_source = ObjectDetectionLabelSourceConfig(
vector_source=GeoJSONVectorSourceConfig(
uri=label_uri, default_class_id=0, ignore_crs_field=True))
return SceneConfig(
id=id, raster_source=raster_source, label_source=label_source)
class_config = ClassConfig(names=['vehicle'])
chip_sz = 1500
dataset = DatasetConfig(
class_config=class_config,
train_scenes=[make_scene(id) for id in train_ids],
validation_scenes=[make_scene(id) for id in val_ids])
chip_options = ObjectDetectionChipOptions(neg_ratio=5.0, ioa_thresh=0.9)
predict_options = ObjectDetectionPredictOptions(
merge_thresh=0.5, score_thresh=0.9)
backend = PyTorchObjectDetectionConfig(
model=ObjectDetectionModelConfig(backbone=Backbone.resnet18),
solver=SolverConfig(
lr=1e-4,
num_epochs=10,
test_num_epochs=2,
batch_sz=16,
one_cycle=True),
log_tensorboard=True,
run_tensorboard=False,
test_mode=test)
return ObjectDetectionConfig(
root_uri=root_uri,
dataset=dataset,
backend=backend,
train_chip_sz=chip_sz,
predict_chip_sz=chip_sz,
chip_options=chip_options,
predict_options=predict_options) |
|
Your chip size is likely too large. Try using something smaller like 256. |
|
I have pretrained model file like "model.pth" that contains information of car type object. but i want to train that "model.pth" with another object like tree or building so that model file contains info of car , tree and building, is that possible? |
|
You can load pretrained weights using the backend = PyTorchObjectDetectionConfig(
model=ObjectDetectionModelConfig(backbone=Backbone.resnet18, init_weights='path/to/model.pth')
...
If I understand correctly, you have a model that currently outputs predictions for a single class, but you would like to use its learned weights in a model that outputs 3 classes. You can achieve this by modifying the last layer of the old model so that it predicts 3 classes instead of one. It should be possible to do this while retaining the weights for the car class. You would then save this modified model into a new
No. But you can train a single model to predict multiple classes. Or you can train separate models for each class and combine them outside of RV. |
|
you can train separate models for each class and combine them outside of RV. how is it possible ? |
Which part specifically are you asking about? |
if i have one model file contains one class "car" and i have another model file contains another class "tree" so how can i combine them outside of RV ? |
for model.pth chip size is 300 but while i used that model for training as you mention that i gave path of model.pth in "init_weights" arg the following error is shown. Size mismatch for models.roi_heads.box_predictor.cls_score_weight copying a param with shape torch.size([3,1024]) from checkpoint the shape in torch.size([4,1024]) errors in loading state_dict for MyFasterRCNN in model.pth i used default object detection code and i select 2_11_potsdam.tif for training and chip size is 300 and for re-training i used default object detection code and select 2_12_postdam.tif for training and chip size is 300. |
|
If I understand correctly, you have a model that currently outputs predictions for a single class, but you would like to use its learned weights in a model that outputs 3 classes. You can achieve this by modifying the last layer of the old model so that it predicts 3 classes instead of one. It should be possible to do this while retaining the weights for the car class. You would then save this modified model into a new how can i change last layer of old model that predict single class ? |
You run both models on the image. One gives you bounding boxes for cars and the other gives you the bounding boxes for trees. So now you have detections for both the classes.
This error means that you are trying to load weights from a model that predicts 3 classes into a model that predicts 4 classes. To be able to load weights, the two model architectures must be identical.
The change I was suggesting would need to happen outside of RV. This would get you a |
|
The change I was suggesting would need to happen outside of RV. This would get you a can you explain that changes of outside of RV ? |
i have many datasets for training. but while training "No RAM space" error is displayed. so i want to divide whole dataset into smaller datasets.
so is it possible to use same "model.pth" for multiple datasets and how ?
The text was updated successfully, but these errors were encountered: