For real-world applications, robots will need to continually learn in their environments through limited interactions with their users. Toward this, previous works in few-shot class incremental learning (FSCIL) and active class selection (ACS) have achieved promising results but were tested in constrained setups. Therefore, in this paper, we combine ideas from FSCIL and ACS to develop a novel framework that can allow an autonomous agent to continually learn new objects by asking its users to label only a few of the most informative objects in the environment. To this end, we build on a state-of-the-art (SOTA) FSCIL model and extend it with techniques from ACS literature. We term this model Few-shot Incremental Active class SeleCtiOn (FIASco). We further integrate a potential field-based navigation technique with our model to develop a complete framework that can allow an agent to process and reason on its sensory data through the FIASco model, navigate towards the most informative object in the environment, gather data about the object through its sensors and incrementally update the FIASco model. Experimental results on a simulated agent and a real robot show the significance of our approach for long-term real-world robotics applications.
We ran two experiments to test active class selection on agent/robot learning:
We used three standard datasets in our tests. Download each one and put in the desired ./*/utils/data/ folder.
- We build from CBCL-PR, an extension of CBCL, a classifier for FSCIL.
- We use YOLO v3 for object detection.
- We use the Project Malmo platform for Minecraft testing.
- We use pre-trained weights for the YOLO model. Download and put in the
./pepper/models/yolo/folder.
@inproceedings{mcclurg2023acsfscil,
title={Active Class Selection for Few-Shot Class-Incremental Learning},
author={McClurg, Christopher and Ayub, Ali and Tyagi, Harsh and Rajtmajer, Sarah M. and Wagner, Alan R.},
booktitle={Conference on Lifelong Learning Agents},
year={2023}}