Line follower example (#1)

* leo_example_line_follower: initial commit for line follower package Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: package.xml: remove redundant dependencies Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: data_saver.py: replaced use od ros parameters with command line arguments and python argparse Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: data_saver.py: added info on node's exit and fixed path for file with labels Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: data_saver.py: Added info logs for creating files and directories Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: data_saver.py: starting time counting with the first message from cmd_vel topic, fixed saved img filename, closing label file on shutdown Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: data_saver.py: added saving images and labels Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: data_saver.py: added ignoring images with label (0.0, 0.0) Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: added line_follower.py - script for autonomous following the line with rover * leo_example_line_followe: added directory with first models, needed for line_follower.py script * leo_example_line_follower: models: added model trained on color masks, with two lines track Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: color_mask: added color_mask.py script with ros node for chosing hsv bounds for color mask Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: color_mask: added cfg directory with yaml files with example hsv_bounds and dynamic_reconfigure config file Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: color_mask: changed package.xml and CMakeLists.txt for use of dynamic_reconfigure Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: data_saver.py: fixed typos and formatted code with black Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: data_saver: removed unused arguments from launch file Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: color mask: added launch file for the color_mask_finder node Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: line_follower.py: added use of ros parameters for hsv color bounds Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: line follower: added launch file for the line follower node Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: moved yaml files to config directory Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: fixed typos, and added descriptions to ros args Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: removed redundant files Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: line_follower.py: switched param names to private namespace Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_exapmle_line_follower: added model trained on more accurate color masks Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: CMakeLists.txt: fixed catkin_lint errors Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: package.xml: updated package description Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: color_mask.py: implemented code review guidlines (about python buildins and instance method) Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: data_saver.py: implemented guidlines from code review Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: scripts: implemented guidlines from code review Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: config: added yaml files with color mask values for blue and red colors Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: color_mask.launch: changed default value of 'file' arg Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: package.xml: fixed typo Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: data_saver.py: implemented guidelines from code review Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: scripts: added script for processing saved data into ready dataset for keras model Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: prepare_data.py: fixed os.join bugs Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: data_saver.py: added base path if output directory is given as relative path Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: data_saver.py: added use of ptahlib home function instead of hardcoding home directory path Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: prepare_data.py: made the script executable with rosrun Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: prepare_data.py: gave the script execution rights with chmod + x Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: prepare_data.py: removing parent directories when train and validation dirs are nested paths Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: prepare_data.py: switched moving and removing files to just copying Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: prepare_data.py: added creating of zip file from processed data, and clearing the working directory from zipped files Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: prepare_data.py: added printing info about zipping files Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: line_follower.launch: fixed ros find package syntax error Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: prepare_data.py: changed argument names to one pattern Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: prepare_data.py: added checking if the zipfile name ends with '.zip' Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: prepare_data.py: formated the code using black Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: added notebook to repository Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: CMakeLists.txt: added prepare_data.py script to catkin_install_python section Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: implemented code review guidlines Signed-off-by: Bitterisland6 <2010olo@wp.pl> * leo_example_line_follower: line_follower.launch: updated arg name in rosparam Signed-off-by: Bitterisland6 <2010olo@wp.pl>
LeoRover · May 20, 2022 · 0e5edd9 · 0e5edd9
1 parent fc63253
commit 0e5edd9
Show file tree

Hide file tree

Showing 15 changed files with 733 additions and 0 deletions.
diff --git a/leo_example_line_follower/CMakeLists.txt b/leo_example_line_follower/CMakeLists.txt
@@ -0,0 +1,32 @@
+cmake_minimum_required(VERSION 3.0.2)
+project(leo_example_line_follower)
+
+find_package(catkin REQUIRED COMPONENTS
+  dynamic_reconfigure
+)
+
+
+generate_dynamic_reconfigure_options(
+  cfg/ColorMask.cfg
+)
+
+catkin_package(CATKIN_DEPENDS
+  dynamic_reconfigure
+)
+
+
+install(
+  DIRECTORY 
+    config
+    launch
+    models
+  DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}
+)
+
+catkin_install_python(
+  PROGRAMS scripts/data_saver.py
+           scripts/line_follower.py
+           scripts/color_mask.py
+           scripts/prepare_data.py
+  DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
+)
diff --git a/leo_example_line_follower/cfg/ColorMask.cfg b/leo_example_line_follower/cfg/ColorMask.cfg
@@ -0,0 +1,16 @@
+#!/usr/bin/env python3
+
+PACKAGE = "leo_example_line_follower"
+
+from dynamic_reconfigure.parameter_generator_catkin import *
+
+gen = ParameterGenerator()
+
+gen.add("hue_min", int_t, 0, "minimum hue value for color_mask", 0, 0, 179)
+gen.add("hue_max", int_t, 0, "maximum hue value for color_mask", 179, 0, 179)
+gen.add("sat_min", int_t, 0, "minimum saturation value for color_mask", 0, 0, 255)
+gen.add("sat_max", int_t, 0, "maximum saturation value for color_mask", 255, 0, 255)
+gen.add("val_min", int_t, 0, "minimum val (V from HSV) value for color_mask", 0, 0, 255)
+gen.add("val_max", int_t, 0, "maximum val (V from HSV) value for color_mask", 255, 0, 255)
+
+exit(gen.generate(PACKAGE, "color_mask_finder", "ColorMask"))
diff --git a/leo_example_line_follower/config/blue.yaml b/leo_example_line_follower/config/blue.yaml
@@ -0,0 +1,6 @@
+hue_min: 94
+hue_max: 136
+sat_min: 38
+sat_max: 141
+val_min: 65
+val_max: 172
diff --git a/leo_example_line_follower/config/red.yaml b/leo_example_line_follower/config/red.yaml
@@ -0,0 +1,6 @@
+hue_min: 175
+hue_max: 4
+sat_min: 27
+sat_max: 188
+val_min: 109
+val_max: 222
diff --git a/leo_example_line_follower/launch/color_mask.launch b/leo_example_line_follower/launch/color_mask.launch
@@ -0,0 +1,8 @@
+<launch>
+    <arg name="file" default="$(find leo_example_line_follower)/config/blue.yaml" doc="path to yaml file, with starting values for color mask"/>
+
+    <node name="color_mask_finder" pkg="leo_example_line_follower" type="color_mask.py" clear_params="true" output="screen">
+        <rosparam command="load" file="$(arg file)" />
+    </node>
+
+</launch>
diff --git a/leo_example_line_follower/launch/line_follower.launch b/leo_example_line_follower/launch/line_follower.launch
@@ -0,0 +1,14 @@
+<launch>
+    <arg name="color_mask_file" default="$(find leo_example_line_follower)/config/blue.yaml" doc="path to yaml file with color mask values"/>
+    <arg name="pub_mask" default="false" doc="flag specifying whether or not publish the color mask"/>
+
+    <arg name="camera" default="camera/image_raw" doc="name of topic with Image messages"/>
+    <arg name="vel" default="cmd_vel" doc="name of topic to which node will publish Twist messages"/>
+    <arg name="model" default="$(find leo_example_line_follower)/models/color_mask_model.tflite" doc="path to neural network model"/>
+
+    <node name="line_follower" pkg="leo_example_line_follower" type="line_follower.py" clear_params="true" output="screen" args="-p $(arg model) -c $(arg camera) -v $(arg vel)">
+        <rosparam command="load" file="$(arg color_mask_file)" />
+        <param name="pub_mask" value="$(arg pub_mask)" />
+    </node>
+
+</launch>
diff --git a/leo_example_line_follower/launch/record_data.launch b/leo_example_line_follower/launch/record_data.launch
@@ -0,0 +1,9 @@
+<launch>
+    <arg name="duration" default="20.0" doc="how long will the data be recorded"/>
+    <arg name="output_dir" default="data" doc="name of the directory for recorded data"/>
+    <arg name="video_topic" default="camera/image_raw" doc="name of the topic with Image messages"/>
+    <arg name="vel_topic" default="cmd_vel" doc="name of the topic with Twist messages"/>
+
+    <node name="data_saver" pkg="leo_example_line_follower" type="data_saver.py" clear_params="true" output="screen" args="$(arg duration) -c $(arg video_topic) -V $(arg vel_topic) -o $(arg output_dir)" />
+
+</launch>
diff --git a/leo_example_line_follower/models/color_mask_model.tflite b/leo_example_line_follower/models/color_mask_model.tflite
diff --git a/leo_example_line_follower/models/smaller_mask.tflite b/leo_example_line_follower/models/smaller_mask.tflite
diff --git a/leo_example_line_follower/notebooks/leorover-line-follower.ipynb b/leo_example_line_follower/notebooks/leorover-line-follower.ipynb
@@ -0,0 +1 @@
+{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import numpy as np\nimport keras as K\nimport keras.layers as KL\nimport pickle\nimport tqdm\nimport os\nimport cv2\nfrom matplotlib import pyplot as plt\n\n!pip3 install -U tensorflow-addons\nimport tensorflow_addons as tfa\nimport tensorflow as tf","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Variables that need to be changed if you are training on your data","metadata":{}},{"cell_type":"code","source":"# name of root directory of the dataset (name of the zip file with your dataset)\nroot_dir = \"full-track\"\n\n# lower and upper bound for catching color mask\nhsv_lower = (94, 38, 65)\nhsv_upper = (136, 141, 172)\n\n# height of the crop area (number of pixels, starting from upper edge of image that will be cropped)\ncrop_height = 220\n\n# path to test image for preprocessing (can be any image)\npreprocess_path = \"../input/full-track/data/21420221938-img1.jpg\"\n\n# save directory for trained model\nsave_dir = \"./model\"\n\n# Chose the name of saved tflite model\ntflite_model_name = \"tflite_model\"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Reading prepared data labels and partition","metadata":{}},{"cell_type":"code","source":"labels = None\npartition = None\n\n\nwith open('../input/' + root_dir + '/labels.pickle', 'rb') as handle:\n    labels = pickle.load(handle)\n    \nwith open('../input/' + root_dir + '/partition.pickle', 'rb') as handle:\n    partition = pickle.load(handle)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Neuarl Network model parts","metadata":{}},{"cell_type":"code","source":"GPUs_num = len(tf.config.list_physical_devices('GPU'))\nCPUs_num = len(tf.config.list_physical_devices('CPU'))\nprint(\"Num GPUs Available: \", GPUs_num)\nprint(\"gpu_devices: \", tf.config.list_physical_devices('GPU'))\nprint(\"Num CPUs Available: \", CPUs_num)\nprint(\"cpu_devices: \", tf.config.list_physical_devices('CPU'))\ndevice = None\ndevice = '/GPU:0' if GPUs_num > 0 else '/CPU:0'\ndevice","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Preprocess function","metadata":{}},{"cell_type":"code","source":"def preprocess(img, dim):\n    # converting to hsv\n    hsv_img = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)\n    # croping the img\n    crop_img = hsv_img[crop_height:hsv_img.shape[0], :]\n    # catching color mask\n    color_mask = cv2.inRange(crop_img, hsv_lower, hsv_upper)\n    # conveting values to float\n    float_img = color_mask.astype(np.float32)\n    # resizing\n    resized_img = cv2.resize(float_img, (dim[1], dim[0]))\n    # normalizing\n    final_img = resized_img / 255.0\n    \n    return final_img[:,:,np.newaxis]\n    ","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Preprocessing test","metadata":{}},{"cell_type":"code","source":"preprocess_test_img = cv2.imread(preprocess_path)\nprint(\"Image shape before preprocess: {}\".format(preprocess_test_img.shape))\nplt.imshow(cv2.cvtColor(preprocess_test_img, cv2.COLOR_BGR2RGB))\nplt.show()\nprocessed = preprocess(preprocess_test_img, (120,160))\nprint(\"Image shape after preprocess: {}\".format(processed.shape))\nplt.imshow(processed, cmap=\"gray\")\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Keras Data Generator","metadata":{}},{"cell_type":"code","source":"class DataGenerator(K.utils.all_utils.Sequence):\n    def __init__(self, list_IDs, labels, batch_size=32, dim=(32,32,32), n_channels=1, shuffle=True):\n        'Initialization'\n        self.dim = dim\n        self.batch_size = batch_size\n        self.labels = labels\n        self.list_IDs = list_IDs\n        self.n_channels = n_channels\n        self.shuffle = shuffle\n        self.on_epoch_end()\n        \n    def on_epoch_end(self):\n        'Updates indexes after each epoch'\n        self.indexes = np.arange(len(self.list_IDs))\n        if self.shuffle == True:\n            np.random.shuffle(self.indexes)\n        \n    def __len__(self):\n        'Denotes the number of batches per epoch'\n        return int(np.floor(len(self.list_IDs) / self.batch_size))\n    \n    def __data_generation(self, list_IDs_temp):\n        'Generates data containing batch_size samples'\n        X = np.empty((self.batch_size, *self.dim, self.n_channels))\n        y_linear = np.empty((self.batch_size, 1), dtype=float)\n        y_angular = np.empty((self.batch_size, 1), dtype=float)\n\n        for i, ID in enumerate(list_IDs_temp):\n            img = cv2.imread('../input/' + root_dir + '/data/' + ID)\n            preprocess_img = preprocess(img, self.dim)\n            X[i,:] = preprocess_img\n\n            y_linear[i] = self.labels['linear'][ID]\n            y_angular[i] = self.labels['angular'][ID]\n        \n        return X, {'linear': y_linear, 'angular': y_angular}\n\n    def __getitem__(self, index):\n        'Generate one batch of data'\n        indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]\n\n        list_IDs_temp = [self.list_IDs[k] for k in indexes]\n\n        X, y = self.__data_generation(list_IDs_temp)\n\n        return X, y","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Keras model\n","metadata":{}},{"cell_type":"code","source":"img_in = KL.Input(shape=(120, 160, 1), name='img_in')\nx = img_in\n\nx = KL.Convolution2D(filters=24, kernel_size=(5, 5), strides=(2, 2), activation='relu')(x)\nx = KL.Convolution2D(filters=32, kernel_size=(5, 5), strides=(2, 2), activation='relu')(x)\nx = KL.Convolution2D(filters=64, kernel_size=(5, 5), strides=(2, 2), activation='relu')(x)\nx = KL.Convolution2D(filters=64, kernel_size=(3, 3), strides=(2, 2), activation='relu')(x)\nx = KL.Convolution2D(filters=64, kernel_size=(3, 3), strides=(1, 1), activation='relu')(x)\n\nx = KL.Flatten(name='flattened')(x)\nx = KL.Dense(units=100, activation='linear')(x)\nx = KL.Dropout(rate=.1)(x)\nx = KL.Dense(units=50, activation='linear')(x)\nx = KL.Dropout(rate=.1)(x)\n\nlinear = KL.Dense(units=1, activation='linear', name='linear')(x)\n\nangular = KL.Dense(units=1, activation='linear', name='angular')(x)\n\nmodel = K.Model(inputs=[img_in], outputs=[linear, angular])\n\nwith tf.device(device):\n    model.compile(optimizer='adam',\n                  loss={'linear': 'mean_squared_error', 'angular': 'mean_squared_error'},\n                  loss_weights={'linear': 0.3, 'angular': 0.7})","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"callbacks = [\n        K.callbacks.ModelCheckpoint(save_dir, save_best_only=True),\n        K.callbacks.EarlyStopping(monitor='val_loss',\n                                  min_delta=.0005,\n                                  patience=20,\n                                  verbose=True,\n                                  mode='auto'),\n        tfa.callbacks.TQDMProgressBar(),\n        K.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.2,\n                              patience=5, min_lr=0.001),\n        K.callbacks.TensorBoard(log_dir='./logs', profile_batch=(0, 10))\n\n\n    ]\nparams = {'dim': (120, 160),\n          'batch_size': 64,\n          'n_channels': 1,\n          'shuffle': True}\n\n\nwith tf.device(device):\n    training_generator = DataGenerator(partition['train'], labels, **params)\n    validation_generator = DataGenerator(partition['validation'], labels, **params)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Training the model","metadata":{}},{"cell_type":"code","source":"with tf.device(device):\n    hist = model.fit(training_generator,\n                           validation_data=validation_generator,\n                           use_multiprocessing=True,\n                           workers=6,\n                           callbacks=callbacks,\n                           epochs=100)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Converting Keras model to tflite","metadata":{}},{"cell_type":"code","source":"converter = tf.lite.TFLiteConverter.from_keras_model(model)\ntflite_model = converter.convert()\n\nwith open(tflite_model_name + '.tflite', 'wb') as f:\n    f.write(tflite_model)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Custom tests","metadata":{}},{"cell_type":"code","source":"# get prefixes of images names\nfiles = os.listdir(\"../input/\" + root_dir + \"/data\")\ndates = set()\nfor file in files:\n    div = file.split(\"-\")\n    dates.add(div[0])\nprint(\"Available prefixes for the images.\")\ndates","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_file = \"21420221938-img953.jpg\" # can be any image of the data directory\ntest_dimension = (120, 160)\ntest_img = cv2.imread(\"../input/\" + root_dir + \"/data/\" + test_file)\n\nplt.imshow(cv2.cvtColor(test_img, cv2.COLOR_BGR2RGB))\nplt.show()\nprint(\"Shape of the test image: {}\".format(test_img.shape))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"preprocessed_test = preprocess(test_img, test_dimension)\nprint(\"Shape of the preprocessed test image: {}\".format(preprocessed_test.shape))\nplt.imshow(preprocessed_test, cmap=\"gray\")\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Keras prediction","metadata":{}},{"cell_type":"code","source":"keras_input = np.empty((1, 120, 160, 1))\nkeras_input[0,:] = preprocessed_test\nprediction = model.predict(keras_input)\nprediction","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Tflite prediction","metadata":{}},{"cell_type":"code","source":"interpreter = tf.lite.Interpreter(model_path=tflite_model_name+'.tflite')\n\ninput_details = interpreter.get_input_details()\noutput_details = interpreter.get_output_details()\n\ninterpreter.allocate_tensors()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"interpreter.set_tensor(input_details[0]['index'], [preprocessed_test])\n    \ninterpreter.invoke()\n\nlinear_predict = interpreter.get_tensor(output_details[0]['index'])\nangular_predict = interpreter.get_tensor(output_details[1]['index'])\nprint(linear_predict)\nprint(angular_predict)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Ground truth","metadata":{}},{"cell_type":"code","source":"print(\"True values\")\nprint(\"Linear: \" + str(labels['linear'][test_file]))\nprint(\"Angular: \" + str(labels['angular'][test_file]))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualizing Model features","metadata":{}},{"cell_type":"code","source":"def get_conv_layers(trained_model):\n    num = 0\n    nums = []\n    for layer in trained_model.layers:\n        if 'conv' in layer.name:\n            nums.append(num)\n        num += 1\n    return nums","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def visualize_kernels(layer_num=1, trained_model=model, filters_num=6):\n    conv_layers_ids = get_conv_layers(trained_model)\n    \n    if layer_num not in conv_layers_ids:\n        print(\"layer_num is not an index of convolutional layer!\")\n        print(\"Indexes of conv layers:\")\n        print(conv_layers_nums)\n        return\n    filters, biases = trained_model.layers[1].get_weights()\n    filter_min, filter_max = filters.min(), filters.max()\n    \n    filters = (filters - filter_min) / (filter_max - filter_min)\n    max_filters_num = filters.shape[3]\n    if filters_num > max_filters_num:\n        print(\"Number of filters to visualize greater than number of filters in chosen conv layer\")\n        print(\"Number of filters in chosen layer: \" + max_filters_num)\n        \n    ix = 1\n    for i in range(filters_num):\n        f = filters[:,:,:,i]\n        for j in range(1):\n            ax = plt.subplot(filters_num, 3, ix)\n            ax.set_xticks([])\n            ax.set_yticks([])\n            plt.imshow(f[:,:,j], cmap='gray')\n            ix += 1\n        \n    plt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def visualize_feature_maps(layer_num=1, base_model=model, input_img=keras_input, row=8, column=3):\n    '''\n        layer_num - number of convolutional layer which features will be visualized\n        base_model - name of the model that we want to visualize features from\n        input_img - input img for the base model to show its features\n        row - number of maps in a row\n        column - number of maps in a column\n    '''\n    conv_layers_ids = get_conv_layers(base_model)\n    if layer_num not in conv_layers_ids:\n        print(\"layer_num is not an conv layer id\")\n        print(\"Conv layers ids:\", conv_layers_ids)\n        return\n    \n    visualization_model = K.models.Model(inputs=base_model.inputs, outputs=model.layers[layer_num].output)\n    feature_maps = visualization_model.predict(input_img)\n    maps_num = feature_maps.shape[3]\n    \n    if row * column > maps_num:\n        print(\"Specified plot row and column exceeds number of feature maps from chosen layer\")\n        print(\"Number of maps in conv layer \" + str(layer_num) + \" is \" + str(maps_num))\n        print(\"cols * row = \", row*column, sep=\" \")\n        return\n    \n    ix = 1\n    for _ in range(row):\n        for _ in range(column):\n            ax = plt.subplot(column, row, ix)\n            ax.set_xticks([])\n            ax.set_yticks([])\n            plt.imshow(feature_maps[0, :, :, ix-1], cmap='gray')\n            ix += 1\n    plt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"visualize_kernels(layer_num=2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"visualize_feature_maps(column=4, row=2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}
diff --git a/leo_example_line_follower/package.xml b/leo_example_line_follower/package.xml
@@ -0,0 +1,22 @@
+<?xml version="1.0"?>
+<package format="2">
+  <name>leo_example_line_follower</name>
+  <version>0.0.0</version>
+  <description>
+    Neural Network model for line track following Example for Leo Rover.
+  </description>
+
+  <maintainer email="support@fictionlab.pl">Fictionlab</maintainer>
+  <author email="aleks@fictionlab.pl">Aleksander Szymański</author>
+
+  <license>MIT</license>
+
+  <buildtool_depend>catkin</buildtool_depend>
+
+  <exec_depend>sensor_msgs</exec_depend>
+  <exec_depend>geometry_msgs</exec_depend>
+  <exec_depend>cv_bridge</exec_depend>
+  <exec_depend>rospy</exec_depend>
+  <depend>dynamic_reconfigure</depend>
+
+</package>