[Documentation] [TitleIndex] [WordIndex] 

Textured_object_detection guide

Training base format

Training base represents a folder with image and PCD files for every object in special format:

Format of config.txt file:

object_folder_name1 images_in_object_folder

object_folder_name2 images_in_object_folder

...

Creating of training base

Creating of training base currently available only in manual mode. Currently under development tool for the automatic processing bag files.

Initialization

Firstly you should convert topics from your bag file to files on HDD. For this purpose package contains is binary file (image_subscriber.cpp).

For running it usually used "image_sub.launch": 

<launch>
  <node pkg="textured_object_detection" type="is" name="is" output="screen">
      <remap from="ci_left_in" to="/narrow_stereo/left/camera_info"/>
      <remap from="ci_right_in" to="/narrow_stereo/right/camera_info"/>
      
      <remap from="im_left_in" to="/narrow_stereo/left/image_raw"/>
      <remap from="im_right_in" to="/narrow_stereo/right/image_raw"/>
      
      <remap from="imt_left_in" to="/narrow_stereo_textured/left/image_raw"/>
      <remap from="imt_right_in" to="/narrow_stereo_textured/right/image_raw"/>
      
      <remap from="dir" to="/home/alex/temp/test" />
  </node>
  <param name="have_textured" value="true" />
</launch>

You should set correspond names of topics; dir - path, where will be created directories with corresponding files. If your bag file doesn't contain textured images you need set have_textured parameter to false.

Then you should launch this file: 

roslaunch textured_object_detection image_sub.launch

and you should run in parallel rosplay for your bag file: 

rosplay 9403_All.bag

After playing the bag file selected directory (see dir parameter) will be contain next files and subfolders:

Sometimes in order to avoid train/test overlap, we use odd frames for training set, and even -- for the test set. For this purpose you can use half_script.py.

Launch command: 

half_script.py <srcdir> <dstdir> <have_textured>

Script copies all images from left and right (and left_tex and right_tex if <have_textured> > 0) subfolders with odd indexes to corresponding subfolders in <dstdir>. Images from left camera (from left subfolder) with even indexes are copied to test subfolder of <dstdir>. Also it copies left_info.txt and right_info.txt to <dstdir> If you don't set third parameter: 

half_script.py <srcdir> <dstdir>

script init <have_textured> parameter with 1. Also there are some restrictions:

Creating of training base

For creating train base you should use crop binary file from textured_object_detection package (source in crop_object.cpp). This tool working on the following algorithm: we iteratively add samples that are recognized poorly to the training base.

Launch command: 

./crop <srcdir> <object> <dstdir>

<srcdir> must contain next files and subfolders:

If <srcdir> contains left_tex and right_tex subfolders, images from them will be used for disparity calculation.

After launching this tool you will see next window: crop_screenshot.png

Then you should select object on left side of the window. You may do it with left click of your mouse. If distance between last click points less than 20 px, contour will be closed. After contour closing you should press 's' key on your keyboard for saving this object. When next not recognized image will be found, it draws it on window. And you will have to select contour again.

Example of selected object: selected_object.png

After the completion of the program <dstdir> will be contain subfolder with <object> name. After that you should copy this subfolder to your training folder and add to config.txt (in training folder) string from config.txt (in <dstdir>).

Recognition

rosrun textured_object_detection rs training_base_folder_path image

Running as ros node

Package contains ros node (it's source code in node.cpp file) for object recognition. For running it you should use narrow.launch file: 

<launch>
  <node pkg="textured_object_detection" type="rsnode" name="rsnode" output="screen">
    <remap from="image_in" to="/narrow_stereo/left/image_raw"/>
  </node>
  <param name="training_base_dir" value="/home/alex/tod_bases/train/" />
</launch>

You should set training_base_dir parameter to your training base directory and image_in parameter to your image topic.

Then you should run it with command: 

roslaunch textured_object_detection narrow.launch

After ROS_INFO message "Training base reading is complete." node will be ready to object recognizing.

2022-05-28 12:26