Edit on GitHub

[ROS 1] Autonomous Driving

NOTE: This instructions were tested on Ubuntu 16.04 and ROS Kinetic Kame.

The contents in e-Manual can be updated without a previous notice. Therefore, some video may differ from the contents in e-Manual.

The following instruction describes how to build the autonomous driving TurtleBot3 on ROS by using AutoRace packages.

What you need for Autonomous Driving

TurtleBot3 Burger

Remote PC

Raspberry Pi camera module with a camera mount

AutoRace tracks and objects

Install Autorace Packages

The following instructions describes how to install packages and to calibrate camera.

  1. Install AutoRace package on both Remote PC and SBC.
    $ cd ~/catkin_ws/src/
    $ git clone https://github.com/ROBOTIS-GIT/turtlebot3_autorace_2020.git
    $ cd ~/catkin_ws && catkin_make
    
  2. Install additional dependent packages on both Remote PC and SBC.
    $ sudo apt-get install ros-kinetic-image-transport ros-kinetic-cv-bridge ros-kinetic-vision-opencv python-opencv libopencv-dev ros-kinetic-image-proc
    
  3. You need to Calibrate a Camera on SBC.

Calibrate a Camera on SBC

Calibrating the camera is very important for autonomous driving. The following describes how to simply calibrate the camera step by step.

Camera Imaging Calibration

  1. Launch roscore on Remote PC.
    $ roscore
    
  2. Trigger the camera on SBC.
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_camera turtlebot3_autorace_camera_pi.launch
    

    WARNING: Be sure to specify ${Autorace_Misson} (i.e, roslaunch turtlebot3_autorace_traffic_light_camera turtlebot3_autorace_camera_pi.launch)

  3. Execute rqt_image_view on Remote PC.
    $ rqt_image_view
    
  4. Select /camera/image/compressed (or /camera/image/) topic on the check box.

  5. Excute rqt_reconfigure on Remote PC.
    $ rosrun rqt_reconfigure rqt_reconfigure
    
  6. Click camera, and modify parameter value in order to see clear images from the camera.

  7. Open camera.yaml file located in turtlebot3autorace[Autorace Misson]_camera/calibration/camera_calibration folder.

  8. Write modified values to the file.

Intrinsic Camera Calibration

Print a checkerboard on A4 size paper. The checkerboard is used for Intrinsic Camera Calibration.

  1. Launch roscore on Remote PC.
    $ roscore
    
  2. Trigger the camera on SBC.
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_camera turtlebot3_autorace_camera_pi.launch
    

    WARNING: Be sure to specify ${Autorace_Misson} (i.e, roslaunch turtlebot3_autorace_traffic_light_camera turtlebot3_autorace_camera_pi.launch)

  3. Run a intrinsic camera calibration launch file on Remote PC.
    $ export AUTO_IN_CALIB=calibration
    $ export GAZEBO_MODE=false
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_camera turtlebot3_autorace_intrinsic_camera_calibration.launch
    
  4. Use the checkerboard to calibrate the camera, and click CALIBRATE.

  5. Click Save to save the intrinsic calibration data.

  6. calibrationdata.tar.gz folder will be created at /tmp folder.

  7. Extract calibrationdata.tar.gz folder, and open ost.yaml.

    ost.yaml

    Intrinsic Calibration Data in ost.yaml

  8. Copy and paste the data from ost.yaml to camerav2_320x240_30fps.yaml.

    camerav2_320x240_30fps.yaml

    Intrinsic Calibration Data in camerav2_320x240_30fps.yaml

Extrinsic Camera Calibration

  1. Launch roscore on Remote PC.
    $ roscore
    
  2. Trigger the camera on SBC.
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_camera turtlebot3_autorace_camera_pi.launch
    

    WARNING: Be sure to specify ${Autorace_Misson} (i.e, roslaunch turtlebot3_autorace_traffic_light_camera turtlebot3_autorace_camera_pi.launch)

  3. Use the command on Remote PC.
    $ export AUTO_IN_CALIB=action
    $ export GAZEBO_MODE=false
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_camera turtlebot3_autorace_intrinsic_camera_calibration.launch
    
  4. Run the extrinsic camera calibration launch file on Remote PC.
    $ export AUTO_EX_CALIB=calibration
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_camera turtlebot3_autorace_extrinsic_camera_calibration.launch
    
  5. Execute rqt on Remote PC.
    $ rqt
    
  6. Click plugins > visualization > Image view; Multiple windows will be present.

  7. Select /camera/image_extrinsic_calib/compressed and /camera/image_projected_compensated topics on each monitors.
    • One of two screens will show an image with a red rectangle box. The other one shows the ground projected view (Bird’s eye view).

    /camera/image_extrinsic_calib/compressed topic

    /camera/image_projected_compensated topic

  8. Excute rqt_reconfigure on Remote PC.
    $ rosrun rqt_reconfigure rqt_reconfigure
    
  9. Adjust parameters in /camera/image_projection and /camera/image_compensation_projection.
    • Change /camera/image_projection parameter value. It affects /camera/image_extrinsic_calib/compressed topic.
    • Intrinsic camera calibration will transform the image surrounded by the red rectangle, and will show the image that looks from over the lane.

    rqt_reconfigure

    Result from parameter modification.

Settings for Recognition

When you complete all the camera calibration (Camera Imaging Calibration, Intrinsic Calibration, Extrinsic Calibration), be sure that the calibration is successfully applied to the camera.
The following instruction describes settings for recognition.

  1. Launch roscore on Remote PC.
    $ roscore
    
  2. Trigger the camera on SBC.
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_camera turtlebot3_autorace_camera_pi.launch
    

    WARNING: Be sure to specify ${Autorace_Misson} (i.e, roslaunch turtlebot3_autorace_traffic_light_camera turtlebot3_autorace_camera_pi.launch)

  3. Run a intrinsic camera calibration launch file on Remote PC.
    $ export AUTO_IN_CALIB=action
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_camera turtlebot3_autorace_intrinsic_camera_calibration.launch
    
  4. Open terminal and use the command on Remote PC.
    $ export AUTO_EX_CALIB=action
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_camera turtlebot3_autorace_extrinsic_camera_calibration.launch
    

    From now, the following descriptions will mainly adjust feature detector / color filter for object recognition. Every adjustment after here is independent to each other’s process. However, if you want to adjust each parameters in series, complete every adjustment perfectly, then continue to next.

Lane Detection

Lane detection package allows Turtlebot3 to drive between two lanes without external influence.

The following instructions describe how to use the lane detection feature and to calibrate camera via rqt.

  1. Place TurtleBot3 between yellow and white lanes.

    NOTE: Be sure that yellow lane is placed left side of the robot and White lane is placed right side of the robot.

  2. Launch roscore on Remote PC.
    $ roscore
    
  3. Trigger the camera on SBC.
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_camera turtlebot3_autorace_camera_pi.launch
    

    WARNING: Be sure to specify ${Autorace_Misson} (i.e, roslaunch turtlebot3_autorace_traffic_light_camera turtlebot3_autorace_camera_pi.launch)

  4. Run a intrinsic camera calibration launch file on Remote PC.
    $ export AUTO_IN_CALIB=action
    $ export GAZEBO_MODE=false
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_camera turtlebot3_autorace_intrinsic_camera_calibration.launch
    
  5. Run a extrinsic camera calibration launch file on Remote PC.
    $ export AUTO_EX_CALIB=action
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_camera turtlebot3_autorace_extrinsic_camera_calibration.launch
    
  6. Run a lane detection launch file on Remote PC
    $ export AUTO_DT_CALIB=calibration
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_detect turtlebot3_autorace_detect_lane.launch
    
  7. Execute rqt on Remote PC.
    $ rqt
    
  8. Click plugins > visualization > Image view; Multiple windows will be present.

  9. Select three topics at each image view: /detect/image_yellow_lane_marker/compressed, /detect/image_lane/compressed, /detect/image_white_lane_marker/compressed
    • Left (Yellow line) and Right (White line) screen show a filtered image.

      Image view of /detect/image_yellow_lane_marker/compressed topic

    Image view of /detect/image_white_lane_marker/compressed topic

    • Center screen is the view of the camera from TurtleBot3.

      Image view of /detect/image_lane/compressed topic

  10. Execute rqt_reconfigure on Remote PC.
    $ rosrun rqt_reconfigure rqt_reconfigure
    
  11. Click Detect Lane then adjust parameters to do line color filtering.

    List of Detect Lane Parameters

    Filtered Image resulted from adjusting parameters at rqt_reconfigure

    TIP: Calibration process of line color filtering is sometimes difficult due to physical environment, such as the luminance of light in the room and etc. To make everything quickly, put the value of lane.yaml file located in turtlebot3autorace[Autorace_Misson]_detect/param/lane/ on the reconfiguration parameter, then start calibration. Calibrate hue low - high value at first. (1) Hue value means the color, and every colors, like yellow, white, have their own region of hue value (refer to hsv map). Then calibrate saturation low - high value. (2) Every colors have also their own field of saturation. Finally, calibrate the lightness low - high value. (3) In the source code, however, have auto-adjustment function, so calibrating lightness low value is meaningless. Just put the lightness high value to 255. Clearly filtered line image will give you clear result of the lane.

  12. Open lane.yaml file located in turtlebot3autorace[Autorace_Misson]_detect/param/lane/. You need to write modified values to the file. This will make the camera set its parameters as you set here from next launching.

  13. Close both rqt_rconfigure and turtlebot3_autorace_detect_lane.

  14. Open terminal and use the command on Remote PC.
    $ export AUTO_DT_CALIB=action
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_detect turtlebot3_autorace_detect_lane.launch
    
  15. Check if the results come out correctly.

    • Open terminal and use the command on Remote PC.
       $ roslaunch turtlebot3_autorace_${Autorace_Misson}_control turtlebot3_autorace_control_lane.launch
      
    • Open terminal and use the command on Remote PC.
       $ roslaunch turtlebot3_bringup turtlebot3_robot.launch
      
  16. After using the commands, TurtleBot3 will start to run.

Traffic Sign Detection

TurtleBot3 can detect traffic signs using a node with SIFT algorithm, and perform programmed tasks while it drives on a built track. Follow the provided instructions to use Traffic sign detection.

NOTE: More edges in the traffic sign increase recognition results from SIFT.

  1. Take pictures of traffic signs by using TurtleBot3’s camera and rqt_image_view node.
  2. Edit the pictures using a photo editor that can be used in Linux OS.
  3. Put TurtleBot3 on the lane. Traffic signes should be placed where TurtleBot3 can see them easily.

    NOTE: Do not have TurtleBot3 run on the lane yet.

  4. Execute rqt_image_view on Remote PC.
    $ rqt_image_view
    
  5. Select /camera/image_compensated topic in the select box. A screen will show the view from TurtleBot3.
  6. Capture the picture from Remote PC,and edit it with a photo editor.
  7. Place the edited picture to turtlebot3_autorace package you’ve placed /turtlebot3_autorace/turtlebot3_autorace_detect/file/detect_sign/ and rename it as you want. (Although, you should change the file name written in the source detect_sign.py file, if you want to change the default file names.)

  8. Open terminal and use the command on Remote PC.
    $ roslaunch turtlebot3_autorace_${Autorace_Misson}_detect turtlebot3_autorace_detect_sign.launch
    
  9. Open terminal and use the command on Remote PC.
    $ rqt_image_view
    
  10. Select /detect/image_traffic_sign/compressed topic in the select box. A screen will show the result of traffic sign detection.

TurtleBot3 AutoRace 2020

The AutoRace is a competition for autonomous driving robot platforms.
To provide various conditions for a robot application development, the game provide structural regulation as less as possible. Provided open sources are based on ROS, and can be applied to this competition. The contents can be continually updated. Join the competition and show your skill.

WARNING: Be sure to read Autonomous Driving in order to start missions.

Mission 1: Traffic Lights

Traffic Light is the first mission of AutoRace. TurtleBot3 recognizes the traffic lights and starts the course.

Camera Calibration for Traffic Lights

  1. Open terminal and use the command on Remote PC.
    $ export AUTO_DT_CALIB=calibration
    $ roslaunch turtlebot3_autorace_traffic_light_detect turtlebot3_autorace_detect_traffic_light.launch
    
  2. Excute rqt on Remote PC.
    rqt
    
  3. Select four topics: /detect/image_red_light, /detect/image_yellow_light, /detect/image_green_light, /detect/image_traffic_light.

  4. Excute rqt_reconfigure.
    $ rosrun rqt_reconfigure rqt_reconfigure
    
  5. Adjust parameters regarding traffic light topics to enhance the detection of traffic signs. The way of adjusting parameters is similar to step 5 at Lane Detection.

  6. Open traffic_light.yaml file located at turtlebot3_autorace_traffic_light_detect/param/traffic_light/.

  7. Write modified values to the file and save.

  8. Terminate both running rqt and rqt_reconfigure in order to test, from the next step, the calibration whether or not it is successfully applied.

  9. Use the command on Remote PC.
    $ export AUTO_DT_CALIB=action
    $ roslaunch turtlebot3_autorace_traffic_light_detect turtlebot3_autorace_detect_traffic_light.launch
    
  10. Execute rqt_image_view.
    $ rqt_image_view
    
  11. See traffic light calibration is successfully applied.

Run Traffic Light Mission

WARNING: Be sure to read Camera Calibration for Traffic Lights before running the traffic light node.

  1. Use the command on Remote PC.
    $ export AUTO_IN_CALIB=action
    $ export GAZEBO_MODE=false
    $ roslaunch turtlebot3_autorace_traffic_light_camera turtlebot3_autorace_intrinsic_camera_calibration.launch
    
  2. Use the command on Remote PC.
    $ export AUTO_EX_CALIB=action
    $ export AUTO_DT_CALIB=action
    $ roslaunch turtlebot3_autorace_traffic_light_core turtlebot3_autorace_core.launch
    
  3. Use the command on Remote PC.
    $ rostopic pub -1 /core/decided_mode std_msgs/UInt8 "data: 3"
    

Mission 2: Intersection

Intersection is the second mission of AutoRace. TurtleBot3 detects a specific traffic sign (such as a curve sign) at the intersection course, and go to the given direction.

How to Run Intersection Mission

  1. Use the command on Remote PC.
    $ export AUTO_IN_CALIB=action
    $ roslaunch turtlebot3_autorace_intersection_camera turtlebot3_autorace_intrinsic_camera_calibration.launch
    
  2. Use the command on Remote PC.
    $ export AUTO_EX_CALIB=action
    $ export AUTO_DT_CALIB=action
    $ roslaunch turtlebot3_autorace_intersection_core turtlebot3_autorace_core.launch
    
  3. Use the command on Remote PC.
    $ rostopic pub -1 /core/decided_mode std_msgs/UInt8 "data: 2"
    

Mission 3: Construction

Construction is the third mission of AutoRace. TurtleBot3 avoids constructions on the track while it is driving.

How to Run Construction Mission

  1. Use the command on Remote PC.
    $ export AUTO_IN_CALIB=action
    $ export GAZEBO_MODE=false
    $ roslaunch turtlebot3_autorace_construction_camera turtlebot3_autorace_intrinsic_camera_calibration.launch
    
  2. Use the command on Remote PC.
    $ export AUTO_EX_CALIB=action
    $ export AUTO_DT_CALIB=action
    $ roslaunch turtlebot3_autorace_construction_core turtlebot3_autorace_core.launch
    
  3. Use the command on Remote PC.
    $ rostopic pub -1 /core/decided_mode std_msgs/UInt8 "data: 2"
    

Mission 4: Parking

Parking is the fourth mission of AutoRace. TurtleBot3 detects the parking sign, and park itself at a parking lot.

How to Run Parking Mission

  1. Use the command on Remote PC.
    $ export AUTO_IN_CALIB=action
    $ export GAZEBO_MODE=false
    $ roslaunch turtlebot3_autorace_parking_camera turtlebot3_autorace_intrinsic_camera_calibration.launch
    
  2. Use the command on Remote PC.
    $ export AUTO_EX_CALIB=action
    $ export AUTO_DT_CALIB=action
    $ roslaunch turtlebot3_autorace_parking_core turtlebot3_autorace_core.launch
    
  3. Use the command on Remote PC.
    $ rostopic pub -1 /core/decided_mode std_msgs/UInt8 "data: 2"
    

Mission 5: Level Crossing

Level Crossing is the fifth mission of AutoRace. When TurtleBot3 encounters the level crossing, it stops driving, and wait until the level crossing opens.

Camera Calibration for Level Crossing

  1. Use the command on Remote PC.
    $ export AUTO_DT_CALIB=calibration
    $ roslaunch turtlebot3_autorace_level_crossing_detect turtlebot3_autorace_detect_level.launch
    
  2. Execute rqt
    $ rqt
    
  3. Select two topics: /detect/image_level_color_filtered, /detect/image_level

  4. Execute rqt_reconfigure.
    $ rosrun rqt_reconfigure rqt_reconfigure
    
  5. Select /detect_level and adjust parameters regarding Level Crossing topics to enhance the detection of the level crossing object. The way of adjusting parameters is similar to step 5 at Lane Detection.

  6. Open level.yaml located at turtlebot3_autorace_stop_bar_detect/param/level/.

  7. Write modified values to the file and save.

  8. Run a detect level lanuch file.
    $ export AUTO_DT_CALIB=action
    $ roslaunch turtlebot3_autorace_detect turtlebot3_autorace_detect_level.launch
    

How to Run Level Crossing Mission

  1. Use the command on Remote PC.
    $ export AUTO_IN_CALIB=action
    $ export GAZEBO_MODE=false
    $ roslaunch turtlebot3_autorace_level_crossing_camera turtlebot3_autorace_intrinsic_camera_calibration.launch
    
  2. Use the command on Remote PC.
    $ export AUTO_EX_CALIB=action
    $ export AUTO_DT_CALIB=action
    $ roslaunch turtlebot3_autorace_level_crossing_core turtlebot3_autorace_core.launch
    
  3. Use the command on Remote PC.
    $ rostopic pub -1 /core/decided_mode std_msgs/UInt8 "data: 2"
    

Mission 6: Tunnel

Tunnel is the sixth mission of AutoRace. TurtleBot3 passes the tunnel successfully.

How to Run Tunnel Mission

  1. Use the command on Remote PC.
    $ export AUTO_IN_CALIB=action
    $ export GAZEBO_MODE=false
    $ roslaunch turtlebot3_autorace_tunnel_camera turtlebot3_autorace_intrinsic_camera_calibration.launch
    
  2. Use the command on Remote PC.
    $ export AUTO_EX_CALIB=action
    $ export AUTO_DT_CALIB=action
    $ export TURTLEBOT3_MODEL=burger
    $ roslaunch turtlebot3_autorace_tunnel_core turtlebot3_autorace_core.launch
    
  3. Use the command on Remote PC.
    $ rostopic pub -1 /core/decided_mode std_msgs/UInt8 "data: 2"