Home Service Challenge Stadium and Objects

Getting Started

Prerequisites

TurtleBot3 Waffle Pi

• It is the basic model to use Home Service Challenge packages for mobile manipulation.
• Provided source codes, Home Service Challenge Packages, are made based on TurtleBot3 Waffle Pi.

OpenMANIPULATOR-X

• OpenMANIPULATOR-X is an official robot platform to use mobile manipulation feature with Turtlebot3 in the Home Service Challenge.

Remote PC

• It communicates with an single board computer (SBC) on Turtlebot3.
• Laptop, desktop, or other devices with ROS 1.

Remote PC Setup

1. [Remote PC] Install Home Service Challenge packages.

   $ cd ~/catkin_ws/src/
   $ git clone -b kinetic-devel https://github.com/ROBOTIS-GIT/turtlebot3_home_service_challenge.git
   $ cd ~/catkin_ws && catkin_make
   

2. [Remote PC] Install additional dependent packages on Remote PC

   $ sudo apt-get install ros-kinetic-ar-track-alvar ros-kinetic-ar-track-alvar-msgs
   

3. [Remote PC] Load the TurtleBot3 Waffle (or Waffle Pi) with OpenMANIPULATOR on RViz.

   $ export TURTLEBOT3_MODEL=${TB3_MODEL}
   $ roslaunch turtlebot3_manipulation_description turtlebot3_manipulation_view.launch use_gui:=true
   

   NOTE: Specify ${TB3_MODEL}: waffle, waffle_pi before excuting the command. Set the permanent export setting by following Export TURTLEBOT3_MODEL instruction.

   

   Rviz View

4. [Remote PC] In order to use your PC remotely, you need to make SSH keys. Create a script file to generate SSH key.

   $ nano ~/tb3_ssh_keygen
   

5. [Remote PC] Copy and paste the followings to the script, and save it.

   #!/bin/bash
   argc=$#
   args=("$@")
   if [ 0 -eq $argc ]
   then
     echo "need to argument that host ip for ssh connection"
     echo "Usage: $0 [ip address] ..."
     exit 1
   fi
   for((index = 0; index < $#; index++ ))
   do
     ssh-keygen -R ${args[$index]}
     ssh-keyscan ${args[$index]} >> ~/.ssh/known_hosts
   done
   

6. [Remote PC] Make the file executable.

   $ chmod +x ~/tb3_ssh_keygen
   

7. [Remote PC] Execute the file with TurtleBot3’s IP address. Be sure Remote PC and TurtleBot3 are connected under the same WiFi.

   $ ~/tb3_ssh_keygen {IP_ADDRESS_OF_TURTLEBOT3}
   

8. Check ~/.ssh/known_hosts file if SSH is successfully generated.

   

SBC Setup

1. [Turtlebot3 SBC] Using a machine tag, create env.bash file located in the /home/pi.

   $ nano ~/env.bash
   

2. [Turtlebot3 SBC] Copy and paste the followings to the script, and save it.

   #!/bin/bash
   # check if other turtlebot3_core is already running
   is_running=`ps ax | grep turtlebot3_core`
   IFS=' ' read -ra is_runnings <<< "$is_running"
   process_name=${is_runnings[4]}
   if [ ${process_name} == "python" ]
   then
     echo "other turtlebot3_core is already running."
     exit 1
   fi
   #### ROS ####
   source /opt/ros/kinetic/setup.bash
   source ~/catkin_ws/devel/setup.bash
   #### ROS Network ####
   ip_address=`hostname -I`
   ip_address_trim=${ip_address%% * }
   ip_address_no_space="$(echo -e "${ip_address_trim}" | tr -d '[:space:]')"
   export ROS_HOSTNAME=${ip_address_no_space}
   ##### Set TURTLEBOT3 Model ####
   export TURTLEBOT3_MODEL=waffle_pi
   exec "$@"
   

3. Make the file executable.

   $ chmod +x ~/env.bash
   

OpenCR Setup

• See the OpenCR Setup

Hardware Setup

• See the Hardware Setup

Camera Calibration

Calibrate a camera to decrease or minimize distortion. Follow the camera calibration process step by step.

Camera Setting

1. [Remote PC] Use the following command to run a camera remotely. Make sure to type IP address where TurtleBot3 and remote pc are connected together.
   Be sure Remote PC and TurtleBot3 are connected under the same IP.

   $ roslaunch turtlebot3_home_service_challenge_tools turtlebot3_pi_cam_remote.launch address:=TURTLEBOT_IP
   

2. [Remote PC] Execute rqt_image_view.

   $ rqt_image_view
   

3. Select /raspicam_node/image/compressed (or /raspicam_node/image/) topic on the check box.

   

4. [Remote PC] Excute rqt_reconfigure.

   $ rosrun rqt_reconfigure rqt_reconfigure
   

5. Click raspicam_node, and modify parameter value to see clear images from the camera.

   

6. Open camera.yaml file located in turtlebot3_home_service_challenge_tools/config/camera_calibration.

7. Write modified values to the file.

   ISO: 400
   awb_mode: auto
   brightness: 50
   contrast: 0
   exposureCompensation: 0
   exposure_mode: auto
   hFlip: false
   saturation: 0
   sharpness: 0
   shutterSpeed: 0
   vFlip: false
   videoStabilisation: false
   zoom: 1.0
   

Intrinsic Camera Calibration

Print a checkerboard on A4 size paper. The checkerboard is used for Intrinsic Camera Calibration.

• Download a checkerboard from ROS Wiki

• For detailed information on the camera calibration, see camera_calibration from ROS Wiki.

  

  Checkerboard

1. [Remote PC] Use the following command to run a camera remotely. Make sure to include IP address when using the command.

   $ roslaunch turtlebot3_home_service_challenge_tools turtlebot3_pi_cam_remote.launch enable_raw:=true address:=TURTLEBOT_IP
   

   NOTE: Be sure Remote PC and TurtleBot3 are connected under the same IP.

2. [Remote PC] Run a intrinsic camera calibration.

   $ rosrun camera_calibration cameracalibrator.py --size 8x6 --square 0.024 image:=/raspicam_node/image camera:=/raspicam_node
   

   NOTE: The size of squres may differ depending on the print paper size A3, A4 or others. In which case, adjust the value (0.024) of –square option in the given command according to the square size in the print paper.

   

3. Use the checkerboard to calibrate the camera, and click CALIBRATE.

   NOTE: Move and tilt the checker board around to the left, right, top, bottom and various angles to get X,Y, Size and Skew calibration data. As the data is computed enough, X,Y, Size and Skew will have a green bar.

   

4. Click Commit to save the intrinsic calibration data to the default folder.(~/.ros/camera_info)

   

5. Open camerav2_1280x720.yaml file located ~/.ros/camera_info from Turtlebot SBC and check the saved data, which has the form like the script below.

   image_width: 1280
   image_height: 720
   camera_name: camerav2_1280x720
    camera_matrix:
    rows: 3
    cols: 3
    data: [1280.628466572832, 0, 658.384219880362, 0, 1277.989132765768, 360.8363923435625, 0, 0, 1]
   distortion_model: plumb_bob
   distortion_coefficients:
    rows: 1
    cols: 5
    data: [0.2032697817127332, -0.4147569733010375, -0.001420915782245743, 0.003110652248245379, 0]
   rectification_matrix:
    rows: 3
    cols: 3
    data: [1, 0, 0, 0, 1, 0, 0, 0, 1]
   projection_matrix:
    rows: 3
    cols: 4
    data: [1312.630981445312, 0, 661.4149459665205, 0, 0, 1312.107055664062, 360.0239481801327, 0, 0, 0, 1, 0]
   

SLAM

SLAM stands for Simultaneous Localization and Mapping. Using this feature, TurtleBot3 can draw a map. More information on this, see SLAM

1. [Remote PC] Use the following command to run a camera remotely. Make sure to include IP address when using the command.

   $ roslaunch turtlebot3_home_service_challenge_tools turtlebot3_robot_remote.launch address:=TURTLEBOT_IP
   

   NOTE: Be sure Remote PC and TurtleBot3 are connected under the same IP.

2. [Remote PC] Run slam node.

   $ roslaunch turtlebot3_home_service_challenge_tools slam.launch
   

3. [Remote PC] Run keyboard teleoperation node.

   $ roslaunch turtlebot3_home_service_challenge_tools mobile_teleop_key.launch
   

   After the node is run, the following controller will be displayed on a terminal window.

    Control Your TurtleBot3!
    ---------------------------
    Moving around:
      w
    a s d
      x
   
    w/x : increase/decrease linear velocity
    a/d : increase/decrease angular velocity
    space key, s : force stop
   
    CTRL-C to quit
   

4. Update a map with the keyboard controller.

   

5. [Remote PC] Save the map when it is fully updated.

   $ ROS_NAMESPACE=tb3_hsc rosrun map_server map_saver -f tb3_hsc
   

   

Missions

Run a Demo and Manager Pacakge

1. [Remote PC] Run the demo pacakge. Make sure to include IP address when using the command. This package contains various features: Remote Bringup with a machine tag, Moveit, Navigation, Camera sensing

   $ roslaunch turtlebot3_home_service_challenge_tools turtlebot3_home_service_challenge_demo_remote.launch address:=TURTLEBOT_IP
   

   NOTE: Be sure Remote PC and TurtleBot3 are connected under the same IP.

   

2. [Remote PC] Run the manager package used to carry out Home Service Challenge’s mission.

   $ roslaunch turtlebot3_home_service_challenge_manager manager.launch
   

Commands

[Remote PC] Use the following commands during Home Service Challenge.

• Ready : TurtleBot3 will prepare to start a mission.

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String ready_mission
  

• Start : TurtleBot3 will start a mission.

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String start_mission
  

• Stop : TurtleBot3 will stop running a mission.

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String stop_mission
  

• Restart : TurtleBot3 will restart a mission by a given scenario.

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String restart_mission:SCENARIO_NAME
  

  NOTE: When using this command, be sure to include one of the senario name from a scenario.yaml file. For detailed information on the scenario, see Configuration description below at this section.

Operation Test

[Remote PC] Publish the following topics to test a navigation or manipulation feature.

• Navigation

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String nav_start
  $ rostopic pub -1 /tb3_hsc/command std_msgs/String nav_ar_marker_0
  $ rostopic pub -1 /tb3_hsc/command std_msgs/String nav_ar_marker_1
  $ rostopic pub -1 /tb3_hsc/command std_msgs/String nav_ar_marker_2
  

• Manipulation

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String arm_home
  $ rostopic pub -1 /tb3_hsc/command std_msgs/String arm_joint
  $ rostopic pub -1 /tb3_hsc/command std_msgs/String arm_task
  $ rostopic pub -1 /tb3_hsc/command std_msgs/String open_gripper
  $ rostopic pub -1 /tb3_hsc/command std_msgs/String close_gripper
  

Configuration

[Remote PC] Modify data in configuration files according to the given environment.

• scenario.yaml : This file contains a scenario’s data.
  • File Path : /turtlebot3_home_service_challenge_manager/script/scenario.yaml
  • Script

    SCENARIO_NAME: # start - scenario - finish
      task: TASK_NAME
      args: [0, 1, 2]
      timeout: 10 #sec, 0 : no time out
      next_scenario: find_object
      scenario_on_failure: standby
      retry_times: 0 #times, 0 : no retry
    

• room.yaml : This file contains data of the Home Service Challenge’s stadium.
  • File Path : /turtlebot3_home_service_challenge_manager/config/room.yaml
  • Script

    room_1:
      name: toilet
      object:
        marker: ar_marker_0
        position: [0.25, 0, 0.15]
      target:
        marker: ar_marker_4
        position: [0.25, 0, 0.15]
      x: [1.5, 0.6]
      y: [1.5, 0.2]
    

• config.yaml : This configuration file contains manager package’s data.
  • File Path : /turtlebot3_home_service_challenge_manager/config/config.yaml

Details about the Home Service Mission

The goal of the Home Service Challenge is to move four different objects from a living room to a specific place with given rules, and to return to the starting point.

Using the demo package, the process of moving objects in Home Service Challenge is as follows.

1. Navigating to a target in the living room.
   • Find a target, and reach it using a Navigation package.

   

2. Approaching the target.
   • For the approach to the target with precise, TurtleBot3 wheels are directly controlled by computing target’s location from AR marker. (Used Topic : /tb3_hsc/cmd_vel). To produce a reliable performance, Closed-loop and control system can be used for the specified number of times.

   

3. Picking the target with OpenMANIPULATOR-X’s gripper.
   • Pick the target object using the moveit package (Joint space control, Task space control and Gripper control)

   

   MoveIt Diagram

   

   Picking a Target Using MoveIt Package.

4. Leaving for the next room to place the object (Used Topic : /tb3_hsc/cmd_vel)
   • When moving back from the target, the wheels are directly controlled by the manager program using /tb3_hsc/cmd_vel topic.
5. Navigating to the place where the object will be placed.
   • Find a next target, and reach it using a Navigation package.

   

6. Approaching the target.

7. Placing the object using the gripper.

8. Returning to the starting point using the navigation package.

   

Simulation

Simulate TurtleBot3 with OpenMANIPULATOR-X in Gazebo.

1. [Remote PC] Run Gazebo.

   $ roslaunch turtlebot3_home_service_challenge_simulation competition.launch
   

   

2. [Remote PC] Run a simulation demo for Gazebo.

   $ roslaunch turtlebot3_home_service_challenge_tools turtlebot3_home_service_challenge_demo_simulation.launch
   

   

3. [Remote PC] Run Home Service Manager.

   $ roslaunch turtlebot3_home_service_challenge_manager manager.launch
   

4. Use the Home Service Challenge commands, See Commands

Home Service Challenge Stadium and Objects

Getting Started

Prerequisites

Remote PC

• ROS 1 Noetic installed Laptop or desktop PC.
• This instruction is based on Gazebo simulation.

Remote PC Setup

1. [Remote PC] Install Home Service Challenge packages.

   $ cd ~/catkin_ws/src/
   $ git clone -b noetic-devel https://github.com/ROBOTIS-GIT/turtlebot3_home_service_challenge.git
   $ git clone -b noetic-devel https://github.com/machinekoder/ar_track_alvar
   $ cd ~/catkin_ws && catkin_make
   

2. [Remote PC] Load the TurtleBot3 Waffle (or Waffle Pi) with OpenMANIPULATOR on RViz.

   $ export TURTLEBOT3_MODEL=${TB3_MODEL}
   $ roslaunch turtlebot3_manipulation_description turtlebot3_manipulation_view.launch use_gui:=true
   

   NOTE: Specify ${TB3_MODEL}: waffle, waffle_pi before excuting the command. Set the permanent export setting by following Export TURTLEBOT3_MODEL instruction.

   

   Rviz View. Specify ${TB3_MODEL} : waffle_pi

Camera Calibration

SLAM

Missions

Run a Demo and Manager Pacakge

1. [Remote PC] Run the Gazebo Simulation.

   $ roslaunch turtlebot3_home_service_challenge_simulation competition.launch
   

2. [Remote PC] Run a simulation demo for Gazebo.

   $ roslaunch turtlebot3_home_service_challenge_tools turtlebot3_home_service_challenge_demo_simulation.launch
   

   

3. [Remote PC] Run the manager package used to carry out Home Service Challenge’s mission.

   $ roslaunch turtlebot3_home_service_challenge_manager manager.launch
   

Commands

[Remote PC] Use the following commands during Home Service Challenge.

• Ready : TurtleBot3 will prepare to start a mission.

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String ready_mission
  

• Start : TurtleBot3 will start a mission.

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String start_mission
  

• Stop : TurtleBot3 will stop running a mission.

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String stop_mission
  

• Restart : TurtleBot3 will restart a mission by a given scenario.

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String restart_mission:SCENARIO_NAME
  

  NOTE: When using this command, be sure to include one of the senario name from a scenario.yaml file. For detailed information on the scenario, see Configuration description below at this section.

Operation Test

[Remote PC] Publish the following topics to test a navigation or manipulation feature.

• Navigation

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String nav_start
  

  

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String nav_ar_marker_0
  

  

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String nav_ar_marker_1
  

  

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String nav_ar_marker_2
  

  

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String nav_ar_marker_3
  

  

• Manipulation

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String arm_home
  

  

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String arm_joint
  

  

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String arm_task
  

  

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String open_gripper
  

  

  $ rostopic pub -1 /tb3_hsc/command std_msgs/String close_gripper
  

  

Configuration

[Remote PC] Modify data in configuration files according to the given environment.

• scenario.yaml : This file contains a scenario’s data.
  • File Path : /turtlebot3_home_service_challenge_manager/script/scenario.yaml
  • Script

    SCENARIO_NAME: # start - scenario - finish
      task: TASK_NAME
      args: [0, 1, 2]
      timeout: 10 #sec, 0 : no time out
      next_scenario: find_object
      scenario_on_failure: standby
      retry_times: 0 #times, 0 : no retry
    

• room.yaml : This file contains data of the Home Service Challenge’s stadium.
  • File Path : /turtlebot3_home_service_challenge_manager/config/room.yaml
  • Script

    room_1:
      name: toilet
      object:
        marker: ar_marker_0
        position: [0.25, 0, 0.15]
      target:
        marker: ar_marker_4
        position: [0.25, 0, 0.15]
      x: [1.5, 0.6]
      y: [1.5, 0.2]
    

• config.yaml : This configuration file contains manager package’s data.
  • File Path : /turtlebot3_home_service_challenge_manager/config/config.yaml

Details about the Home Service Mission

The goal of the Home Service Challenge is to move four different objects from a living room to a specific place with given rules, and to return to the starting point.

Using the demo package, the process of moving objects in Home Service Challenge is as follows.

1. Navigating to a target in the living room.
   • Find a target, and reach it using a Navigation package.

   

2. Approaching the target.
   • For the approach to the target with precise, TurtleBot3 wheels are directly controlled by computing target’s location from AR marker. (Used Topic : /tb3_hsc/cmd_vel). To produce a reliable performance, Closed-loop and control system can be used for the specified number of times.

   

3. Picking the target with OpenMANIPULATOR-X’s gripper.
   • Pick the target object using the moveit package (Joint space control, Task space control and Gripper control)

   

   MoveIt Diagram

4. Leaving for the next room to place the object (Used Topic : /tb3_hsc/cmd_vel)
   • When moving back from the target, the wheels are directly controlled by the manager program using /tb3_hsc/cmd_vel topic.
5. Navigating to the place where the object will be placed.
   • Find a next target, and reach it using a Navigation package.

   

6. Approaching the target.

7. Placing the object using the gripper.

8. Returning to the starting point using the navigation package.

   

Simulation

Simulate TurtleBot3 with OpenMANIPULATOR-X in Gazebo.

1. [Remote PC] Run Gazebo.

   $ roslaunch turtlebot3_home_service_challenge_simulation competition.launch
   

   

2. [Remote PC] Run a simulation demo for Gazebo.

   $ roslaunch turtlebot3_home_service_challenge_tools turtlebot3_home_service_challenge_demo_simulation.launch
   

   

3. [Remote PC] Run Home Service Manager.

   $ roslaunch turtlebot3_home_service_challenge_manager manager.launch
   

4. Use the Home Service Challenge commands, See Commands

NOTE: This feature is available for Kinetic and Noetic.

NOTE: This feature is available for Kinetic and Noetic.

NOTE: This feature is available for Kinetic and Noetic.

NOTE: This feature is available for Kinetic and Noetic.