Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
tutorials:demo:fetch_and_place [2019/10/18 11:01] arthurtutorials:demo:fetch_and_place [2021/12/13 10:42] (current) – [Tutorial 0: Getting into Emacs] arthur
Line 3: Line 3:
 This tutorial is from the "Demo Tutorials" category, which assumes no prior knowledge of Linux, ROS, Emacs, Lisp or robotics. This tutorial is from the "Demo Tutorials" category, which assumes no prior knowledge of Linux, ROS, Emacs, Lisp or robotics.
  
 +===== Important Links =====
  
 +  * The {{ :tutorials:demo:cram_fall_school_2021.pdf |presentation slides}} give a brief overview and introduction for this lecture and contain the most important informations on Emacs, our IDE for Lisp.
 +  * The [[https://seafile.zfn.uni-bremen.de/f/a28c1de4acab4609b4f3/?dl=1|virtual machine (VM) file]] to run with VirtualBox.
 +  * An explanation of the setup and first tutorial is also available with voice explanations (around 1h) as [[https://seafile.zfn.uni-bremen.de/f/c057cd48e1244d7997b8/|download]] or [[https://www.youtube.com/watch?v=0uJN-jRb7J4&feature=youtu.be|stream]], for those with spare time on another day.
  
  
 +===== Motivation =====
 +
 +The aim of the tutorial is to give an intuition about the complexity of robot mobile manipulation tasks and about the knowledge that is required to execute actions successfully.
 +
 +The tutorial:
 +  - Gives an intuition of what knowledge does the robot need (and how much knowledge) to execute even a simple fetch and place, e.g., robot needs to answer questions such as "where should I stand?", "which grasp pose should I use?", etc.
 +  - Shows how many different things can go wrong, and teaches writing simple failure handling strategies.
 +  - Teaches how to call existing actions from the CRAM framework, i.e. how to use designators (but not program new ones).
 +  
  
 ===== Setting Up ===== ===== Setting Up =====
Line 15: Line 28:
 ==== Technical Requirements ==== ==== Technical Requirements ====
  
-The VM image with VirtualBox has been successfully tested under Windows, MacOS and Ubuntu Linux+  * The VM image with VirtualBox has been successfully tested for Windows, MacOS and Ubuntu. 
- +  * Some kind of GPUdedicated or integrated, is required to visualize the running OS
-The machine on which you will be running the system needs a graphics cardit can be a built in laptop GPU as well. +  * At least 8GB of excess RAM for the VM is recommended4GB minimum. The more the better
- +
-We recommend to run the VM on a machine with 8 GB of RAM or more+
-(The VM itself needs 4 GB of RAM, so the host machine should have at least 5 GB but 8 GB or more would be optimal.)+
  
  
Line 42: Line 52:
 == Windows == == Windows ==
  
-Download [[https://download.virtualbox.org/virtualbox/5.2.32/VirtualBox-5.2.32-132073-Win.exe | version 5.2.32 for Windows Systems]], execute the .exe file and follow the installation instructions.+Download [[https://download.virtualbox.org/virtualbox/5.2.42/VirtualBox-5.2.42-137960-Win.exe | version 5.2.42 for Windows Systems]], execute the .exe file and follow the installation instructions.
  
 The CPU needs to support virtualization to run a VM on Windows. The virtualization can be enabled in the BIOS. To get into the BIOS settings restart your machine, hit the settings button while booting (try F2, F8 and F10) and go into the CPU settings. There is something like 'Intel Virtual Technology' or 'CPU Virtualization' which needs to be enabled. The CPU needs to support virtualization to run a VM on Windows. The virtualization can be enabled in the BIOS. To get into the BIOS settings restart your machine, hit the settings button while booting (try F2, F8 and F10) and go into the CPU settings. There is something like 'Intel Virtual Technology' or 'CPU Virtualization' which needs to be enabled.
Line 48: Line 58:
 == MacOS == == MacOS ==
  
-Download [[https://download.virtualbox.org/virtualbox/5.2.32/VirtualBox-5.2.32-132073-OSX.dmg | the .dmg installer for OS X systems]], execute the file and follow the instructions.+Download [[https://download.virtualbox.org/virtualbox/5.2.42/VirtualBox-5.2.42-137960-OSX.dmg | the .dmg installer for OS X systems]], execute the file and follow the instructions.
 Hint: If the VirtualBox installer fails with The **installation failed** where it encountered an //error that caused the installation to fail.// Go to System Preferences > Security & Privacy. Click the **‘Allow’** button at the bottom and re-run the installer.  Hint: If the VirtualBox installer fails with The **installation failed** where it encountered an //error that caused the installation to fail.// Go to System Preferences > Security & Privacy. Click the **‘Allow’** button at the bottom and re-run the installer. 
  
Line 54: Line 64:
 ==== VirtualBox Setup ==== ==== VirtualBox Setup ====
  
-First of all, download the Virtual Disk Image [[https://seafile.zfn.uni-bremen.de/d/60fda63a3a4047aaa59b/|cram_vm.vdi here]]. It contains an Ubuntu 16.04 with ROS kinetic and CRAM v7.0 pre-installed.+First of all, download the Virtual Disk Image [[https://seafile.zfn.uni-bremen.de/f/a28c1de4acab4609b4f3/?dl=1|cram_vm.vdi here]]. It contains an Ubuntu 18.04 with ROS melodic pre-installed, as well as CRAM v0.7, the latest dev-version CRAM for Unreal, Urobosim for the Unreal interface, KnowRob as the knowledge hub, and Giskard, the motion planner
  
 Then, launch VirtualBox, which you installed in the previous step. Then, launch VirtualBox, which you installed in the previous step.
Line 68: Line 78:
   * the OS as 'Linux'   * the OS as 'Linux'
   * the version as 'Ubuntu (64-bit)'   * the version as 'Ubuntu (64-bit)'
-Choose the VM's memory size (RAM), depending on your machine's capacity, but less than 4GB could cause performance issues. +Choose the VM's memory size (RAM), depending on your machine's capacity, but less than 8GB will cause performance issues. 
-We suggest 4096 MB (or you can also give it 8192 MB as shown in the screenshot):+We recommend to offer as much as possible, when working with unreal. CRAM on it's own can get by with less.
  
 {{ :tutorials:demo:vm3setup.png?800 |}} {{ :tutorials:demo:vm3setup.png?800 |}}
Line 77: Line 87:
 === Settings (optional) === === Settings (optional) ===
  
-For better performance the VM could use some of the CPU's power. Choose the newly created VM and hit 'Settings'. Go to 'Systemand 'Processorand choose how many processor cores you want to give the VM. +If the VM needs to communicate with the host or other machines in the network, especially when running the UnrealEngine on the host machine, VirtualBox needs to allow Bridged Networking for the VM. Choose the newly created VM and hit 'Settings', then go to 'Network'. With checked 'Enable Network Adapter' choose 'Bridged Adapter' and the preferred network interface. In this case, 'eno1is the host's ethernet interface.
-{{ :tutorials:demo:vm4proc.png?800 |}} +
- +
-Everything is set up. The VM can now be started. The Ubuntu 16.04 system should start and automatically log into the only accountwith the username and password 'cram'. +
  
 +{{ :tutorials:demo:virtualbox_bridged_network.png?800 |}}
  
 +For better performance the VM could use some of the CPU's power. Go to 'System' and 'Processor' and choose how many processor cores you want to give the VM.
 +{{ :tutorials:demo:vm4proc.png?800 |}}
  
 +Everything is set up. The VM can now be started. The Ubuntu 18.04 system should start and automatically log into the only account, with the username and password 'cram'.
  
 +=== Last adjustments ===
  
 +There are two versions of CRAM available in the VM. By default, the setup is designed to connect and work with a project from the Unreal Engine, for intricate physics simulation. We don't need any of that for the upcoming tutorials, instead we change to the stable, standalone version of CRAM. In the VM, launch a terminal with Ctrl-Alt-T and open up the file ''.bashrc''.
  
 +<code>
 +nano .bashrc
 +</code>
  
 +Put a ''#'' symbol in front of the ''source'' command for day 1 & 2 of the FallSchool, and remove it from the line below, referring to day 3 of the FallSchool. Save and close the file and type ''source .bashrc'' to activate the changes in your current terminal. From now on the changes are applied on every newly opened terminal.
  
 +By doing so, the terminal uses a different ROS workspace. Sourcing a ROS workspace enables the terminal session to access everything from that workspace, like loading a programming environment, or changing a namespace.
  
 ===== Understanding the Basics ===== ===== Understanding the Basics =====
Line 105: Line 122:
 Alternatively, you can use a keyboard shortcut for starting a terminal: "''Ctrl + Alt + T''" Alternatively, you can use a keyboard shortcut for starting a terminal: "''Ctrl + Alt + T''"
  
-==== Emacs ====+ 
 +==== ROS ==== 
 + 
 +ROS stands for Robot Operating System. 
 +It is software that makes a robot programmer's life easier. 
 +For example, it allows programs written in different programming languages to talk to each other. 
 + 
 + 
 +==== Lisp ==== 
 + 
 +We are going to program our robot in the Lisp programming language. 
 + 
 +This language has a convenient command line. 
 +If you are curious why Lisp, [[http://cram-system.org/doc/package/why_lisp|go here]]. 
 + 
 + 
 + 
 +===== Tutorial 0: Getting into Emacs =====
  
 Emacs is one of the oldest text editors and one of the two most popular text editors in the Linux world. Emacs is one of the oldest text editors and one of the two most popular text editors in the Linux world.
Line 127: Line 161:
 To see useful key shortcuts for your Emacs IDE, [[http://cram-system.org/doc/ide?&#key_bindings|follow this link]]. To see useful key shortcuts for your Emacs IDE, [[http://cram-system.org/doc/ide?&#key_bindings|follow this link]].
  
 +{{ :tutorials:demo:emacs_intro.pdf |These slides}} contain a few important shortcuts, and introduce you to the popular 'Orc Battle' Lisp example from 'The Land of Lisp'.
  
-==== ROS ====+Run and play the orc-battle.lisp program to get a feeling for Emacs.
  
-ROS stands for Robot Operating System. +===== Tutorial 1: Simple Fetch and Place =====
-It is software that makes a robot programmer's life easier. +
-For example, it allows programs written in different programming languages to talk to each other.+
  
 +Head to the [[http://cram-system.org/tutorials/demo/fetch_and_place_2|exercise to create a simple fetch and place plan]].
  
-==== Lisp ====+===== Tutorial 2 & 3: Failure Handling =====
  
-We are going to program our robot in the Lisp programming language.+When you got the fetch and place plan go to the [[http://cram-system.org/tutorials/demo/fetch_and_place_3|exercise for failure handling]].
  
-This language has a convenient command line. +===== Tutorial 4: Alternative Grasping Poses (Advanced) ===== 
-If you are curious why Lisp, [[http://cram-system.org/doc/package/why_lisp|go here]].+ 
 +Picking objects from different angles can be helpful in occluded environments. Go to the [[http://cram-system.org/tutorials/demo/fetch_and_place_4|exercises on grasping poses]] to find out more. 
 + 
 + 
 + 
 +<html><!--
  
  
Line 217: Line 256:
 Pressing and dragging the right mouse button translates the camera. Pressing and dragging the right mouse button translates the camera.
 Pressing and dragging the middle click doesn't do anything, but you can zoom in and zoom out by scrolling. Pressing and dragging the middle click doesn't do anything, but you can zoom in and zoom out by scrolling.
 +
 +If the bullet simulation is stuck call the following function in the REPL to reset the window:
 +<code lisp>
 +PP-TUT> (btr-utils:reset-debug-window)
 +</code>
 ==== Moving Around ==== ==== Moving Around ====
  
Line 439: Line 483:
  
 (defparameter *base-pose-near-counter* (defparameter *base-pose-near-counter*
-  (make-pose "base_footprint" '((-0.150d0 2.0d0 0.0d0) (0.0d0 0.0d0 -1.0d0 0.0d0))))+  (make-pose "map" '((-0.150d0 2.0d0 0.0d0) (0.0d0 0.0d0 -1.0d0 0.0d0))))
 </code> </code>
  
Line 493: Line 537:
    
 (defparameter *base-pose-near-counter* (defparameter *base-pose-near-counter*
-  (make-pose "base_footprint" '((-0.15 2 0) (0 0 -1 0))))+  (make-pose "map" '((-0.15 2 0) (0 0 -1 0))))
    
 (defparameter *final-object-destination* (defparameter *final-object-destination*
Line 800: Line 844:
 <code lisp> <code lisp>
 (defun pick-up-object (?perceived-object ?grasping-arm) (defun pick-up-object (?perceived-object ?grasping-arm)
-  (let* ((?possible-grasps '(:left-side :right-side :front :back)) +  (let* ((?possible-grasps '(:left-side :right-side :front :back))  ;define all possible grasps 
-         (?grasp (first ?possible-grasps))) +         (?remaining-grasps (copy-list ?possible-grasps))           ;make a copy to work though when trying each grasp 
-    (setf ?possible-grasps (rest ?possible-grasps)) +         (?grasp (first ?remaining-grasps)))                        ;this is the first one to try 
-  +     
-    (cpl:with-retry-counters ((arm-change-retry 1)) +      (cpl:with-retry-counters ((arm-change-retry 1))               ;there is one alternative arm if the first one fails 
-      ;; Outer handle failure handling arm change +       
-      (handle-failure object-unreachable +         ;; Outer handle-failure handling arm change 
-          ;; Iner handle-failure handling grasp change +         (handle-failure object-unreachable 
-          ((handle-failure (or manipulation-pose-unreachable gripper-closed-completely)+            ((setf ?remaining-grasps (copy-list ?possible-grasps))  ;make sure to try all possible grasps for each arm 
 +             (setf ?grasp (first ?remaining-grasps))                ;get the first grasp to try 
 +             (setf ?remaining-grasps (rest ?remaining-grasps))     ;update the remaining grasps to try 
 +              
 +            ;; Inner handle-failure handling grasp change 
 +            (handle-failure (or manipulation-pose-unreachable gripper-closed-completely)
                ;; Try to perform the pick up                ;; Try to perform the pick up
                ((perform (an action                ((perform (an action
Line 815: Line 864:
                              (grasp ?grasp)                              (grasp ?grasp)
                              (object ?perceived-object))))                              (object ?perceived-object))))
-             ;; When pick-up fails this block gets executed +  
-             (format t "Grasp failed! Error: ~a~%Grasp: ~a~%Arm: ~a~%" +               ;; When pick-up fails this block gets executed  
-                     ?grasp ?grasping-arm+               (format t "~%Grasping failed with ~a arm and ~a grasp~%" ?grasping-arm ?grasp) 
-             ;; Checks if we have any possible grasps left. +               ;;(format t "~%Error: ~a~%" e)                       ;uncomment to see the error message 
-             ;; If yes, then the block nested to it gets executed, which will +               
-             ;; set the grasp that is used to the new value and trigger retry +               ;; Check if we have any remaining grasps left. 
-             (when (first ?possible-grasps) +               ;; If yes, then the block nested to it gets executed, which will 
-               (format t "Retyring! Trying to grasp from ~using ~a arm" +               ;; set the grasp that is used to the new value and trigger retry 
-                       ?grasp ?grasping-arm) + 
-               (setf ?grasp (first ?possible-grasps)) +               (when (first ?remaining-grasps)                      ;if there is a grasp remaining 
-               (setf ?possible-grasps (rest ?possible-grasps)) +                  (setf ?grasp (first ?remaining-grasps))           ;get it 
-               (park-arms) +                  (setf ?remaining-grasps (rest ?remaining-grasps)) ;update the remaining grasps to try 
-               (cpl:retry)) +           (format t "Retrying with ~a arm and ~a grasp~%" ?grasping-arm ?grasp) 
-             ;; This will get executed when there are no more elements in the  +                  (park-arms) 
-             ;; ?possible-grasps list. We print the error message and throw a new error +                  (cpl:retry)) 
-             ;; which will be caught by the outer handle-failure +                ;; This will get executed when there are no more elements in the  
-             (print  "No more grasp retries left :(") +                ;; ?possible-grasps list. We print the error message and throw a new error 
-             (cpl:fail 'object-unreachable))+                ;; which will be caught by the outer handle-failure 
-  +                (print  "No more grasp retries left :(") 
-        ;; This is the failure management of the outer handle-failure call +                (cpl:fail 'object-unreachable)))
-        ;; It changes the arm that is used to grasp +
-        (print "Manipulation failed!!"+
-        (print e)+
    
-        ;; Here we use the retry counter we defined. The value is decremented automatically +             ;; This is the failure management of the outer handle-failure call 
-        (cpl:do-retry arm-change-retry +             ;; It changes the arm that is used to grasp 
-          ;; if the current grasping arm is right set left, else set right +             (format t "Manipulation failed with the ~a arm"?grasping-arm) 
-          (setf ?grasping-arm (if (eq ?grasping-arm :right)  +             ;; (print e)                                           ;uncomment if you want to see the error 
-                                  :left +             ;; Here we use the retry counter we defined. The value is decremented automatically 
-                                  :right)) +             (cpl:do-retry arm-change-retry 
-          (cpl:retry)) +                ;; if the current grasping arm is right set left, else set right 
-        ;; When all retries are exhausted print the error message. +                (setf ?grasping-arm (if (eq ?grasping-arm :right)  
-        (print "No more arm change retries left :(")))) +                                        :left 
-  ?grasping-arm)+                                        :right)) 
 +                (park-arms) 
 +                (cpl:retry)) 
 +             ;; When all retries are exhausted print the error message. 
 +             (print "No more arm change retries left :(")))) 
 +  ?grasping-arm) ; function value is the arm that was used to grasp the object
 </code> </code>
 With this, the ''pick-up-object'' can now iterate through all possible grasp configurations stored in ''?possible-grasps''. With this, the ''pick-up-object'' can now iterate through all possible grasp configurations stored in ''?possible-grasps''.
Line 900: Line 951:
 Save the file, compile and switch back to the REPL. Save the file, compile and switch back to the REPL.
  
-You should see a result that looks like the one below. //[Some messages that would come up are suppressed here for readability.]//+You should see a result that looks like the one below.  
 +[Some messages have been suppressed here for readability.]
 <code lisp>  <code lisp> 
 PP-TUT> (move-bottle '((-1.0 -0.75 0.860) (0 0 0 1))) PP-TUT> (move-bottle '((-1.0 -0.75 0.860) (0 0 0 1)))
-[(PICK-PLACE PICK-UP) INFO] 1550504321.279: Opening gripper +… 
-[(PICK-PLACE PICK-UP) INFO] 1550504321.279: Reaching +[(PICK-PLACE PICK-UP) INFO] 1621250741.518: Opening gripper 
-[(GRASP-FAILURE) WARN] Failed to grasp from LEFT-SIDE using RIGHT arm  +[(PICK-PLACE PICK-UP) INFO] 1621250741.522: Reaching 
-[(TRYING-NEW-GRASP) INFO] 1550504800.749: Trying to grasp from RIGHT-SIDE using RIGHT arm +… 
-[(PICK-PLACE PICK-UP) INFO] 1550504800.789: Opening gripper +Grasping failed with RIGHT arm and LEFT-SIDE grasp 
-[(PICK-PLACE PICK-UP) INFO] 1550504800.789: Reaching +Retrying with RIGHT arm and RIGHT-SIDE grasp 
-[(GRASP-FAILURE) WARN] Failed to grasp from RIGHT-SIDE using RIGHT arm  +[(PICK-PLACE PICK-UP) INFO] 1621250742.709: Opening gripper 
-[(TRYING-NEW-GRASP) INFO] 1550504801.577Trying to grasp from BACK using RIGHT arm +[(PICK-PLACE PICK-UP) INFO] 1621250742.709: Reaching 
-[(PICK-PLACE PICK-UP) INFO] 1550504801.601: Opening gripper +… 
-[(PICK-PLACE PICK-UP) INFO] 1550504801.602: Reaching +Grasping failed with RIGHT arm and RIGHT-SIDE grasp 
-[(PICK-PLACE PICK-UP) INFO] 1550504801.939: Gripping +Retrying with RIGHT arm and FRONT grasp 
-[(PICK-PLACE PICK-UP) INFO] 1550504801.973: Assert grasp into knowledge base +[(PICK-PLACE PICK-UP) INFO] 1621250744.157Opening gripper 
-[(PICK-PLACE PICK-UP) INFO] 1550504801.974: Lifting +[(PICK-PLACE PICK-UP) INFO] 1621250744.159: Reaching 
-[(PICK-PLACE PLACE) INFO] 1550504802.356: Reaching +… 
-[(PICK-PLACE PLACE) INFO] 1550504802.508: Putting +Grasping failed with RIGHT arm and FRONT grasp 
-[(PICK-PLACE PLACE) INFO] 1550504802.619: Opening gripper +Retrying with RIGHT arm and BACK grasp 
-[(PICK-PLACE PLACE) INFO] 1550504802.655: Retract grasp in knowledge base +[(PICK-PLACE PICK-UP) INFO] 1621250745.425: Opening gripper 
-[(PICK-PLACE PLACE) INFO] 1550504802.660: Retracting+[(PICK-PLACE PICK-UP) INFO] 1621250745.430: Reaching 
 +[(PICK-PLACE PICK-UP) INFO] 1621250746.218: Gripping 
 +[(PICK-PLACE PICK-UP) INFO] 1621250746.273: Assert grasp into knowledge base 
 +[(PICK-PLACE PICK-UP) INFO] 1621250746.277: Lifting 
 +[(PICK-PLACE PLACE) INFO] 1621250746.751: Reaching 
 +[(PICK-PLACE PLACE) INFO] 1621250746.961: Putting 
 +[(PICK-PLACE PLACE) INFO] 1621250747.064: Opening gripper 
 +[(PICK-PLACE PLACE) INFO] 1621250747.142: Retract grasp in knowledge base 
 +[(PICK-PLACE PLACE) INFO] 1621250747.185: Retracting
 </code> </code>
 {{:tutorials:intermediate:btw-tut-grasp-again.png?800|}} {{:tutorials:intermediate:btw-tut-grasp-again.png?800|}}
 +
 +The robot has once again succeeded in grasping the object. 
 +
 +Now let's move the bottle even further away from the robot so that it is out of reach of the right arm.  This time, after trying all of the grasp poses with the right arm, it should switch to the left arm.
 +<code lisp>
 +PP-TUT> (spawn-object '((-0.9 -0.75 0.860) (0 0 0 1)))
 +</code>
 +You should see a result that looks like the one below, succeeding in grasping the bottle with the first grasp (LEFT-SIDE) using the LEFT arm. 
 +[Again, some messages have been suppressed here for readability.]
 +<code lisp> 
 +PP-TUT> (move-bottle '((-0.9 -0.75 0.860) (0 0 0 1)))
 +
 +[(PICK-PLACE PICK-UP) INFO] 1621251445.989: Opening gripper
 +[(PICK-PLACE PICK-UP) INFO] 1621251445.993: Reaching
 +
 +Grasping failed with RIGHT arm and LEFT-SIDE grasp
 +Retrying with RIGHT arm and RIGHT-SIDE grasp
 +[(PICK-PLACE PICK-UP) INFO] 1621251447.290: Opening gripper
 +[(PICK-PLACE PICK-UP) INFO] 1621251447.291: Reaching
 +
 +Grasping failed with RIGHT arm and RIGHT-SIDE grasp
 +Retrying with RIGHT arm and FRONT grasp
 +[(PICK-PLACE PICK-UP) INFO] 1621251448.443: Opening gripper
 +[(PICK-PLACE PICK-UP) INFO] 1621251448.454: Reaching
 +
 +Grasping failed with RIGHT arm and FRONT grasp
 +Retrying with RIGHT arm and BACK grasp
 +[(PICK-PLACE PICK-UP) INFO] 1621251449.628: Opening gripper
 +[(PICK-PLACE PICK-UP) INFO] 1621251449.650: Reaching
 +
 +[(PICK-PLACE PICK-UP) INFO] 1621251450.450: Gripping
 +[(PICK-AND-PLACE GRIP) WARN] 1621251450.480: There was no object to grip
 +Retrying
 +[(PICK-AND-PLACE GRIP) WARN] 1621251450.491: No retries left. Propagating up.
 +#<CRAM-COMMON-FAILURES:GRIPPER-CLOSED-COMPLETELY {100CD890B3}> 
 +Grasping failed with RIGHT arm and BACK grasp
 +"No more grasp retries left :(" 
 +#<CRAM-COMMON-FAILURES:OBJECT-UNREACHABLE {100B7A2583}> Manipulation failed with the RIGHT arm
 +[(PICK-PLACE PICK-UP) INFO] 1621251450.606: Opening gripper
 +[(PICK-PLACE PICK-UP) INFO] 1621251450.608: Reaching
 +[(PICK-PLACE PICK-UP) INFO] 1621251451.151: Gripping
 +[(PICK-PLACE PICK-UP) INFO] 1621251451.234: Assert grasp into knowledge base
 +[(PICK-PLACE PICK-UP) INFO] 1621251451.235: Lifting
 +[(PICK-PLACE PLACE) INFO] 1621251451.709: Reaching
 +[(PICK-PLACE PLACE) INFO] 1621251451.892: Putting
 +[(PICK-PLACE PLACE) INFO] 1621251451.988: Opening gripper
 +[(PICK-PLACE PLACE) INFO] 1621251452.027: Retract grasp in knowledge base
 +[(PICK-PLACE PLACE) INFO] 1621251452.074: Retracting
 +</code>
 +{{:tutorials:demo:btw-tut-grasp-again-right.png?800|}}
  
 The robot has once again succeeded in grasping the object.  The robot has once again succeeded in grasping the object. 
Line 1087: Line 1197:
 ===== Defining a New Grasp ===== ===== Defining a New Grasp =====
  
-We discussed predefined possible grasps for a bottle in the previous section, which were 4 in number. Even though there are a lot of different possible grasps in real life, using only 4 limited our possible ways of grasping the bottle. So can we add more? Of course! Let's see that in this section. Let's try to grasp the bottle diagonally between positive x and y axes. We'll call this the front-left-diagonal grasp.+We discussed predefined possible grasps for a bottle in the previous section, which were 4 in number. Even though there are a lot of different possible grasps in real life, using only 4 limits our possible ways of grasping the bottle. So can we add more? Of course! Let's see that in this section. Let's try to grasp the bottle diagonally between positive x and y axes. We'll call this the front-left-diagonal grasp.
  
-There are two things to be carried out during any grasp pose: First is the translation the gripper will have to make to perform the grasp and the other one is the orientation the gripper will have to make to match the grasp that is to be performed +The grasps are defined as ''obj_T_grp'', that is, as the coordinate frame of the gripper with respect to the object. That is, all the grasps are defined in the object coordinate frame. It is important to note that the more uniformly the coordinate frames of the manipulated objects are defined in a robotic system, the easier it will be to define generic grasps for them. In our system, we define the origin of the object coordinate frame as the center of the bounding box of the object. For the orientation, we typically choose Z to point upwards, whereby the object is oriented as it would be typically standing on a supporting surfaceThe X is usually either the longer principle axis of the object, or the axis from the handle towards the head of the object. Defining X and Y is more ambiguous, and we will not discuss this issue deeper in this tutorial.
-You car refer the image below again to see what axes correspond to what face of the object. +
-{{ :tutorials:demo:object_orientations.png?600 |}}+
  
-For a front-left-diagonal graspthe gripper has to come positive x and y-axis side and then close in and finally do a lift to complete the object pick-upSo we'll define some parameters that will help us do the translation part of the grasping. The values are taken from the predefined grasp offset values in  <your workspace>/cram/cram_knowrob/cram_knowrob_pick_place/src/grasping.lisp+The gripper frame in ''obj_T_grp'' is typically the tool frame of the gripper. 
 + 
 +There are two things that need to be calculated for any grasp pose: the translation the gripper will have to make to perform the grasp and the orientation the gripper will have to make to match the grasp that is to be performed. 
 + 
 +To calculate the translation part, we'll define some offsets as parameters. The values are taken from the predefined grasp offset values in  ''<your workspace>/cram/cram_knowrob/cram_knowrob_pick_place/src/grasping.lisp''
  
 <code lisp> <code lisp>
Line 1104: Line 1216:
 </code> </code>
  
-As for the orientation, first, we need to know what the axes of the gripper are like to match it with the direction we want to approach it from. For standard gripper, Z-axis points towards the object,  X is along the palm, Y is defined by the right-hand ruleThus, to make the front-left diagonal grasp, the z-axis of the gripper should point 45 degrees in between the x and y-axes of the object. And since we are grasping across the z-axis of the bottle, since the bottle is standing along the z-axis, we'll keep our palm, which is our x-axis, perpendicular to the z-axis of the bottle.+For the orientation, we need to know the coordinate frame of the object and the gripper in order to define grasp. 
 +front-left-diagonal grasp means that the gripper has to come from the positive x and y-axis side of the object. 
 +You car refer to the image below again to see what axes correspond to what face of the object. 
 +{{ :tutorials:demo:object_orientations.png?600 |}}
  
-To visualize this easilyopen your left palm with the thumb perpendicular to the other 4 fingersnow imagine the y-axis going along your thumb, the x-axis along your other 4 fingers and the z-axis coming from inside your palm perpendicular to the surface. Now to grasp a bottle, you'll face the palm to the bottle which is the z-axis and your 4 fingers will be horizontally perpendicular to the height (z-axisof the bottle. +The coordinate frame of a standard two-finger parallel gripper is defined as following: Z-axis points towards the objectX is defined along the translation of the fingers (the axis that connects the left and the right finger), and Y is defined according to the right-hand rule. For examplebelow you can see the frames of our Boxy robot's grippers, which are popular industrial parallel grippers (manufactured by Schunk or Weiss, we use both interchangeably): 
 + 
 +{{:tutorials:demo:boxy_left_gripper.png?400|}} {{:tutorials:demo:boxy_right_gripper.png?400|}} 
 + 
 +Thus, to make the front-left diagonal grasp, the Z-axis of the gripper should point 45 degrees in between the X and Y-axes of the object. And since we are grasping across the Z-axis of the bottle (the bottle is standing along the z-axis), we'll keep the X axis perpendicular to the Z-axis of the bottle and the Y of the gripper aligned with the Z of the bottle.
  
 <code lisp> <code lisp>
Line 1117: Line 1236:
           (0 1 0)))           (0 1 0)))
 </code> </code>
-The rotation matrix given above is the rotation matrix required to bring the gripper from the identity pose to the pose that we need to make a left-diagonal grasp. This is achieved by rotating the identity pose along the x-axis by +90 degrees first, and then rotating the new pose-along the y-axis by -45 degrees. Try to visualize these rotations mentally. +The rotation matrix given above is the rotation matrix required to bring the gripper from the identity pose to the pose that we need to make a left-diagonal grasp. This is achieved by rotating the identity pose around the x-axis by +90 degrees first, and then rotating the new pose around the new Y-axis by -45 degrees. Try to visualize these rotations mentally. 
 Note:-Since sin(pi/4) = cos(pi/4), we have only defined one variable and used it interchangeably in the defined rotation matrix Note:-Since sin(pi/4) = cos(pi/4), we have only defined one variable and used it interchangeably in the defined rotation matrix
  
-Now let's define the method that tie up all this and let ROS know that a new grasp is available+Now let's define the method that tie up all this and let CRAM know that a new grasp is available
 <code lisp> <code lisp>
 (cram-object-interfaces:def-object-type-to-gripper-transforms '(:drink :bottle) '(:left :right) :fl-diagonal (cram-object-interfaces:def-object-type-to-gripper-transforms '(:drink :bottle) '(:left :right) :fl-diagonal
   :grasp-translation `(,(- *bottle-grasp-xy-offset*) ,(- *bottle-grasp-xy-offset*) ,*bottle-grasp-z-offset*)   :grasp-translation `(,(- *bottle-grasp-xy-offset*) ,(- *bottle-grasp-xy-offset*) ,*bottle-grasp-z-offset*)
   :grasp-rot-matrix *diagonal-rotation*   :grasp-rot-matrix *diagonal-rotation*
-  :pregrasp-offsets `(,*bottle-grasp-xy-offset* ,*bottle-pregrasp-xy-offset* ,*lift-z-offset*) +  :pregrasp-offsets `(,*bottle-pregrasp-xy-offset* ,*bottle-pregrasp-xy-offset* ,*lift-z-offset*) 
-  :lift-offsets *lift-offset*)+  :2nd-pregrasp-offsets `(,*bottle-pregrasp-xy-offset* ,*bottle-pregrasp-xy-offset* 0.0) 
 +  :lift-offsets *lift-offset* 
 +  :2nd-lift-offsets *lift-offset*)
 </code> </code>
-The summary of this code is pretty simple, we have defined a grasp called ''fl-diagonal'' for objects ''drink'' and ''bottle'', which can be accessed with the ''left'' or ''right'' arm of the robot. The pre-grasp offset gives you the distance the gripper will be positioned before the grasp. The grasp-traslation gives the translation that the gripper will perform during the grasping. You can see that they're coming from the positive x and y axis and closing that distance during translation. The grasp-rot-matrix gives the orientation required for gripping. +The summary of this code is pretty simple, we have defined a grasp called ''fl-diagonal'' for objects ''drink'' and ''bottle'', which can be accessed with the ''left'' or ''right'' arm of the robot. The pre-grasp offset gives you the distance the gripper will be positioned before the grasp. The grasp-translation gives the translation that the gripper will perform during the grasping. You can see that they're coming from the positive x and y axis and closing that distance during translation. The grasp-rot-matrix gives the orientation required for gripping. 
  
 Now that we have defined our new grasp let's see it in action. First, spawn the bottle again and position the robot ready to pick up. Now that we have defined our new grasp let's see it in action. First, spawn the bottle again and position the robot ready to pick up.
Line 1171: Line 1292:
                            (object ?perceived-bottle)))))                            (object ?perceived-bottle)))))
 </code> </code>
-You will see that PR2 will successfully grasp the bottle from the grasp pose that we defined.+You will see that PR2 will successfully grasp the bottle from the grasp pose that we defined. Another useful documentation on this point is [[http://cram-system.org/doc/package/cram_designators]]. Here you can understand further what the 'desig:an' does and why we use it. Designators are a very useful thing in cram, furthermore they are used in every high-level plan.  
 + 
 +{{ :tutorials:demo:diagonal_grasp.png?600 |}} 
 + 
 +To visualize the gripper coordinates, you can always use the following call of the ''visualize-coordinates'' function: 
 + 
 +<code lisp> 
 +(visualize-coordinates (btr:link-pose (btr:get-robot-object) "r_gripper_tool_frame")) 
 +</code> 
 +where ''"r_gripper_tool_frame"'' is the name of PR2's right gripper tool frame. The left gripper's tool frame is called, correspondingly, ''"l_gripper_tool_frame"''
 + 
 +There is one caveat: PR2's gripper coordinate frame is not defined according to the common industry standard that we just described. But in order to keep our grasps applicable to any gripper, we define the grasps for the standard gripper and each robot that transforms the standard grasp into its own gripper frame. Therefore, even if we're defining the grasps for the PR2, whose gripper frames are defined as following: 
 + 
 +{{ :tutorials:demo:pr2_grippers.png?600 |}} 
 + 
 +we still define them for a standard gripper coordinate frame. PR2 transforms our standard grasp into its own grasp under the hood through the CRAM hardware abstraction layer.
  
-{{:tutorials:demo:diagonal_grasp.png?600|}} 
  
  
Line 1198: Line 1333:
 ===== Credits ===== ===== Credits =====
  
-This tutorial was created with the combined efforts of Amar Fayaz, Arthur Niedzwiecki and Gayane Kazhoyan. +This tutorial was created with the combined efforts of Amar Fayaz, Arthur Niedzwieckiand Gayane Kazhoyan.
  
  
  
 +--></html>
  
  
Line 1212: Line 1347:
  
 Arthur got the following feedback from students: 3h isn't enough to complete the tasks, the intro of the tutorial was too long, designators should be explained. Maybe put a link to those topics at appropriate position. Arthur got the following feedback from students: 3h isn't enough to complete the tasks, the intro of the tutorial was too long, designators should be explained. Maybe put a link to those topics at appropriate position.
 +
 +Alina got the following feedback from students:
 +- an introduction to how to navigate and handle Emacs would have been nice at the start
 +- VM is too slow for some. Laptops's we can lend with everything pre-installed would be nice (be it VM or native doesn't matter)
 +- Too little time. Nobody finished Exercise 1
 +- visualization of poses could be done better (we used Rviz and "point clicked", but rotation remains a problem)
 +- Someone also suggested to do the following: Emacs crash course + basic shortcuts introduction, LISP introduction (just show some basic functions since they look very different to other languages. Like, variable definitions, let, if, mapcar (it's cool)) and then go to CRAM. (Maybe emphasize rather on plans/actions which look more like "natural language"?)
 +
 +Some questions the students had and some answers we weren't sure about
 +- What licence is cram uder (bsd?)
 +- Can it be used without ROS? -> no. unless someone writes something for it since it's open source
 +- How does the PR2 know that he dropped something
 +- Would the belief state work with multiple robots at the same time? -> yes?
  
  
 --></html> --></html>