Visual servoing of a robotic manipulator based on fuzzy logic

P.Th. Zacharia^a, I.G.Mariolis^b, N.A. Aspragathos^a, E.S. Dermatas^b

^a Department of Mechanical & Aeronautics Engineering, Rion, Patras, Greece

^b Department of Electrical & Computer Engineering, Rion, Patras, Greece

Outline

• The objectives of this work
• Use of fuzzy controller
• Vision feedback
• Experiments - Results
• Conclusions
• Future Work

Objectives of this work

• Enable manipulators to guide a fabric towards a sewing machine and feed it along a desired seam line
• Production of high seam quality

Basic points

• Use of an image-based control system
• no need for the 3D model of the object
• robust with respect to robot calibration errors
• Use of a fuzzy controller for handling a fabric towards the sewing machine

Other approaches

• PI-controller – gains selected by trial and error
• Parallel process decomposition – position controlled system modeled as mass-spring-damper system

Robot sewing process

• Move along r until the fabric’s edge reaches the needle with the desired orientation (defined by the seam line)

• Sewing this fabric’s side along the seam line

• Rotation of the fabric around the needle until next side is ready for sewing

  
  

 The control system

 

Membership functions

Fuzzy rules

The rules are based on the following two propositions:

“The larger the distance from the needle and the smaller its change rate is, the faster the fabric should move towards the needle”

“The larger the angle between the fabric’s side and the seam line and the smaller its change rate is, the faster the fabric should rotate towards the seam line.”

Vision feedback

• Image features' extraction
• The vision algorithm

Image features’ extraction

• The used features  is basically a set of points called ‘corners’ that are actually the vertices of the tracked fabric in counter clock wise order defining its perimeter.
• Due to occlusion however, the features are used to define fragments of the fabric generating hypotheses and testing them in a two stages procedure.

The vision algorithm

• First stage:  a model description (MD) of the fabric is created

• Second stage:  the position of the fabric in each image is extracted, which generates a scene description (SD)

Model Description (MD)

• Image acquisition of the fabric onto the working area
• Harris Corner Detector and post-processing
• Geometric features extracted

Scene Description (SD)

• Image acquisition of the fabric handled by the robotic arm

• Processing

• RGB to grayscale
• Image Histogram,Thresholding results to binary image
• 8-connected components criterion results to binary image containing only the fabric
• Harris Corner Detector to both gray scale and binary image and fusion of the results
• Generation of hypotheses based on geometric rules
• Matching using λ, which is the ratio of the detected fabric’s area to the area of the fabric calculated in the binary image

a. Original RGB image. b. The grayscale counterpart of the original image. c. Histogram of the grayscale image b. d. Binary image after thresholding. e. Corners detected (red crosses) after Harris corner detection. f. The position of the polygon’s corners

Experiments

Fabric

Shape: non-convex polygon

Color: red

Property: large resistance to bending

Video camera

Resolution: 768×576 pixels
Position: 2m above the working area

 

(a) Image of the fabric on the working table (b) Image of the fabric handled by the robot arm

Results from the images

• 100 RGB images containing the fabric handled by the robotic arm
• The actual coordinates of 323 not occluded vertices in those images has been manually determined and their Euclidean Distance from the detected has been calculated
• The mean Euclidean distance of all vertices is 2.0269 pixels while their standard deviation is 0.9876 pixels.

  chart of the results

 

Experimental results

• Several experiments were carried out

• For each one experiment

• the fabric starts from a different location

• the gripper is on a different location on the fabric

• The algorithm has proved to be robust and efficient in all cases

• accepted position error is set equal to 2

• accepted orientation error is set equal to 0.5° 

Conclusions

• Introduction of visual servoed manipulator controller based on fuzzy logic for sewing
• No need for many mathematical equations/calculations
• Effective and efficient method for handling the fabric towards the sewing machine
• Simple and robust technique

Future work

• Extension of the proposed algorithm, so that it can cope with buckling, folding or puckering

• Application to apparel manufacturing, where the up-to-date fabric handling systems are semi-automatic