Thank you for your comment and sorry for my late answer.

Tha advantages of the edge-based approach are
- real-time capability
- the possibilty to estimate all needed parameters (x, y, orientation and scale) of the target objects using the shape space of euclidean similarities
- tracking of deformable objects, using key frames

The drawbacks are:
- A low robustness against additive noise (in comparision to the other approaches), which results in a lower accuracy (few pixel) of the pose estimation
- robustness against clutter requires modeling/training of the edge detector

We are using this approach for deformation measurements of long and thin nanoobjects, like carbon nanotubes and nanowires.

Best regards,

Torsten

Thanks for your explicit reply and particularly thanks for the quick involvment at the very beginning of the conference. The paper is very comprehensive and insightful. it'd be interesting for me to see nanoobjects inside a SEM some day.

best wishes

Liu

Dear Prof. T. Sievers,

It is an interesting and excellent research in nano-robotics. The presented paper deals with 2D handling of the nano-objects based on SEM images. Do you have any ideas or hints for 3D control and handling the nano-objects? This is a big concern for the application in areas of medicine and biology.

Many thanks,

Le, Cardiff University, Wales, UK

The theoretical resoultion of an SEM is about 3 nm, thus it is possible to image nanobjects today.
For example, in Fig. 1 a carbon nanotube with a diameter of approximately 150 nm is shown.

Best regards,

Torsten

Thank you for your comment.

You’re a completely right! Getting 3D information is an important key challenge for successful nanohandling. We are investigating two approaches in our lab. The first one is depth-from-focus and the second one is 3D image processing. The latter approach is carried out by tilting the electron beam in order to get two images with different perspectives. This approach has been published by my colleague.

M.Jähnisch, M. Schiffner “Stereoscopic Depth-Detection for Handling and Manipulation Tasks in a Scanning Electron Microscope” IEEE Int. Conf. on Robotics & Automation (ICRA),Orlando, USA, May 2006

Best regards,

Torsten

Dear Emmanuel

Thank you very much for these good questions.

I still test the approaches with different target objects and under changing conditions. But I would say that the regions based approach is the most successful solution for tracking micro- and nanoobjects inside an SEM in consideration of the mentioned requirements (real-time capability, robustness against noise, computable transformation parameters).

At the moment, we are working with a pixel clocks between 125 ns and 250 ns (nanoseconds), which is typical for most available SEMs. A few days ago I talked to the CTO of TESCAN, a Czech SEM manufacturer, about that topic. He told me that the next SEM generation will operate with a pixel clock down to 80 ns. This would be very close to the minimum that is physically possible. The main physical limit is that only a few secondary electrons can be generated by the electron beam during this short time period, even if the beam current is very high. A further reduction of the pixel clock will interrupt the secondary electron process, thus, no images can be obtained.

Best regards,

Torsten

Dear Prof. Torsten,

Many thanks for your reply. I have a look at the paper that you mentioned:

Stereoscopic depth-detection for handling and manipulation tasks in a scanning electron microscope

And I found that the research is realy interesting more and more!

The first impression is that the Stereo technique, one of the early methods for 3D reconstruction and Reverse Engineering are able to be used for a state-of-the art area today: nano-robotics.

The disadvantage of the stereo technique was mentioned in this paper is that: ”this approach is not suitable for the generation of stereo images for the observation of handling processes”.

Generally, in stereo technique, to find correspondence between images is extremely difficult; this can produce erroneous results from mismatches. In addition, although requiring very simple hardware, the accuracy of the stereo technique is poor.

Today, in order to solve the problem of accuracy and finding correspondence, stereo vision techniques can be combined with colour structured light techniques for 3D range data acquisition. These active techniques are most well-known in the areas of Reverse Engineering, in which the projection of light sources (lasers or structured light) onto the object is used for data acquisition.

For my opinion, if we can solve the problem of accuracy and reducing the complexity in hardware in ACTIVE Reverse Engineering techniques, then this may be a good approach for getting and controlling 3D information in nano-handling.

Another technique for getting 3D information is “Interferometry”. The interferometry technique is well known in dimension inspections as well as flatness and deformation measurements, in which structured light patterns are projected onto a surface to produce shadow Moiré effects. The light contours produced by the Moiré effects are captured in an image and analyzed to determine distances between the lines. This distance is proportional to the height of the surface at the point of interest and so the surface coordinates can be calculated.

The accuracy of the Interferometry techniques can reach to nano-level. Therefore, it would be good to consider the Interferometry technique to get 3D information for nano-handling as well.

Best wishes,

Le

Dear Le!

Thank you for your comment. I forwarded your posting to my colleague Mr. Jähnisch. I think the first step to do is to evaluate (accuracy, performance) the stereo approach, because the advantage of the simple hardware setup is significant. Especially the vacuum chamber of an SEM offers limited space and only infrared light can be used simultaneous to the SE detector.

Best regards,

Torsten