Arrigo benedetti microsoft

Indicator segments are identified and a rotation matrix is computed, and the rotation is applied to the wrapped phases. So doing reduces a three-dimensional analysis to a two-dimensional analysis, advantageously reducing computational time and overhead. So doing identifies indicator points and corresponding aliasing intervals. In some embodiments an optimized rotation matrix preferably is computed and applied to the indicator segments before identifying the indicator points and corresponding aliasing intervals.

Preferably during runtime operation of the TOF system input phase data is rotated to find closest indicator points to line segments.

In some embodiments these points can be labeled as valid or invalid based upon their computed distance from an indicator point, with validity confidence being determined by a static or by a dynamic threshold test.

The unwrapping interval associated with each indicator point is used to unwrap the phase measurements, which measurements preferably are averaged, which averaging also reduces noise magnitude in the phase data. The applied geometric analysis results in optimal selection of m 1 , m 2 , and m 3 to unwrap and disambiguate phase for the TOF system.

The ability to dynamically assign confidence labels to acquired phase data can advantageously reduce motion blur error due to target objects that move during data acquisiton using N modulation frequencies.

Other features and advantages of the application will appear from the following description in which the preferred embodiments have been set forth in detail, in conjunction with their accompanying drawings. Co-pending U. Other embodiments of the ' application apply a two-step hierarchical approach while operating the TOF system using three modulation frequencies. Assume there are m modulation frequencies f 1 , f 2 ,. Each of the intermediate frequencies is a function of the modulation frequencies f 1 , f 2 ,.

The final unwrapped phase for f E is:. Preferably uncertainty of the amplifying factor at each step is sufficiently small such that the probability of choosing m k incorrectly is low. The total unambiguous range of the system is determined by f D , which is a function of the three modulation frequencies. As indicated by FIG. Referring now to FIGS. Note that the amplification ratio for the de-aliasing steps are.

Advantageously, this method achieves the desired large ratio. The beneficial result occurs because the de-aliasing intervals m and n are determined separately and the noise is amplified by a much smaller ratio at each method step.

Let the three modulation frequencies be f 1 , f 2 and f 3. Embodiments of the ' application seek to achieve an effective frequency f E that is as close as possible to the TOF system maximum modulation frequency f m. The final effective frequency can be. Referring to FIG. Completing the first step, one can next find the correct dealiasing interval m by minimizing.

Consider next step two, which involves dealiasing. Although it is desired to dealias f E , one cannot get the phase corresponding to. As shown in FIG. This will cause problems in the second step of dealiasing unless additional constraints on the frequencies are imposed, e.

One can then find the correct de-aliasing interval by minimizing:. The unwrapped phase for each frequency f i is computed as:. The above described hierarchical dealiasing methods preferably used at least three close-together modulation frequencies and created a low dealiasing frequency and at least one intermediate frequency to successfully dealias an increased disambiguated distance d max for a TOF system.

Further unlike two-frequency dealiasing, the methods described in the ' application amplified noise by only a small ratio coefficient at each dealiasing step. But while the methods described in the ' application represent substantial improvements in dealiasing or unwrapping phase, it is difficult for the TOF system to know in real-time how best to dynamically make corrections to further reduce acquisition error.

In one sense, the analyses associated with the ' application are simply too complicated to provide a real-time sense of how to further improve quality of the data stream being output by the TOF system sensor array. As will now be described, embodiments of the present application enable substantially lossless phase unwrapping or dealiasing , using multiple modulation frequencies, while providing a mathematical graphical analysis useable to make real-time dynamic adjustments to the TOF system.

Typically module will include at least a graphics processing unit, a CPU, and memory functions. According to the present application, phase unwrapping is concerned with the computation of the indicator function. The relation between wrapped and unwrapped phase can be made explicit by the wrapping operator:.

Phase unwrapping is an ill-posed problem, since equation 3 has infinite solutions. While the above-described unwrapping method is straightforward it unfortunately fails in most practical cases. Failure occurs because equation 5 is not satisfied, typically for several reasons incuding the presence of noise, and the absence of adequate signal bandlimiting.

Methods for overcoming these limitations will now be described. Embodiments of the present application employ a phase unwrapping algorithm that uses N multiple modulation frequencies and functions adequately in most real world applications. In these embodiments, the modulation frequency of active optical source 30 see FIG. The modulation frequency is dynamically changed during sensor array acquisition of an image frame, which is to say the sensor array exposure time to incoming optical energy S in is divided into N parts whose sum equals the exposure time T for the sensor pixel array.

During the second part, the active light modulation frequency is set to f 2 and the second phase image is acquired, and so on. In some embodiments, the N parts may be of equal time duation. For a single modulation frequency, the wrapping distance U is:. It is assumed for each data point d[i] the pixel sensor array will produce two phase value according to.

In a perfect TOF system with no noise, the various plot points would fall on the parallel segment lines. The distance from the plot points to the nearest segment line is a measure of the noise present on the acquired phase data. The singular points falling along the vertical or horizontal axis at about 6. In the plot depicted in FIG. Let the ratio. In doing so, magnitude of the wrapping distance is advantageously extended from. The indicator segment preferably is found by first computing the orthogonal distance between the measured point and each of the lines containing the segments.

The closest such line uniquely determines indicator functions for the two frequencies. Referring, for example, to FIG. In addition to the segments noted by the parallel sloped lines in FIG. A further optimization can be realized by rotating the indicator segments such that they are parallel with the x-axis. As such, the optimal indicator segment can then be found by simply identifying the segment closest to the point on the x-axis. Embodiments of the present application extend the above-described two frequency method to use of three modulation frequencies, f 1 , f 2 , and f 3 , as follows.

Small integers work well with embodiments of the present application, but possibly other approaches would function where m 1 , m 2 , and m 3 were not small integers. There may exist a more general but less optimal solution, in which these modulation frequencies are not required to be prime to each other. The relationship defines the sphere size see FIG. For the case of three modulation frequencies that are co-prime to each other, one can show that the wrapping distance U is:. The axis of this rotation is parallel to the vector n,.

The angle of rotation is defined by:. One can use Rodrigues' formula to write the expression for the rotation matrix R as. How then to compute the three-dimensional coordinates for the indicator segments end-points, and how to computer values of the indicator functions for the case of three modulation frequencies m 1 , m 2 , m 3. The exemplary algorithm may be written as follows:. Once the three-dimensional points with coordinates P X,i , P Y,i , P Z,i are known, they can be projected onto the plane orthogonal to vector v defined in Eq.

Specific derivation of the two-dimensional coordinates of these points is very similar to the above-described calculation. Details are omitted herein as methods of derivation regarding these points are known to those skilled in the relevant art.

Let p i denote projected indicator segments endpoint i with coordinates p Xi ,p Yi , and let R denote the rotation matrix used to rotate the indicator endpoint segments. An exemplary algorithm is as follows:. In the case of multiple values of i in X i. Turning now to FIG. Data points falling within the gray shaded volume within the cube surrounding the spheres are error prone, due to noise. By contrast, FIG. This is a more optimum case and the increased sphere radii represent reduced noise, or higher confidence in the phase data point.

A data rejection mechanism described with respect to FIG. Comparing FIG. But the same point in FIG. In essence the radial distance to such remote points from the center of the nearest sphere is calculated and compared to a diameter threshold. As this juncture, it is useful to describe a more final version of a preferred phase unwrapping algorithm. This algorithm preferably is stored in memory in electronics module and preferably is executable therein by processors associated with module Inputs to this phase unwrapping algorithm are the three wrapped phase values measured at the phase modulation frequencies f 1 , f 2 , f 3 , the indicator points of the projections p X,i , p Y,i , and the values of the indicator functions N 1,i , N 2,i , N 3,i returned by the above-described exemplary three-dimensional coordinate and indicator function algorithm.

The exemplary phase unwrapping algorithm may be represented as follows:. It will be appreciated from the description of FIGS. This error is roughly proportional to the jitter on each modulation frequency.

Jitter is defined herein as the temporal standard deviation, and is a measurement of magnitude of noise on the phase input. Noise sources include shot noise, thermal noise, multipath noise due to partial reflection of optical energy from the source target. An image is captured at each modulation frequency, but the image may have moved between captures. Referring to FIGS. As such radii are reduced, it becomes easier to test to see what points now fall outside the reduced spheres of confidence.

Filed: February 10, Date of Patent: September 10, Fenton, Jayachandra Gullapalli. Publication date: May 24, Publication date: August 10, These frequencies can be changed during N time intervals that together define an exposure time for the TOF pixel array. A first phase image is acquired during perhaps the first third of exposure time using f1, then for the next third of exposure time a second phase image is acquired using f2, and then f3 is used to acquire a third phase image during the last third of exposure time.

Geometric analysis yields desired values for m1, m2, and m3 to unwrap and thus disambiguate phase. Identification of valid and invalid data is identified using dynamically adjustable error tolerances. Filed: August 15, Publication date: February 20, Fast bus image coprocessing.

Abstract: An inspection system having a sensor array that provides image data. A process node includes a memory to receive the image data, a commercially available central processing unit to receive and coprocess at least a first portion of the image data within the memory, and a field programmable gate array to receive and coprocess at least a second portion of the image data within the memory.

In this manner, there are two elements in the process node that are used to simultaneously process the image data, and the image data analysis thereby proceeds at a much faster rate than it would with just a single processor in a commercially available computer.

However, the system as described has very little custom hardware, and thus is relatively inexpensive, and highly versatile.

Filed: October 18, Date of Patent: June 3, Inventors: Lawrence R. Miller, Krishnamurthy Bhaskar, Mark J. Roulo, Arrigo Benedetti. Real-time reconfigurable vision computing system.

Abstract: An image processing system uses an FPGA and an external memory to form neighborhoods for image processing. The FPGA is connected to the external memory in a way that reuses address lines, and increases the effective bandwidth of the operation. Filed: April 1, Date of Patent: June 5, Assignee: California Institute of Technology. Inventors: Arrigo Benedetti, Pietro Perona. Justia Legal Resources.

Find a Lawyer. Law Students. US Federal Law.