Home > SIFT The SIFT (Scale Invariant Feature Transform) Detector and Descriptor developed by David Lowe University of British Columbia Initial paper IC

The SIFT (Scale Invariant Feature
Transform) Detector
and Descriptor

developed by David Lowe

University of British Columbia

Initial paper ICCV 1999

Newer journal paper IJCV 2004

2

Review: Matt Brown��s Canonical Frames

3

Multi-Scale Oriented
Patches

- Extract oriented patches at multiple scales

[ Brown, Szeliski, Winder CVPR 2005 ]

4

Application: Image
Stitching

[ Microsoft Digital Image Pro version 10 ]

Ideas from Matt��s
Multi-Scale Oriented Patches

- 1. Detect an interesting patch with an interest operator. Patches are translation invariant.
- 2. Determine its dominant orientation.
- 3. Rotate the patch so that the dominant orientation points upward. This makes the patches rotation invariant.
- 4. Do this at multiple scales, converting them all to one scale through sampling.
- 5. Convert to illumination ��invariant�� form

5

6

Implementation Concern:

How do you rotate a patch?

- Start with an ��empty�� patch whose dominant direction is ��up��.
- For each pixel in your patch, compute the position in the detected image patch. It will be in floating point and will fall between the image pixels.
- Interpolate the values of the 4 closest pixels in the image, to get a value for the pixel in your patch.

7

Rotating a Patch

empty canonical
patch

patch detected in the image

x�� = x cos�� – y sin��

y�� = x sin�� + y cos��

T

T

counterclockwise rotation

**(x,y)**

**(x��,y��)**

8

Using Bilinear Interpolation

- Use all 4 adjacent samples

x

y

**I**_{00}

**I**_{10}

**I**_{01}

**I**_{11}

9

SIFT: Motivation

- The Harris operator is not
invariant to scale and correlation is not invariant to rotation
^{1.}

- For better image matching,
Lowe��s goal was to develop an interest operator that is invariant
to scale and rotation.

- Also, Lowe aimed to create
a descriptor that was robust to the variations corresponding
to typical viewing conditions. The
descriptor is the most-used part of SIFT.

^{1}But Schmid and Mohr developed
a rotation invariant descriptor for it in 1997.

10

Idea of SIFT

- Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging parameters

**SIFT Features**

11

Claimed Advantages
of SIFT

**Locality:**features are local, so robust to occlusion and clutter (no prior segmentation)**Distinctiveness:**individual features can be matched to a large database of objects**Quantity:**many features can be generated for even small objects**Efficiency:**close to real-time performance**Extensibility:**can easily be extended to wide range of differing feature types, with each adding robustness

12

Overall Procedure
at a High Level

- Scale-space extrema detection

- Keypoint localization

- Orientation assignment

- Keypoint description

Search over multiple scales and image
locations.

Fit a model to detrmine location and scale.

Select keypoints based on a measure of
stability.

Compute best orientation(s) for each
keypoint region.

Use local image gradients at selected scale and rotation

to describe each keypoint region.

13

1. Scale-space extrema
detection

- Goal: Identify locations and scales that can be repeatably assigned under different views of the same scene or object.
- Method: search for stable features across multiple scales using a continuous function of scale.
- Prior work has shown that under a variety of assumptions, the best function is a Gaussian function.
- The scale
space of an image is a function
*L(x,y,****)*that is produced from the convolution of a Gaussian kernel (at different scales) with the input image.

14

Aside: Image Pyramids

Bottom level is the original image.

2^{nd} level is derived from
the

original image according to

some function

3^{rd} level is derived from
the

2^{nd} level according to the
same

funtion

And so on.

15

Aside: Mean Pyramid

Bottom level is the original image.

At 2^{nd} level, each pixel is
the mean

of 4 pixels in the original image.

At 3^{rd} level, each pixel is
the mean

of 4 pixels in the 2^{nd} level.

And so on.

mean

16

Aside: Gaussian Pyramid

At each level, image is smoothed and reduced in size.

Bottom level is the original image.

At 2^{nd} level, each pixel is
the result

of applying a Gaussian mask to

the first level and then subsampling

to reduce the size.

And so on.

Apply Gaussian filter

17

Example: Subsampling
with Gaussian pre-filtering

G 1/4

G 1/8

Gaussian 1/2

18

Lowe��s Scale-space Interest Points

**Laplacian of Gaussian**kernel- Scale normalised (x by scale
^{2}) - Proposed by Lindeberg
- Scale-space detection
- Find local maxima across scale/space
- A good ��blob�� detector

[ T. Lindeberg IJCV 1998 ]

19

Lowe��s Scale-space Interest Points:

Difference of Gaussians

- Gaussian is an ad hoc solution of heat diffusion equation

- Hence

- k is not necessarily very
small in practice

20

Lowe��s Pyramid Scheme

- Scale space is separated into octaves:
- Octave 1 uses scale
- Octave 2 uses scale 2
- etc.

- In each octave, the initial
image is repeatedly convolved

with Gaussians to produce a set
of scale space images.

- Adjacent Gaussians are subtracted to produce the DOG

- After each octave, the Gaussian
image is down-sampled

by a factor of 2 to produce an image ¼ the size to start

the next level.

21

Lowe��s Pyramid Scheme

s+2 filters

_{s+1}=2^{(s+1)/s}_{0}

**.**

**.**

_{i}=2^{i/s}_{0}

**.**

**.**

_{2}=2^{2/s}_{0}

_{1}=2^{1/s}_{0}

_{0}

s+3

images

including

original

s+2

differ-

ence

images

The parameter **s**
determines the number of images per octave.

22

**Key point localization**

- Detect maxima and minima of difference-of-Gaussian in scale space

- Each point is compared to
its 8 neighbors in the current image and 9 neighbors each in the scales
above and below

For each max or min found,

output is the **location**
and

the **scale**.

s+2 difference images.

top and bottom ignored.

s planes searched.

23

Scale-space extrema
detection: experimental results over 32 images that were synthetically
transformed and noise added.

- Sampling in scale for efficiency
- How many scales should be used per octave? S=?
- More scales evaluated, more keypoints found
- S < 3, stable keypoints increased too
- S > 3, stable keypoints decreased
- S = 3, maximum stable keypoints found

% detected

% correctly matched

average no. detected

average no. matched

Stability

Expense

24

Keypoint localization

- Once a keypoint candidate is found, perform a detailed fit to nearby data to determine
- location, scale, and ratio of principal curvatures
- In initial work keypoints were found at location and scale of a central sample point.
- In newer work, they fit a 3D quadratic function to improve interpolation accuracy.
- The Hessian matrix was used to eliminate edge responses.

25

Eliminating the Edge
Response

- Reject flats:
- < 0.03
- Reject edges:

- r < 10

- What does this look like?

Let be the eigenvalue with

larger magnitude and
the smaller.

Let r = /.

So = r

(r+1)^{2}/r
is at a

min when the

2 eigenvalues

are equal.

26

3. Orientation assignment

- Create histogram of local gradient directions at selected scale
- Assign canonical orientation at peak of smoothed histogram
- Each key specifies stable 2D coordinates (x, y, scale,orientation)

If 2 major orientations, use both.

27

Keypoint localization
with orientation

832

729

536

233x189

initial keypoints

keypoints after

gradient threshold

keypoints after

ratio threshold

28

4. Keypoint Descriptors

- At this point, each keypoint has
- location
- scale
- orientation
- Next is to compute a descriptor for the local image region about each keypoint that is
- highly distinctive
- invariant as possible to variations such as changes in viewpoint and illumination

29

Normalization

- Rotate the window to standard orientation

- Scale the window size based
on the scale at which the point was found.

30

Lowe��s Keypoint Descriptor

(shown with 2 X 2 descriptors over 8 X 8)

In experiments, 4x4 arrays of 8 bin histogram is used,

a total of 128 features for one keypoint

31

Lowe��s Keypoint Descriptor

- use the normalized region about the keypoint
- compute gradient magnitude and orientation at each point in the region
- weight them by a Gaussian window overlaid on the circle
- create an orientation histogram over the 4 X 4 subregions of the window
- 4 X 4 descriptors over 16 X 16 sample array were used in practice. 4 X 4 times 8 directions gives a vector of 128 values.

...

32

Using SIFT for Matching ��Objects��

33

34

Uses for SIFT

- Feature points are used also for:
- Image alignment (homography, fundamental matrix)
- 3D reconstruction (e.g. Photo Tourism)
- Motion tracking
- Object recognition
- Indexing and database retrieval
- Robot navigation
- �� many others

[ Photo Tourism: Snavely et al. SIGGRAPH 2006 ]

- TOUCHPAD MODEL TSM9921-34 Specification
- เคาโครงวิทยานิพนธ
- 'Documentary Studies': Dimensions of Transition and Continuity John Corner Over the last 30 years there has been a huge gro
- AWSTATS DATA FILE 6.6 (build 1.887)
- March Calendar - FW
- Design & Implementation of Cost Effective Wireless Power Transmission Model: GOOD BYE Wires
- Study Tip
- Conditional Fuzzy C Means A fuzzy clustering approach for mining event-related dynamics Christos N. Zigkolis 2 Contents
- Alienation, Irony, and German Romanticism Rado Pribic Lafayette College, Easton, PA At the end of the eighteenth century, there
- Title 12--DEPARTMENT OF
- Printable PDF Graph Paper - 5 mm Squares - Dark
- HamptonRoads_Economic Impact of The Accounting Industry
- NSF PROPOSAL
- Mark Schemes for the Units June 2006 Chemistry
- The Matrix, Liberal Education, and Other Splinters in the Mind
- 1. The Market Economy
- Untitled
- SISTEMAS BRASILEIROS DE DESENVOLVIMENTO CIENTÍFICO E TECNOLÓGICO
- Digitális felzárkóztatás – könyvtárosok az e-közszolgáltatásokért A Magyar Könyvtárosok Egyesületének a Miniszterelnöki Hivatal által m
- Interface between data terminal equipment (DTE) and data circuit-terminating equipment (DCE), ...
- Yi Lu Page 1 Curriculum Vitae Yi Lu Department of Chemistry University of Illinois at Urbana-Champaign Urbana, IL 61801 (217) 333-2619 (phone) (217) 244

- Polishing manipulator
- Trajectory planning
- VC++
- Simulation
- microemulsion
- micriemulsion based gels
- drug delivery
- sulfadiazine sodium
- metronidazole
- 4 acetamino phenol
- Helicobacter pylori
- β glucuronic acid glycosidase
- Primary hepatolithiasis
- Pressure sores
- Prevention
- Treatment
- Community Care
- Education and Training
- Radio frequency identification
- Antenna
- Impedance matching

All Rights Reserved Powered by Free Document Search and Download

Copyright © 2011This site does not host pdf,doc,ppt,xls,rtf,txt files all document are the property of their respective owners. complaint#nuokui.com