Home > SIFT The SIFT (Scale Invariant Feature Transform) Detector and Descriptor developed by David Lowe University of British Columbia Initial paper IC
The SIFT (Scale Invariant Feature
Transform) Detector
and Descriptor
developed by David Lowe
University of British Columbia
Initial paper ICCV 1999
Newer journal paper IJCV 2004
2
Review: Matt Brown��s Canonical Frames
3
Multi-Scale Oriented
Patches
[ Brown, Szeliski, Winder CVPR 2005 ]
4
Application: Image
Stitching
[ Microsoft Digital Image Pro version 10 ]
Ideas from Matt��s
Multi-Scale Oriented Patches
5
6
Implementation Concern:
How do you rotate a patch?
7
Rotating a Patch
empty canonical
patch
patch detected in the image
x�� = x cos�� – y sin��
y�� = x sin�� + y cos��
T
T
counterclockwise rotation
(x,y)
(x��,y��)
8
Using Bilinear Interpolation
x
y
I00
I10
I01
I11
9
SIFT: Motivation
1But Schmid and Mohr developed a rotation invariant descriptor for it in 1997.
10
Idea of SIFT
SIFT Features
11
Claimed Advantages
of SIFT
12
Overall Procedure
at a High Level
Search over multiple scales and image
locations.
Fit a model to detrmine location and scale.
Select keypoints based on a measure of
stability.
Compute best orientation(s) for each
keypoint region.
Use local image gradients at selected scale and rotation
to describe each keypoint region.
13
1. Scale-space extrema
detection
14
Aside: Image Pyramids
Bottom level is the original image.
2nd level is derived from the
original image according to
some function
3rd level is derived from the
2nd level according to the same
funtion
And so on.
15
Aside: Mean Pyramid
Bottom level is the original image.
At 2nd level, each pixel is the mean
of 4 pixels in the original image.
At 3rd level, each pixel is the mean
of 4 pixels in the 2nd level.
And so on.
mean
16
Aside: Gaussian Pyramid
At each level, image is smoothed and reduced in size.
Bottom level is the original image.
At 2nd level, each pixel is the result
of applying a Gaussian mask to
the first level and then subsampling
to reduce the size.
And so on.
Apply Gaussian filter
17
Example: Subsampling
with Gaussian pre-filtering
G 1/4
G 1/8
Gaussian 1/2
18
Lowe��s Scale-space Interest Points
[ T. Lindeberg IJCV 1998 ]
19
Lowe��s Scale-space Interest Points:
Difference of Gaussians
20
Lowe��s Pyramid Scheme
with Gaussians to produce a set
of scale space images.
by a factor of 2 to produce an image ¼ the size to start
the next level.
21
Lowe��s Pyramid Scheme
s+2 filters
s+1=2(s+1)/s0
.
.
i=2i/s0
.
.
2=22/s0
1=21/s0
0
s+3
images
including
original
s+2
differ-
ence
images
The parameter s determines the number of images per octave.
22
Key point localization
For each max or min found,
output is the location and
the scale.
s+2 difference images.
top and bottom ignored.
s planes searched.
23
Scale-space extrema
detection: experimental results over 32 images that were synthetically
transformed and noise added.
% detected
% correctly matched
average no. detected
average no. matched
Stability
Expense
24
Keypoint localization
25
Eliminating the Edge
Response
Let be the eigenvalue with
larger magnitude and
the smaller.
Let r = /.
So = r
(r+1)2/r is at a
min when the
2 eigenvalues
are equal.
26
3. Orientation assignment
If 2 major orientations, use both.
27
Keypoint localization
with orientation
832
729
536
233x189
initial keypoints
keypoints after
gradient threshold
keypoints after
ratio threshold
28
4. Keypoint Descriptors
29
Normalization
30
Lowe��s Keypoint Descriptor
(shown with 2 X 2 descriptors over 8 X 8)
In experiments, 4x4 arrays of 8 bin histogram is used,
a total of 128 features for one keypoint
31
Lowe��s Keypoint Descriptor
...
32
Using SIFT for Matching ��Objects��
33
34
Uses for SIFT
[ Photo Tourism: Snavely et al. SIGGRAPH 2006 ]
All Rights Reserved Powered by Free Document Search and Download
Copyright © 2011