Home > EVC: Image Processing & Computer Vision

EVC: Image Processing & Computer Vision

Page 1
04.03.2013 1
Einf��hrung in Visual Computing (EVC) Image Processing & Computer Vision
186.822 VU 5.0 6 ECTS Robert Sablatnig
1
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
EVC: Image Processing & Computer Vision
▪ Content:
▪ What are the basic concepts of Image Processing and Computer
http://www.caa.tuwien.ac.at/cvl/teaching/sommersemester/evc
Vision and how are they used in applications? The course answers these questions by describing the creation of digital images using digital cameras and the subsequent steps in order to derive
information kept in digital images automatically.
▪ A closer look is taken into classical image processing techniques like
image enhancement and compression.
▪ The next step consists in the development of digital filters and
segmentation techniques in order to be able to extract specific
information.
Interest Points Computational Photography, 3D and motion are
further topics.
▪ Application of Algebra and Analysis in reality
2
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 2
04.03.2013 2
Logistics
▪ Lectures: 13:00 - 15:00 ▪ Instructors: Robert Sablatnig (VO) and
g ( ) Sebastian Zambanini (UE)
▪ Textbook: 4 A4 pages available at Lectures and Website ▪ Further Reading:
▪ Richard Szeliski, Computer Vision: A Modern Approach
http://szeliski.org/Book/
▪ Sonka Hlavac Boyle: Image Processing Analysis and
Sonka, Hlavac, Boyle: Image Processing, Analysis, and Machine Vision, 2nd Edition
▪ Webpage:
http://www.caa.tuwien.ac.at/cvl/teaching/sommersemester/evc
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
3
Readings
4
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 3
04.03.2013 3
Introduction: What is Image Processing? Computer Graphics vs. Computer Vision
6
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 4
04.03.2013 4
Categorization
▪ Image Processing
Manipulation of Image Data,
p f g ,
▪ Like removal of Noise, Correction of Sharpness on digital
images.
▪ Computer Vision
Generation of non-graphical Data from images, ▪ Like Character-and Text Recognition, Segmentation of images
into „interesting�� parts, Detection of lines and corners. „ g p ,
▪ Computer Graphics
Generation of Images from non-graphical data, ▪ like bar charts, 3d graphics „VR�� in real time, graphical outputs
7
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
Categorization
▪ Image Editing: Manipulation of Images (e.g. Photoshop)
▪ Visually ▪ Visually ▪ Interactive ▪ User-defined Parameters
▪ Image Processing: Mathematical algorithmic processes
▪ Image enhancement
g
▪ Image transformation (geometric) ▪ Image compression ▪ Image segmentation
8
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 5
04.03.2013 5
Kategorisierung
9
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
Example Image Processing: Filter (Noise Removal)
10
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 6
04.03.2013 6
Example Image Processing: Image Enhancement
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
11
Example: Image Restoration
12
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 7
04.03.2013 7
Example: Special Effects
13
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
Difference: Pattern Recognition – Image Processing?
▪ Pattern Recognition:
Classification of Patterns into a (finite) number of pre-defined
( ) p
classes
▪ like 2-dimensional patterns, OCRStandard book: Duda and Hart 1973, "Pattern Classification and
Scene Analysis"
▪ Image Processing:
Processing of an image to get a new image that is better suited
g
g g
g
for a specific task.
▪ Image enhancement, image transformation, image
compression, image segmentation, image restauration��
Standard book: Rosenfeld and Kak 1982, "Digital Picture
Processing", 2nd Edition
14
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 8
04.03.2013 8
Example Pattern Recognition
15
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
Example Pattern Recognition
16
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 9
04.03.2013 9
Examples for Pattern Recognition
17
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
Example: Computer Vision
▪ Face Detection
18
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 10
04.03.2013 10
Google Street View
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
19
Google Street View
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
20

Page 11
04.03.2013 11
Introduction: What is Computer Vision? Computer Vision
▪ Vision is derived from Human 
Vision (Human Visual System)
( y )
▪ Humans „see�� in 3 Dimensions
=> Computer Vision has 3d components
▪ Evolution millions of years: Human
visual system not faultless
∎ => if human visual system is not
faultless how can we expect from a machine that it is?
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
22

Page 12
04.03.2013 12
What is Computer Vision ?
"Computer Vision describes the automatic deduction of the structure and the properties of a (possible dynamic) three- p p f (p y )
dimensional world from either a single or multiple two- dimensional images of the world"
- Vishvjit S. Nalwa: A guided tour of computer vision. Addison-Wesley 1993
▪ Images: Color or Grayscale
C Fi d bl
▪ Camera: Fixed or movable
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
23
Computer Vision – Industry Related
▪ Computer Vision is an exciting new
research area that studies how to make
computers efficiently perceive, process, and understand visual data such as images and videos. The ultimate goal is for computers to emulate the striking
perceptual capability of human eyes and brains, or even to surpass and assist the
human in certain ways. – Microsoft Research
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
24

Page 13
04.03.2013 13
Computer Vision
▪ At least three goals: 1
Understand biological visual systems
1.
Understand biological visual systems
2. Build machines that see 3.
Understand fundamental processes of seeing
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
25
Computer Vision
We still do not know
▪ Is vision a well organized process with fundamental principles or ▪ a bag of tricks
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
26

Page 14
04.03.2013 14
Goals and Applications of Computer Vision
▪ It is not the goal of Computer Vision to develop a robot that is
similar to humans [Whitney86]
[ y ]
▪ Goal is to surpass and assist humans ▪ Applications:
▪ Automation (Assembly line) ▪ Inspection (Measuring of Parts) ▪ Remote Sensing (Maps) ▪ Human - Computer Interfaces ▪ Systems for Disabled ▪ Many more����
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
27
Computer Vision vs. Human Vision
▪ Why not simply copy human vision researched by
neurophysiologists, psychologists, and psychophysics? [Levine91] p y g ,py g , py p y [ ]
Eye research is finished Human Vision research is not! ▪ Seeing is not only a process within the eye – eye is only
producing images formed to ��impressions�� by the brain
▪ => Beginning of Computer Vision in the area of Artificial
I llintegence
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
28

Page 15
04.03.2013 15
Computer Vision vs. Seeing
▪ Seeing has adopted itself to environment und therefore not
faultless! Is Seeing an integral part of intelligence?
▪ Is Seeing an integral part of intelligence? ▪ Do we see reality – or what we want to see? ▪ Is Seeing and Thinking separable?
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
29
It��s Just An Illusion: Visual Illusions
▪ Classical optical illusions
Zöllner Illusion (1860) Poggendorf Illusion (1860)
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
30

Page 16
04.03.2013 16
Visual Illusions
▪ Classical optical illusions
Helmholtz Squares (1866) M��ller-Lyer Illusion (1860)
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
31
Visual Illusions
▪ Non existing 3D objects:
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
32

Page 17
04.03.2013 17
Perspective Illusions by Julian Beever
Make Poverty History Babyfood...
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
33
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
34

Page 18
04.03.2013 18
M.C. Escher Ambigious Interpretations
Indian vs. Inuit Young/Old Lady
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
36

Page 19
04.03.2013 19
Rotary Effects
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
37
Are these phenomena caused by manipulation of the visual system by unreal images?
▪ After all, if we cannot believe what we see, what are we to
believe?
Absolute faith in human visual system is not justified for 2-
dimensional images!
∎ Either: 3d images -> real world ∎ or: right limitations of scene features (perspective ∎ or: right limitations of scene features (perspective,
lighting, direction, etc.)
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
38

Page 20
04.03.2013 20
Are these phenomena caused by manipulation of the visual system by unreal images?
Every image is an image of an object, which is understandable only to those who know about its origins and are able to create a y
g corresponding image in their imagination (Helmholtz, 1910)
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
39
History
▪ Serious attempts to create computer vision systems have now a
history of 40-50 years. y y
▪ First digital image 1964 (Mariner 4) ▪ Focus on Sensors �� Digital Image Processing ▪ Analysis Focus �� Computer Vision
Mariner 4: First the first close-up image ever taken of Mars 1964
Source: NASA Source: NASA
40
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 21
04.03.2013 21
Computer Vision
The systems today are still exceedingly limited in their performance �� considerable room for improvement p
p
41
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
Future Challenges of Computer Vision
▪ Where do the innovations come from? ▪ 1. Hardware
Wavi Xtion Kinect
▪ 2. Algorithms/Software
Kinect
http://www.gigapixel.com
42
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 22
04.03.2013 22
Hardware
First time that HW is no longer a real limitation !! ▪ Processing ▪ Image Resolution ▪ Storage ▪ Internet ▪ Mobile Devices ▪ Networks of cameras
43
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
Processing
▪ Moore��s Law still holds! ▪ Multi-core CPUs ▪ Highly Parallel �� GPUs (+ Software eg. Cuda)
44
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 23
04.03.2013 23
Processing
▪ What to expect: ▪ Image Processing (Feature Extraction) will be instantaneous ▪ Image Processing (Feature Extraction) will be instantaneous ▪ Real-time Libs: Basic algorithms (IPP,Cuda ��) ▪ Real time vision (cf. Real time Rendering) ▪ Parallelization (GPU implementation) is a feature of an algorithm
45
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
Resolution
Ever growing resolution:
▪ 1975: 100 x 100 = 0 01 MP ▪ 1975: 100 x 100 = 0.01 MP ▪ 2009: 13.280 x 9.184 Pixel = 120 MP ▪ UltraCamx: 216 MP
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
46

Page 24
04.03.2013 24
What can we do with that?
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
47
Some Questions to Tackle
▪ How/What shall we sample in space, time,
wavelengths, polarization, ��.? g , p ,
48
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 25
04.03.2013 25
Some Questions to Tackle
Optimal sampling strategies in 3D/4D ��
▪ No constraints of view-points ▪ No constraints of view points ▪ Multiple Images �� Redundancy ▪ Control the illumination of each pixel ▪ How real are images?
49
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
Storage
▪ We have huge disks and we fill them ▪ A color VGA image ~1 MB ▪ A color VGA image 1 MB ▪ Every 10 second 1 image
�� 8.4 GB a day �� 240 TB a Life
▪ Can we index that? �� Algorithms
50
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 26
04.03.2013 26
Internet
Huge repository of images
▪ Flickr:
Aug 2011 ~ 6 Billion Photos On line
▪ Aug. 2011 ~ 6 Billion Photos On-line ▪ 4.5 million photo added per day
▪ YouTube:
▪ 65.000 new Videos a day ▪ 1 trillion video playbacks ▪ 20% of Internet Traffic
▪ Facebook:
▪ 600 Million users ▪ 3 Billion photos added per month
What can we do with these images?
Source: Internet 2011 in numbers
51
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
What is Vision?
▪ What does it mean, to see? ��to know what is where by looking��. ▪ How to discover from images what is present in the world where ▪ How to discover from images what is present in the world, where
things are, what actions are taking place.
from Marr, 1982
52
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 27
04.03.2013 27
The Importance of Images
▪ Some images are more important
than others
▪ 100 million $
��Dora Maar au Chat�� Pablo Picasso, 1941
53
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
Where is now Computer Vision? (only a few examples)

Page 28
04.03.2013 28
Assisted Driving
Pedestrian and car detection
meters meters Ped Ped Car
Lane detection
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
55
• Collision warning systems with adaptive cruise control, • Lane departure warning systems, • Rear object detection systems,
Iris Recognition
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
56
JOHN DAUGMAN
http://www.cl.cam.ac.uk/~jgd1000/iriscollage.jpg

Page 29
04.03.2013 29
Image Stitching
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
57
Image Stitching
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
58
Brown, Lowe, 2007

Page 30
04.03.2013 30
Photo Turism
PhotoSynth
Snavely et al. 2006
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
59
(Goesele et al. 2007).
Finding Paths through the World's Photos
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
60

Page 31
04.03.2013 31
Why is Vision hard – The Plenoptic Function The Structure of Ambient Light
62
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 32
04.03.2013 32
The Structure of Ambient Light
63
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
The Plenoptic Function
Adelson & Bergen, 91 The intensity P can be parameterized as:
P (��, ϕ, t, ��, Vx, Vy, Vz)
��The complete set of all convergence points constitutes the permanent possibilities of vision.�� Gibson
Image coordinates (sperical) Color Time 3D space
64
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 33
04.03.2013 33
Measuring the Plenoptic Function
▪ ��The significance of the plenoptic function is this: The
world is made of 3D objects but these objects do not world is made of 3D objects, but these objects do not
communicate their properties directly to an observer.
Rather, the objects fill the space around them with the
pattern of light rays that constitutes the plenoptic
function, and the observer takes samples from this function.�� Adelson & Bergen 91. function. Adelson & Bergen 91.
65
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
Measuring the Plenoptic Function
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
66
Why is there no picture appearing on the paper?

Page 34
04.03.2013 34
Measuring the Plenoptic Function
▪ Light rays from many different parts of the scene strike the same
point on the paper. p p p
Forsyth & Ponce
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
67
Camera Obscura
68

Page 35
04.03.2013 35
Measuring the Plenoptic Function
The camera obscura The pinhole camera
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
69
Camera Obscura
▪ Latin:
▪ Camera for "vaulted
chamber/room"
▪ obscura for "dark" ▪ together "darkened
chamber/room��
"When images of illuminated objects penetrate through a small
▪ When images of illuminated objects ... penetrate through a small
hole into a very dark room ... you will see [on the opposite wall] these objects in their proper form and color, reduced in size ... in a reversed position, owing to the intersection of the rays". - Da Vinci
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
70
http://www.acmi.net.au/AIC/CAMERA_OBSCURA.html (Russell Naughton)

Page 36
04.03.2013 36
Camera Obscura
▪ Used to observe eclipses (eg., Bacon, 1214-1294) ▪ By artists (eg Vermeer) ▪ By artists (eg., Vermeer).
71
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
Camera Obscura
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
72

Page 37
04.03.2013 37
Camera Obscura
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
73
Jetty at Margate England, 1898.
Pinhole Camera
▪ Simple Model of Camera Obscura: Pinhole camera
▪ Very small hole (aperture ~ 0), Light passes through the hole
and forms image on back (upside down and flipped)
74
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 38
04.03.2013 38
Pinhole Camera
▪ Abstract camera model - box with a small hole in it ▪ Pinhole cameras work in practice ▪ Pinhole cameras work in practice
75
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
Commercial Pinhole Cameras
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
76

Page 39
04.03.2013 39
Playing with Pinholes
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
77
Effect of Pinhole Size
Wandell, Foundations of Vision, Sinauer, 1995
78
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction

Page 40
04.03.2013 40
Limits of Pinhole Cameras
▪ A picture of a filament taken with a pinhole camera. In the image
on the left, the hole was too big (blurring), and in the image on , g ( g), g the right, the hole was too small (diffraction).
Ruechardt, 1958
79
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
Pinhole Camera Images with Variable Aperture
▪ Why not making the aperture
as small as possible?
l h h h
2  mm 1  mm 0.6 mm 0.35 mm
▪ Less light gets through ▪ Diffraction effect
80
Robert Sablatnig, Computer Vision Lab, EVC-2: Introduction
0.15 mm 0.07 mm
Search more related documents:EVC: Image Processing & Computer Vision

Set Home | Add to Favorites

All Rights Reserved Powered by Free Document Search and Download

Copyright © 2011
This site does not host pdf,doc,ppt,xls,rtf,txt files all document are the property of their respective owners. complaint#nuokui.com
TOP