From Homography to Image Stitching
cold
2020/08/19 发布于 技术 分类
This presentation was on 2020.8.3, introducing the topic of image stitching. First, I clarify the deduction process and application constraints of homography, which I found was the most important concept in terms of image stitching. After that, I gave the demo of the general pipeline of image stitching, followed by detailed introduction on several classic works, including AutoStitch, DWH, SVA, APAP, AANAP, Seamdriven methods.
登录发表评论
文字内容
1. From Homography to Image Stitching 张寒萌 2020.08.03 1
2. Homography 2
3. Homography y x x y z z H1 z H2 y x • In camera calibration, we take photos of the checkboard from different perspectives. • For each image, we detect the inner corners and thus calculate a homography between the image and the checkboard plane. 3
4. Homography • From world coordinate to camera 1 coordinate, 1 1 • Suppose the checkboard plane falls on z = 0, we have 1 1 1 • Let , we derive 1 1 Case1: homography from a realworld plane to an image 4
5. Homography • Perform the same conduction for both H1 and H2, we have 1 1 2 2 1 2 • From image1 to image2: 1 Case2: homography from an image to another image 5
6. • For case1, from image1 to plane: • Using calibration board, we can naturally set (X, Y, Z) of keypoints in the world coordinates, thus calculating homography. 1 1 1 • For case2, from image1 to image2: • By establishing pointcorrespondences, we can derive their homography. 6
7. Homography • Homography can describe the transformation • between an image and a plane • between two images take from the same scene, but only under the condition that the scene is planar!!! No parallax(no image depth) The scene is roughly planar (or far enough) or Views differ purely by rotation (no translation) 7
8. parallax • Parallax problem in image stitching. • Notice the horizontal spatial order of the tower and the red sculpture. 8
9. Image stitching 9
10. pipeline feature extraction feature matching calculate H using RANSAC Image warping Image blending • Let us compose these images into a larger image 10
11. pipeline feature extraction feature matching calculate H using RANSAC Image warping Image blending • Find keypoints and represent by descriptors • Most classic descriptors: SIFT, SURF, ORB… 11
12. pipeline feature extraction feature matching calculate H using RANSAC Image warping Image blending • Matching features and establish pointcorrespondences. • Matching methods: BruteForce Matcher, crossCheck matcher, KNN matcher, FLANN matcher 12
13. pipeline feature extraction feature matching calculate H using RANSAC Image warping Image blending • RANSAC: Random Sample Consensus • Remove wrong correspondences and calculate H 13
14. pipeline feature extraction feature matching calculate H using RANSAC Image warping Image blending • Represent the images in common coordinates with translation • Now we can compose the images 14
15. pipeline feature extraction feature matching calculate H using RANSAC Image warping Image blending
16. challenges Repeated texture weak texture foreground occlusion Large parallax 16
17. Autostitch • Autostitch • An early and classic method for image stitching, widely used in many applications like OpenCV stitcher, Photoshop… • algorithms • Kd tree • Bundle adjustment • Multiband blending Brown M, Lowe D G. Recognising Panoramas [C]. ICCV. 2003 Brown M, Lowe D G. Automatic Panoramic Image Stitching using Invariant Features [J]. IJCV. 2007. 17
18. DWH • Divide into all feature points into two cluster using Kmeans. For each cluster, calculate its Homography and thus get and • The relationship between two overlapping images uses a blending of two homographes, Hg and Hd, expressed as: (1) • Assign a weight to each pixel location pij based on spatial proximity to determine how much of each homography should be used. (2) where and are the distances to the closest feature points in the sets and respectively Gao J, Kim S J, Brown M S. Constructing image panoramas using dualhomography warping [C]// CVPR, 2011. 18
19. SVA • Model the relationship of two images with a set of Affine transformations. ps: Affine transformation has 6 degrees of freedom, while projective transformation has 8 degrees of freedom Lin W Y, Liu S, Matsushita Y, et al. Smoothly varying affine stitching [C]// CVPR, 2011. 19
20. APAP • APAP: • Asprojectiveaspossible • objective: • find warps that aim to be globally projective, yet allow local deviations to account for model inadequacy Zaragoza J, Chin T J, Brown M S, et al. Asprojectiveaspossible image stitching with moving DLT [C]// CVPR, 2013. 20 Zaragoza J, Chin T J, Tran Q H, et al. AsProjectiveAsPossible Image Stitching with Moving DLT [J]. TPAMI, 2014.
21. APAP • DLT(Direct Linear Transformation) • A basic method to estimate H from a set of point matches across and • Let and be matching points across overlapping images, A projective warp or homography aims to map to following the relation (1) • Eq.(1) is rewritten as the implicit condition and linearised (2) 2 • Only two of the rows are linearly independent，Let ith . DLT estimates the nine elements of H as be the firsttwo rows of (2) computed for the (3) • The solution is simply the least significant right singular vector of A. Zaragoza J, Chin T J, Brown M S, et al. Asprojectiveaspossible image stitching with moving DLT [C]// CVPR, 2013. 21 Zaragoza J, Chin T J, Tran Q H, et al. AsProjectiveAsPossible Image Stitching with Moving DLT [J]. TPAMI, 2014.
22. APAP • Moving DLT(Moving Direct Linear Transformation) • Main idea is to warp each using a location dependent homography (4) • where H∗ is estimated from the weighted problem (5) • The scalar weights change according to x∗ and are calculated as (6) Here, σ is a scale parameter, and xi is the coordinate in the source image I of onehalf of the ith point match • The problem in (5) can be written in the matrix form (7) • This is a weighted SVD problem, and the solution is simply the least significant right singular vector of Zaragoza J, Chin T J, Brown M S, et al. Asprojectiveaspossible image stitching with moving DLT [C]// CVPR, 2013. 22 Zaragoza J, Chin T J, Tran Q H, et al. AsProjectiveAsPossible Image Stitching with Moving DLT [J]. TPAMI, 2014. .
23. APAP • Efficient Learning for Image Stitching • Partitioning into cells • partition the source image I into a grid of C1×C2 cells. • For each cell, the centre coordinate is chosen as x∗ , and all pixels within the same cell are warped using the same H∗ . Zaragoza J, Chin T J, Brown M S, et al. Asprojectiveaspossible image stitching with moving DLT [C]// CVPR, 2013. 23 Zaragoza J, Chin T J, Tran Q H, et al. AsProjectiveAsPossible Image Stitching with Moving DLT [J]. TPAMI, 2014.
24. AANAP Based on APAP, (1) linearize the homography in the regions that do not overlap (2) estimate a global similarity transform using a subset of corresponding points in the overlapping regions (3) interpolate smoothly between the homography and the global similarity in the overlap ping regions Lin C C, Pankanti S U, Ramamurthy K N, et al. Adaptive asnaturalaspossible image stitching [C]// CVPR, 2015. 24
25. SeamDriven • • Mainstream: • step1 estimate H with the best geometric fit to align the images, using RANSAC; • step2 postprocess, like seamcutting Their work: • seamdriven image stitching strategy: evaluate the goodness of a transform based on the resulting visual quality of the seamcut Gao J, Li Y, Chin T J, et al. SeamDriven Image Stitching [C]// Eurographics (Short Papers), 2013. 25
26. SeamDriven • • Generating homography candidates • First, standard RANSAC is applied with a comparatively large n (500 in implementation) to fit the dominant plan • After this, remove the inliers of the dominant plane and apply a sequence of RANSAC instances each with a smaller n (50 in implementation) on the remaining matched points. • This helps to find homographies that align other smaller planar structures which may result in better seamcuts Computing the SeamCut • The seam computation can be formulated as a labeling problem on a Markov Random Field (MRF) which minimizes a global energy with the following form: where the datacost of each pixel is defined to be the gradient at that location: the smoothness cost between two pixels p and q is defined as:'>as: • Evaluating the Cut • For each pixel, p, along the seam, we estimate an error value, E(p) by extracting a 17 × 17 patch, P, centered at p and searching for its most similar patch in either I1 or I2. This can be expressed as:'>as: Gao J, Li Y, Chin T J, et al. SeamDriven Image Stitching [C]// Eurographics (Short Papers), 2013. 26
27. SeamDriven • Evaluating the Cut • For each pixel, p, along the seam, we estimate an error value, E(p) by extracting a 17 × 17 patch, P, centered a and searching for its most similar patch in either I1 or I2. This can be expressed as: Gao J, Li Y, Chin T J, et al. SeamDriven Image Stitching [C]// Eurographics (Short Papers), 2013. 27
28. SeamDriven: RANSAC Intro • RANSAC: Random Sample Consensus • RANSAC loop: • Select four feature pairs at random • Compute homography H (exact) • compute inliers where SSD(p, H(p)) < ε • Keep largest set of inliers • Recompute leastsquares H estimate on all of the inliers • This random sampling process is repeated n times and the hypothesis with the highest consensus size is chosen. • Gao J, Li Y, Chin T J, et al. SeamDriven Image Stitching [C]// Eurographics (Short Papers), 2013. 28
推荐

labuladong的算法小抄官方完整版
imikay

从开源项目汲取养分助力业务发展 景罗 PH...
PHPConChina

次时代Swoole 青年PHP的无尽探索 ...
PHPConChina

疫情之下K12在线教育系统的稳定性建设 陈...
PHPConChina

从总监到自由 Lukin PHPCon2020
PHPConChina

PHP 下 AOP 的实现与原理 黄朝晖 ...
PHPConChina

SRE提高服务质量和稳定性的套路 刘征 P...
PHPConChina

PHP大规模服务化的实践过程 丁盼盼 PH...
PHPConChina

PHP安全开发规范与审查 汤青松 PHPC...
PHPConChina

开源治理与数字化转型 庄表伟 PHPCon...
PHPConChina
分享