计算机视觉157.2structurefrommotionII

《计算机视觉157.2structurefrommotionII》由会员分享，可在线阅读，更多相关《计算机视觉157.2structurefrommotionII（34页珍藏版）》请在装配图网上搜索。

1、Multi-frame Structure from MotionIssues in SFMTrack lifetimeNonlinear lens distortionDegeneracy and critical surfacesPrior knowledge and scene constraintsMultiple motionsTrack lifetimeevery 50th frame of a 800-frame sequenceTrack lifetimelifetime of 3192 tracks from the previous sequenceTrack lifeti

2、metrack length histogramNonlinear lens distortionNonlinear lens distortioneffect of lens distortionPrior knowledge and scene constraintsadd a constraint that several lines are parallelPrior knowledge and scene constraintsadd a constraint that it is a turntable sequenceFactorizationTomasi&Kanade,IJCV

3、 92Problem statementNotationsn 3D points are seen in m viewsq=(u,v,1):2D image pointp=(x,y,z,1):3D scene point:projection matrix:projection functionqij is the projection of the i-th point on image jij projective depth of qij)(ijijpq)/,/(),(zyzxzyxzijStructure from motionEstimate Mj and pi to minimiz

4、e);(log),(1111ijijmjniijnmPwqppp otherwisej in view visibleis if 01iijpw Assume isotropic Gaussian noise,it is reduced to21111)(),(ijijmjniijnmwqpppSFM under orthographic projection2D image pointorthographicprojectionmatrix3D scenepointimageoffsettpq12321312Trick Choose scene origin to be centroid o

5、f 3D points Choose image origins to be centroid of 2D points Allows us to drop the camera translation:pq factorization(Tomasi&Kanade)n332n2n21n21pppqqqprojection of n features in one image:n332mn2m2121212222111211nmmnmmnnpppqqqqqqqqqprojection of n features in m imagesW measurementM motionS shapeKey

6、 Observation:rank(W)=3n33m2n2mSMW Factorization Technique W is at most rank 3(assuming no noise)We can use singular value decomposition to factor W:Factorization S differs from S by a linear transformation A:Solve for A by enforcing metric constraints on M)(ASMASMW1n33m2n2mSMWknownsolve forMetric co

7、nstraintsOrthographic Camera Rows of are orthonormal:Enforcing“Metric”Constraints Compute A such that rows of M have these propertiesMAM 1001TTrick(not in original Tomasi/Kanade paper,but in followup work)Constraints are linear in AAT:Solve for G first by writing equations for every i in M Then G=AA

8、T by SVD(since U=V)TTTTwhereAAGGAA1001nm2n33m2n2mESMWFactorization with noisy data SVD gives this solution Provides optimal rank 3 approximation W of Wnm2n2mn2mEWW Approach Estimate W,then use noise-free factorization of W as before Result minimizes the SSD between positions of image features and pr

9、ojection of the reconstructionResultsResults2/3/2005Structure from Motion21ExtensionsParaperspective Poelman&Kanade,PAMI 97Sequential Factorization Morita&Kanade,PAMI 97Factorization under perspective Christy&Horaud,PAMI 96 Sturm&Triggs,ECCV 96Factorization with Uncertainty Anandan&Irani,IJCV 2002Pe

10、rspective and Perspective FactorizationObject-centered projectionthe object-centered projection modelPerspective and Perspective Factorizationthe object-centered projection modelIn practice,after an initial reconstruction,the values of j can be estimated independently for each frame by comparing rec

11、onstructed and sensed point positions.Once the j have been estimated,the feature locations can then be corrected before applying another round of factorization.Bundle AdjustmentBundle Adjustment The term”bundle”refers to the bundles of rays connecting camera centers to 3D points.The term”adjustment”

12、refers to the iterative minimization of re-projection error.Alternative terms for this in the vision community include optimal motion estimation and non-linear least squares.2/3/2005Structure from Motion25Bundle Adjustment2/3/2005Structure from Motion26The formula for the radial distortion function

13、isThe feature location measurements xij now depend not only on the point(track index)i,but also on the camera pose index j,Bundle Adjustment2/3/2005Structure from Motion27The leftmost box performs a robust comparison of the predicted and measured 2D locations after re-projection.is the noise covaria

14、nce2/3/2005Structure from Motion28Bundle AdjustmentWhat makes this non-linear minimization hard?many more parameters:potentially slow poorer conditioning(high correlation)potentially lots of outliers gauge(coordinate)freedom2/3/2005Structure from Motion29Lots of parameters:sparsityOnly a few entries

15、 in Jacobian are non-zero(a)Bipartite graph for a toy structure from motion problem and(b)its associated Jacobian J and(c)Hessian A.2/3/2005Structure from Motion30Sparse Cholesky(skyline)First used in finite element analysisApplied to SfM by Szeliski&Kang 1994 structure|motion fill-in2/3/2005Structu

16、re from Motion31Conditioning and gauge freedomPoor conditioning:use 2nd order method use Cholesky decompositionGauge freedom fix certain parameters(orientation)or zero out last few rows in Cholesky decomposition2/3/2005Structure from Motion32Robust error modelsOutlier rejection use robust penalty ap

17、pliedto each set of jointmeasurements for extremely bad data,use random sampling RANSAC,Fischler&Bolles,CACM812/3/2005Structure from Motion33RANdom SAmple ConsensusRelated to least median squares Stewart991.Repeatedly select a small(minimal)subset of correspondences2.Estimate a solution(structure&mo

18、tion)3.Count the number of“inliers”,|e|(for LMS,estimate med(|e|)4.Pick the best subset of inliers5.Find a complete least-squares solution2/3/2005Structure from Motion34CorrespondencesCan refine feature matching after a structure and motion estimate has been produced decide which ones obey the epipolar geometry decide which ones are geometrically consistent(optional)iterate between correspondences and SfM estimates using MCMCDellaert et al.,Machine Learning 2003