1 | % * This code was used in the following articles:
|
---|
2 | % * [1] Learning 3-D Scene Structure from a Single Still Image,
|
---|
3 | % * Ashutosh Saxena, Min Sun, Andrew Y. Ng,
|
---|
4 | % * In ICCV workshop on 3D Representation for Recognition (3dRR-07), 2007.
|
---|
5 | % * (best paper)
|
---|
6 | % * [2] 3-D Reconstruction from Sparse Views using Monocular Vision,
|
---|
7 | % * Ashutosh Saxena, Min Sun, Andrew Y. Ng,
|
---|
8 | % * In ICCV workshop on Virtual Representations and Modeling
|
---|
9 | % * of Large-scale environments (VRML), 2007.
|
---|
10 | % * [3] 3-D Depth Reconstruction from a Single Still Image,
|
---|
11 | % * Ashutosh Saxena, Sung H. Chung, Andrew Y. Ng.
|
---|
12 | % * International Journal of Computer Vision (IJCV), Aug 2007.
|
---|
13 | % * [6] Learning Depth from Single Monocular Images,
|
---|
14 | % * Ashutosh Saxena, Sung H. Chung, Andrew Y. Ng.
|
---|
15 | % * In Neural Information Processing Systems (NIPS) 18, 2005.
|
---|
16 | % *
|
---|
17 | % * These articles are available at:
|
---|
18 | % * http://make3d.stanford.edu/publications
|
---|
19 | % *
|
---|
20 | % * We request that you cite the papers [1], [3] and [6] in any of
|
---|
21 | % * your reports that uses this code.
|
---|
22 | % * Further, if you use the code in image3dstiching/ (multiple image version),
|
---|
23 | % * then please cite [2].
|
---|
24 | % *
|
---|
25 | % * If you use the code in third_party/, then PLEASE CITE and follow the
|
---|
26 | % * LICENSE OF THE CORRESPONDING THIRD PARTY CODE.
|
---|
27 | % *
|
---|
28 | % * Finally, this code is for non-commercial use only. For further
|
---|
29 | % * information and to obtain a copy of the license, see
|
---|
30 | % *
|
---|
31 | % * http://make3d.stanford.edu/publications/code
|
---|
32 | % *
|
---|
33 | % * Also, the software distributed under the License is distributed on an
|
---|
34 | % * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
|
---|
35 | % * express or implied. See the License for the specific language governing
|
---|
36 | % * permissions and limitations under the License.
|
---|
37 | % *
|
---|
38 | % */
|
---|
39 | function [] = OcclusionRemoveRefineMent(defaultPara, Wrlname, PairList, PostFixStrAfter) |
---|
40 | |
---|
41 | % This function do three thing |
---|
42 | % 1) detect the occlusion from the current model |
---|
43 | % 2) decide redundent plane |
---|
44 | % 3) Re-Render by not rendering the redundent plane information |
---|
45 | |
---|
46 | % Input: |
---|
47 | % 1) defaultPara - all parameter share for the whole stitching3d folder |
---|
48 | % 2) Wrlname - the name of the model, different by the PairList |
---|
49 | % 3) PairList - the linearly order of image that been added to the model |
---|
50 | |
---|
51 | % Fixed constant --- need to be put in defaultPara by Min later |
---|
52 | NegI = diag([1 1 -1]); |
---|
53 | PostFixStr = 'NonMono';%'NonMonoOccluMatched' |
---|
54 | if nargin <4 |
---|
55 | PostFixStrAfter = 'OccluRemove'; |
---|
56 | end |
---|
57 | DisCtsThre = 0.1; % fractional error |
---|
58 | OcclusionRatioThre = 1.1; |
---|
59 | |
---|
60 | % 1) Run occlusion detection and Matcheing for every pair of images |
---|
61 | |
---|
62 | ImgList = unique(PairList); |
---|
63 | NumImg = length(ImgList); |
---|
64 | |
---|
65 | % copy the ImgInfo of each image to another "PostFixStrAfter".mat file |
---|
66 | for i = 1:NumImg |
---|
67 | load([defaultPara.Fdir '/data/' ImgList{i} '/' Wrlname '_' ImgList{i} '_' PostFixStr '.mat']); |
---|
68 | save([defaultPara.Fdir '/data/' ImgList{i} '/' Wrlname '_' ImgList{i} '_' PostFixStrAfter '.mat'],'model'); |
---|
69 | end |
---|
70 | |
---|
71 | % detect occlusion for every pair of image |
---|
72 | for i = 1:NumImg |
---|
73 | for j = (i+1):NumImg |
---|
74 | |
---|
75 | [ImgInfo1 ImgInfo2 Pair GlobalScale] = ... % What to do with Pairs haven't been matched ??????????????????? |
---|
76 | LoadDataForFindOcclu(defaultPara, Wrlname, ImgList{i}, ImgList{j}, ... |
---|
77 | 0, PostFixStrAfter); % load all imformation needed for occlusion detection |
---|
78 | |
---|
79 | % Define variables |
---|
80 | H = 2274; |
---|
81 | V = 1704; |
---|
82 | Img1 = ImgInfo1.ExifInfo.IDName; |
---|
83 | Img2 = ImgInfo2.ExifInfo.IDName; |
---|
84 | I1=imreadbw([defaultPara.Fdir '/pgm/' Img1 '.pgm']); % function from sift |
---|
85 | I2=imreadbw([defaultPara.Fdir '/pgm/' Img2 '.pgm']); % function from sift |
---|
86 | ImgScale1 = size(I1); |
---|
87 | ImgScale2 = size(I2); |
---|
88 | |
---|
89 | % time consuming about 5mins |
---|
90 | if ~defaultPara.Flag.FlagPreloadOccluDetect |
---|
91 | [PointPix1 PointDepth1 FaceSetPickedIND1 POriReprojM1 FieldOccluPix1 OccluDist1 OccluedFaceSetIND1 ... |
---|
92 | PointPix2 PointDepth2 FaceSetPickedIND2 POriReprojM2 FieldOccluPix2 OccluDist2 OccluedFaceSetIND2] = ... |
---|
93 | FindOccluPair(defaultPara, ImgInfo1, ImgInfo2, Pair, GlobalScale, 0); % detect occlsion (use Ray as detector not the surf Features) |
---|
94 | % data Struction define: |
---|
95 | % PointPix PointDepth FaceSetPickedIND POriReprojM FieldOccluPix OccluDist OccluedFaceSetIND |
---|
96 | % -- allof the size that single ray pass through both FaceSet |
---|
97 | % FaceSetPickedIND1 used in DepthMap size or surfFeature size |
---|
98 | % OccluedFaceSetIND1 used in DepthMap size |
---|
99 | |
---|
100 | save([ defaultPara.Fdir '/data/' Img1 '_' Img2 '_OccluDetect_Ray.mat'],... |
---|
101 | 'PointPix1','PointDepth1','FaceSetPickedIND1','POriReprojM1','FieldOccluPix1','OccluDist1','OccluedFaceSetIND1',... |
---|
102 | 'PointPix2','PointDepth2','FaceSetPickedIND2','POriReprojM2','FieldOccluPix2','OccluDist2','OccluedFaceSetIND2'); |
---|
103 | else |
---|
104 | load([defaultPara.Fdir '/data/' Img1 '_' Img2 '_OccluDetect_Ray.mat']); |
---|
105 | end |
---|
106 | |
---|
107 | % Define occlusion if OccluDist > CentainThreshold |
---|
108 | Mask1 = PointDepth1./OccluDist1 > OcclusionRatioThre;%defaultPara.OccluDistThre; |
---|
109 | Mask2 = PointDepth2./OccluDist2 > OcclusionRatioThre;%defaultPara.OccluDistThre; |
---|
110 | |
---|
111 | % Pick the redundent plane and make the corresponding Sup to zero |
---|
112 | [model1 model2] = RedundentPlanePick(defaultPara, ImgInfo1, ImgInfo2, Pair, GlobalScale, ... |
---|
113 | FaceSetPickedIND1(Mask1), OccluedFaceSetIND1(Mask1), ... |
---|
114 | FaceSetPickedIND2(Mask2), OccluedFaceSetIND2(Mask2)); |
---|
115 | |
---|
116 | % storage the new ImgInfo.Model.PlaneParaInfo.SupOri |
---|
117 | model = model1; |
---|
118 | save([defaultPara.Fdir '/data/' Img1 '/' Wrlname '_' Img1 '_' PostFixStrAfter '.mat'],'model'); |
---|
119 | model = model2; |
---|
120 | save([defaultPara.Fdir '/data/' Img2 '/' Wrlname '_' Img2 '_' PostFixStrAfter '.mat'],'model'); |
---|
121 | end |
---|
122 | end |
---|
123 | |
---|
124 | % 2) Render of each image individiually |
---|
125 | |
---|
126 | for i = 1:NumImg |
---|
127 | |
---|
128 | % Data preparing and ReInfernece |
---|
129 | Default.Wrlname{1} = [Wrlname '_' ImgList{i} '_' PostFixStrAfter]; |
---|
130 | Default.filename{1} = ['../' ImgList{i}]; |
---|
131 | load([defaultPara.Fdir '/data/' ImgList{i} '/' Wrlname '_' ImgList{i} '_' PostFixStrAfter '.mat']); |
---|
132 | Default.OutPutFolder = [ defaultPara.OutPutFolder ImgList{i} '/']; |
---|
133 | |
---|
134 | if false |
---|
135 | DiffHori = conv2(model.PlaneParaInfo.FitDepth, [1 -1], 'valid'); |
---|
136 | DiffVert = conv2(model.PlaneParaInfo.FitDepth, [1; -1], 'valid'); |
---|
137 | maskH = [ (DiffHori./model.PlaneParaInfo.FitDepth(:,2:end) > DisCtsThre) zeros(size(model.PlaneParaInfo.FitDepth,1),1)]; |
---|
138 | maskH(end,:) = 0; |
---|
139 | maskV = [ (DiffVert./model.PlaneParaInfo.FitDepth(2:end,:) > DisCtsThre); zeros(1, size(model.PlaneParaInfo.FitDepth,2))]; |
---|
140 | maskV(:,end) = 0; |
---|
141 | model.PlaneParaInfo.SupOri(logical( maskH) ) = 0; |
---|
142 | model.PlaneParaInfo.SupOri(logical( maskV) ) = 0; |
---|
143 | end |
---|
144 | |
---|
145 | WrlFacestHroiReduce(model.PlaneParaInfo.Position3DFited, model.PlaneParaInfo.PositionTex, model.PlaneParaInfo.SupOri, ... |
---|
146 | Default.filename{1}, Default.Wrlname{1}, ... |
---|
147 | Default.OutPutFolder, 0, 0); |
---|
148 | % SingleImgReInference(defaultPara, model, ImgList{i}, LoadModelStatus(defaultPara.Fdir, Wrlname, ImgList{i}, 'GroundLevel')); |
---|
149 | |
---|
150 | % Build Meta Wrl file |
---|
151 | |
---|
152 | Path = [ defaultPara.OutPutFolder Wrlname '_' PostFixStrAfter '.wrl']; |
---|
153 | InLinePath = ['./' ImgList{i} '/' Default.Wrlname{1} '.wrl']; |
---|
154 | R = LoadModelStatus( defaultPara.Fdir, Wrlname, ImgList{i}, 'R'); |
---|
155 | Q = Rotation2Q(NegI*R*NegI);% for Wrl (-) z component; |
---|
156 | if any(isnan(Q)) |
---|
157 | Q = zeros(4,1); |
---|
158 | end |
---|
159 | % =============================== |
---|
160 | T = LoadModelStatus( defaultPara.Fdir, Wrlname, ImgList{i}, 'T'); |
---|
161 | WRLT = T; |
---|
162 | WRLT(3) = -WRLT(3);% for Wrl (-) z component; |
---|
163 | Scale = LoadModelStatus(defaultPara.Fdir, Wrlname, ImgList{i}, 'Scale'); |
---|
164 | if i == 1 |
---|
165 | BuildVrmlMetaModel(1, defaultPara.OutPutFolder, Path, InLinePath, Q, WRLT, repmat(Scale,3,1)); |
---|
166 | else |
---|
167 | BuildVrmlMetaModel(0, defaultPara.OutPutFolder, Path, InLinePath, Q, WRLT, repmat(Scale,3,1)); |
---|
168 | end |
---|
169 | end |
---|
170 | |
---|
171 | return; |
---|