1 | % $Author: svoboda $ |
---|
2 | % $Revision: 2.2 $ |
---|
3 | % $Id: README.txt,v 2.2 2005/05/24 09:15:30 svoboda Exp $ |
---|
4 | % $State: Exp $ |
---|
5 | |
---|
6 | WWW: |
---|
7 | http://cmp.felk.cvut.cz/~svoboda/SelfCal/ |
---|
8 | look at the home page to get the newest information, latest sources, |
---|
9 | publications, sample data etc. |
---|
10 | |
---|
11 | Authors: |
---|
12 | - Tomas Svoboda, svoboda@cmp.felk.cvut.cz, (design of the package, |
---|
13 | most of the codes), corresponding author |
---|
14 | |
---|
15 | - Daniel Martinec and Tomas Pajdla, {martid1,pajdla}@cmp.felk.cvut.cz |
---|
16 | (filling points) |
---|
17 | |
---|
18 | - Ondrej Chum, chum@fel.cvut.cz (RANSAC implementation) |
---|
19 | |
---|
20 | - Tomas Werner, werner@cmp.felk.cvut.cz (Projective Bundle Adjustment) |
---|
21 | |
---|
22 | - Jean-Yves Bouguet, jean-yves.bouguet@intel.com, (part of the Radial |
---|
23 | distortion computation) |
---|
24 | |
---|
25 | Just a short how-to for multicamera selfcalibration: |
---|
26 | ---------------------------------------- |
---|
27 | svoboda@vision.ee.ethz.ch, 08/2002 |
---|
28 | updated 12/2002, 01/2003, 02/2003, 03/2003, 06/2003, 07/2003 |
---|
29 | |
---|
30 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% |
---|
31 | %%% For BlueC users |
---|
32 | %%% Very short how-to for them who know |
---|
33 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% |
---|
34 | |
---|
35 | (the steps denoted by * are typically needed to be performed just once |
---|
36 | for each user) |
---|
37 | |
---|
38 | - Capture *.pvi files |
---|
39 | |
---|
40 | - convert *.pvi files to sequences of images |
---|
41 | |
---|
42 | * Go to */BlueCCal/BlueCFindingPoints and check all the three files |
---|
43 | there. They have some local settings which has to be set for each |
---|
44 | user differently. |
---|
45 | |
---|
46 | - run ./findpointsBlueC |
---|
47 | This script starts finding process on each of the cluster machine |
---|
48 | |
---|
49 | - collect the data stored locally on the cluster machines. |
---|
50 | |
---|
51 | * Edit configdata.m and expname.m in */BlueCCal/CommonCfgAndIO |
---|
52 | |
---|
53 | - Go to */BlueCCal/MultiCamSelfCal, run matlab |
---|
54 | |
---|
55 | From now on in matlab window: |
---|
56 | - >> gocal |
---|
57 | Wait for the results. It may take several minutes if you have many |
---|
58 | cameras and many points. |
---|
59 | |
---|
60 | Wait ...., and you are done if you are lucky ;-) |
---|
61 | |
---|
62 | |
---|
63 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% |
---|
64 | %%% Information relevant to the BlueC project |
---|
65 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% |
---|
66 | |
---|
67 | The CamSelfCal code is in /home/svoboda/Work/BlueCCal |
---|
68 | The auxiliary scripts and acquisition SW are in /home/svoboda/Work/BlueCAcquire |
---|
69 | |
---|
70 | You will also need to set some local variables. |
---|
71 | ############################################################################ |
---|
72 | # blue-c |
---|
73 | if ( -f /opt/modules/modules/init/tcsh) then |
---|
74 | source /opt/modules/modules/init/tcsh |
---|
75 | if ( -d ${HOME}/lib/modulefiles ) then |
---|
76 | module use ${HOME}/lib/modulefiles |
---|
77 | endif |
---|
78 | |
---|
79 | # module add blue-c/linux-gcc3.2 # needed for 3DVideoRecorder |
---|
80 | module add blue-c/linux # needed for the ClientTester |
---|
81 | |
---|
82 | setenv LD_LIBRARY_PATH $ACE_ROOT/ace/:/home/lamboray/lib/:$LD_LIBRARY_PATH |
---|
83 | setenv LD_LIBRARY_PATH /pub/blue-c_lib/BCL/version_1.0/lib/linux-gcc3.2/:$LD_LIBRARY_PATH |
---|
84 | # additional library I had to add |
---|
85 | setenv LD_LIBRARY_PATH /pub/blue-c/development/pwes/lamboray/3DVideoRecorder/lib/:$LD_LIBRARY_PATH |
---|
86 | setenv LD_LIBRARY_PATH /pub/blue-c_lib/BCL/version_1.0/lib/linux-gcc3.2/old/:$LD_LIBRARY_PATH |
---|
87 | endif |
---|
88 | |
---|
89 | # Set local variabled for the auxiliary scripts |
---|
90 | # machine name |
---|
91 | set machine_basename = `echo $HOST | sed 's/[0-9]//g'` |
---|
92 | setenv BlueC_MNAME "$machine_basename" |
---|
93 | # local directory on the machines where the data is stored |
---|
94 | setenv BlueC_LOCALDIR "/local/tomas" |
---|
95 | # image base name |
---|
96 | setenv BlueC_IMNAME "$machine_basename" |
---|
97 | # basepath for binaries and auxiliary scripts |
---|
98 | setenv BlueC_BASEPATH "/home/svoboda/Work/BlueCAcquire" |
---|
99 | ### |
---|
100 | # full indexes and working machine to process the data |
---|
101 | # need to be set differently for atlantics and arctics |
---|
102 | if ( $machine_basename == "atlantic" ) then |
---|
103 | # setenv BlueC_INDEXES `(seq 3 18)` |
---|
104 | setenv BlueC_INDEXES "3 4 5 6 7 8 9 10 11 12 14 15 16 17 18" |
---|
105 | setenv BlueC_WORKMACHINE "${machine_basename}2" |
---|
106 | else if ($machine_basename == "arctic") then |
---|
107 | setenv BlueC_INDEXES `(seq 1 16)` |
---|
108 | # setenv BlueC_INDEXES "1 2 3 4 5 6 7 8 9 10 11 12 13 15 16" |
---|
109 | setenv BlueC_WORKMACHINE "${machine_basename}19" |
---|
110 | endif |
---|
111 | ############################################################################ |
---|
112 | |
---|
113 | |
---|
114 | The scripts are in Perl or C-shell and are very simple and not very |
---|
115 | robust. They use "convert" and "montage" tools from "ImageMagick" |
---|
116 | package whis a standard part of most Linux distributions. |
---|
117 | |
---|
118 | BlueC acquisition software is mostly written by Edouard Carlo Lamboray, |
---|
119 | lamboray@inf.ethz.ch. |
---|
120 | |
---|
121 | Run the administration script "firewire" written by Stephan Wuermlin, |
---|
122 | wuermlin@inf.ethz.ch, if the 3DVideoRecorder does not work. |
---|
123 | Try: |
---|
124 | >> firewire stop |
---|
125 | >> firewire reload |
---|
126 | >> firewire chmod |
---|
127 | This should help. |
---|
128 | |
---|
129 | Config files "configdata.m" and "expname.m" are in the sub-directory |
---|
130 | Cfg. The configdata contains necessary paths to the data and the |
---|
131 | expname determines the relevant subset of config data. |
---|
132 | |
---|
133 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% |
---|
134 | |
---|
135 | 1) Image acqusition |
---|
136 | |
---|
137 | - Check if the video.cfg is OK. The shuter should be usualy at 590. It |
---|
138 | very depends on the laser. The triggering should MUST be 1! |
---|
139 | |
---|
140 | - Switch off the triggering signal. |
---|
141 | |
---|
142 | - Start "./3DVideoRecorder -u -s -b 0 -n #number_of_images" to capture |
---|
143 | sequences on each machine with camera attached to. The |
---|
144 | 3DVideoRecorder expects the config files in ../Cfg/ |
---|
145 | directory. #number_of_images should be between 300-700. More images |
---|
146 | gives more robustness however, their processing takes accordingly |
---|
147 | longer. The processing time is linear in terms of images. |
---|
148 | |
---|
149 | - Switch the triggering signal on and ... |
---|
150 | |
---|
151 | - Wave laser pointer. Try to fill the whole working volume. Try to |
---|
152 | keep the laser pointer visible to as many cameras as |
---|
153 | possible. Especially is necessary to fill the volume close to the |
---|
154 | cave floor. |
---|
155 | |
---|
156 | - The 3DVideoRecorder stores big *.pvi files in /local/ directory on |
---|
157 | each of the machine then ... |
---|
158 | |
---|
159 | - Run the "collectdata" to extract, transform and collect |
---|
160 | images. Check if the paths are specified correctly. |
---|
161 | |
---|
162 | - You can use "createMontages" script to create 4x4 composed |
---|
163 | images. Check the image quality and the VISIBILITY of points. |
---|
164 | |
---|
165 | - Change the config.files.idxcams accordingly if some cameras are |
---|
166 | missing. |
---|
167 | |
---|
168 | 2) Computing images statitistics and finding laser projections in |
---|
169 | images. |
---|
170 | |
---|
171 | The codes are in the sub-directory FindingPoints |
---|
172 | |
---|
173 | Important note: To run the image processing in parallel we need: be |
---|
174 | able to run "matlab" in each of the machines and have ssh access |
---|
175 | without password (http://www.cs.umd.edu/~arun/misc/ssh.html). |
---|
176 | |
---|
177 | |
---|
178 | Alternatively, you can run the script im2points in one matlab, which |
---|
179 | is also much more certain if you are not sure what you are doing. It |
---|
180 | runs accordingly longer, 16 cameras, 500 images each needs about 30 |
---|
181 | minutes at PIII @ 1GHz (it also depends on the required |
---|
182 | precision). The computation time is linear in terms of cameras and |
---|
183 | images. |
---|
184 | |
---|
185 | |
---|
186 | - Edit configdata.m and put correct paths and all config constants you |
---|
187 | want here. |
---|
188 | |
---|
189 | - Important! Set the correct experiment name into expname.m. |
---|
190 | |
---|
191 | - Run "im2pmultiproc.pl", check if it uses the right *.pm config. This |
---|
192 | perl script will create some temporary files in the working |
---|
193 | directory. Also it needs access rights. |
---|
194 | |
---|
195 | - Warning! If you stop "im2pmultiproc.pl" by Ctrl^C, the matlab |
---|
196 | processes will be still running. They have to be yet killed |
---|
197 | manually. |
---|
198 | |
---|
199 | |
---|
200 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% |
---|
201 | |
---|
202 | 3) Selfcalibration |
---|
203 | |
---|
204 | - Run "gocal" in Matlab. Again, be sure to have correct name of |
---|
205 | the experiment in "expname.m" |
---|
206 | |
---|
207 | - Check the graphical output. Some inside check points are |
---|
208 | applied. Nevertheless, it may happen that the reprojection error is |
---|
209 | small and the results spoiled. The graphical outputs has to be |
---|
210 | checked especially if only few points used for the |
---|
211 | computation. Check the detected points in the graphical windows. The |
---|
212 | points should ideally span the whole area of images. Some "holes" |
---|
213 | may indicate bad camera setting or a bad movement of the calibrator. |
---|
214 | |
---|
215 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% |
---|
216 | |
---|
217 | 4) What to do if the SelfCalibration crashes |
---|
218 | |
---|
219 | - the computation itself is rather robust. However, it may crash if |
---|
220 | some cameras have really strange points. |
---|
221 | |
---|
222 | - Check visually the detected points by typing |
---|
223 | "figure(100), imshow(IdMat.loaded)" |
---|
224 | in the main Matlab command window. The frameIds are on the x-axis, |
---|
225 | the cameraIds on the y-axis. Detected points are white, otherwise |
---|
226 | black. The whiter image the better. Black lines signalize some |
---|
227 | camera problems. |
---|
228 | |
---|
229 | - To check the quality and mainly the reliability of the point |
---|
230 | detection you can use the script "showpoints". It plots graphical |
---|
231 | information to the images. |
---|
232 | |
---|
233 | |
---|
234 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% |
---|
235 | %%% Explanation of the configuration variables |
---|
236 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% |
---|
237 | ../CommonCfgAndIO/configdata.m |
---|
238 | |
---|
239 | - config.paths.data = ['/scratch/WorkingCalib/']; |
---|
240 | |
---|
241 | The main working directory containg all data. Some data may be in |
---|
242 | subdirectories. |
---|
243 | |
---|
244 | |
---|
245 | - config.files.basename = 'arctic'; |
---|
246 | |
---|
247 | Basename for all. This basename will be the main common identifier |
---|
248 | in all exported files. |
---|
249 | |
---|
250 | |
---|
251 | - config.paths.img = [config.paths.data,config.files.basename,'%d/']; |
---|
252 | |
---|
253 | Basename for image sub-directories. |
---|
254 | |
---|
255 | |
---|
256 | - config.files.imnames = [config.files.basename,'%d.pvi.*.']; |
---|
257 | |
---|
258 | Basename for images. |
---|
259 | |
---|
260 | |
---|
261 | - config.files.idxcams = [1:10,12:16]; % related to the imnames |
---|
262 | |
---|
263 | Numbers that index the image sub-directories and names of various |
---|
264 | data files. These indexes must correspond to what is on the disk. |
---|
265 | |
---|
266 | - config.imgs.LEDsize = 7; % avg diameter of a LED in pixels |
---|
267 | |
---|
268 | Parameter used in the finding procedure. When unsure, better to make |
---|
269 | it slightly larger. Rather robust value. "7" works well for both of |
---|
270 | the BlueC installations. |
---|
271 | |
---|
272 | |
---|
273 | - config.imgs.LEDcolor = 'green'; % color of the laser pointer |
---|
274 | |
---|
275 | Used in the finding points. Color of the laser pointer used. |
---|
276 | |
---|
277 | |
---|
278 | - config.imgs.LEDthr = 100; |
---|
279 | |
---|
280 | Optional parameter for the finding points procedure. Default value |
---|
281 | (hidden) is 70. Sometimes, it may help to resolve problems with |
---|
282 | misdetection. The higher value the brighter points accepted as valid |
---|
283 | projections. Useful only in really special cases. |
---|
284 | |
---|
285 | - config.imgs.subpix = 1/5; |
---|
286 | |
---|
287 | Used in the finding points. Required subpixels accuracy. Values |
---|
288 | 1/3-1/5 give quite nice results. Higher values like 1 or 1/2 |
---|
289 | increase the speed of the finding procedure significantly which may |
---|
290 | be useful in some fast try-and-test experiments |
---|
291 | |
---|
292 | |
---|
293 | - config.cal.INL_TOL = 7; |
---|
294 | |
---|
295 | Rather important value. Initial tolerance value for epipolar |
---|
296 | geometry verification. It influences both the pair-wise point |
---|
297 | validation through the epipolar geometry computation and the |
---|
298 | iterative refinement at the end. It should correpond to the expected |
---|
299 | radial distortion in the cameras. This value is iteratively |
---|
300 | decreased during the optimization process. |
---|
301 | |
---|
302 | |
---|
303 | - config.cal.NUM_CAMS_FILL = 10; |
---|
304 | |
---|
305 | How many camera may be filled by "artificial" points. This value |
---|
306 | should depend to expected visibility of the calibration |
---|
307 | points. Higher values are typically needed if the laser pointer is |
---|
308 | not optimally visible in many cameras. Typically, the higher value |
---|
309 | the slower run of the complete procedure. On the other hand, in case |
---|
310 | of bad visibility, the high value may improve the robustness. From |
---|
311 | the principle, this value can be maximally #CAMS-3. If a higher |
---|
312 | value is set, automatic correction is applied. |
---|
313 | |
---|
314 | |
---|
315 | - config.cal.DO_BA = 0; |
---|
316 | |
---|
317 | Do the Bundle Adjustment of the projective reconstruction a the end |
---|
318 | of the all iterations. It is quite slow for many points and |
---|
319 | cameras. It may improve the overall accuracy. Often not need at all. |
---|
320 | |
---|
321 | - config.cal.START_BA = 1; |
---|
322 | |
---|
323 | Optional parameters. When set, it does the Projective Bundle |
---|
324 | Adjustment in each step in the final interation for removing |
---|
325 | outliers. It may improves the performance for bad data sets. The |
---|
326 | whole process it than accordingly slower. |
---|
327 | |
---|
328 | |
---|
329 | - config.cal.UNDO_RADIAL= 1; |
---|
330 | |
---|
331 | Undo the radial distortion by using the paramaters from the CalTech |
---|
332 | camera calibration toolbox? |
---|
333 | |
---|
334 | |
---|
335 | - config.cal.UNDO_HEIKK = 0; |
---|
336 | |
---|
337 | Undo the radial distortion by using the parameters from the Jann |
---|
338 | Heikkila calibration toolbox? |
---|
339 | |
---|
340 | |
---|
341 | - config.cal.NTUPLES = 3; % currently, support for [2-5] implemented |
---|
342 | |
---|
343 | How many cameras are to be used for on sample of the reconstruction? |
---|
344 | It turned out that "3" is optimal for most of the cases. "2" is |
---|
345 | faster however, sometimes less robust. "4-5" more robust but slower. |
---|
346 | |
---|
347 | |
---|
348 | - config.cal.MIN_PTS_VAL = 30; |
---|
349 | |
---|
350 | Used in the MultiCamera validation. How many points must be |
---|
351 | simultaneously visible in config.cal.NTUPLES cameras to do the |
---|
352 | reconstruction step? In fact, not an important value. It might be |
---|
353 | useful if more points are required. This value must not be higher |
---|
354 | than the total number of frames acquired for the particular |
---|
355 | experiment. In practice, it should be below 1/2 if the theoretical |
---|
356 | maximal value. |
---|
357 | |
---|
358 | |
---|
359 | - config.cal.cams2use = [1:10,13:16]; |
---|
360 | |
---|
361 | Which cameras are to be used in the particular |
---|
362 | experiments. Sometimes it is useful not to use all cameras specified |
---|
363 | in config.files.idxcams. If not set, all cameras will be used. |
---|
364 | |
---|
365 | |
---|
366 | - config.cal.nonlinpar = [70,0,1,0,0,0]; |
---|
367 | |
---|
368 | Default initial settings for the estimation of the nonlinear distortion |
---|
369 | (1) ... camera view angle |
---|
370 | (2) ... estimate principal point? |
---|
371 | (3:4) ... estimate parameters of the radial distortion? |
---|
372 | (5:6) ... estimate parameters of the tangential distortion? |
---|
373 | |
---|
374 | It is better to start with the default settings and leave the other |
---|
375 | parameters to be estimated during the global optimization |
---|
376 | |
---|
377 | |
---|
378 | - config.cal.NL_UPDATE = [1,1,1,1,1,1]; |
---|
379 | |
---|
380 | Which nonlinear parameteres would you like to update during the |
---|
381 | global optimization. If you have noisy data with many outliers you |
---|
382 | may want to stabilize the optimization by fixing some |
---|
383 | parameteres. It also depends on what parameters are you actually |
---|
384 | using for undoing distortions. |
---|
385 | |
---|
386 | |
---|
387 | - config.cal.DO_GLOBAL_ITER = 1; |
---|
388 | |
---|
389 | Would you like to perform global optimization? If you already have |
---|
390 | good parameters of the non-linear distortion you may want to disable |
---|
391 | this. |
---|
392 | |
---|
393 | |
---|
394 | - config.cal.GLOBAL_ITER_THR = 0.3; |
---|
395 | |
---|
396 | Rather important value. This is one of the stopping condition for |
---|
397 | the global optimization. The process ends if the maximum of the |
---|
398 | reprojection error (average in each of the cameras) is lower than |
---|
399 | this threshold. Do not be too optimistic. The precision of the |
---|
400 | complete camera model can be hardly better than the precision of the |
---|
401 | finding points. |
---|
402 | |
---|
403 | |
---|
404 | - config.cal.GLOBAL_ITER_MAX = 10; |
---|
405 | |
---|
406 | If the threshold above is set too optimistic, the optimization may |
---|
407 | start to oscilate without actually reaching the desired |
---|
408 | precision. The 10 iteration should be really enough for all |
---|
409 | cases. If not than the data are simply worse than you think. |
---|
410 | |
---|
411 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% |
---|
412 | % Strategy |
---|
413 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% |
---|
414 | Having the images on a local disk speeds up the finiding procedeure |
---|
415 | significantly. |
---|
416 | |
---|
417 | |
---|
418 | You have to proceed the whole optimization process if you calibrate |
---|
419 | the system for the first time or if you change the camera |
---|
420 | linses. More often, however only the camera positions or orientations |
---|
421 | change. You may then re-use the distortion parameters to speed up the |
---|
422 | optimization process. Be sure you have the *.rad files in the working |
---|
423 | directory and run the process with "conig.cal.UNDO_RADIAL=1" and |
---|
424 | "config.cal.nonlinpar = [70,1,1,1,1,1]" config variables. |
---|
425 | |
---|
426 | Set "config.cal.DO_BA=1" if you have enough time and really insist on |
---|
427 | the highest possible precision. Actually, this is mosty not need at |
---|
428 | all. It typically improves the final numbers but not that much the |
---|
429 | real camera models. |
---|
430 | |
---|
431 | Setting "config.cal.START_BA = 1" might help resolve problems with |
---|
432 | really bad data (many outliers, bad sychronization etc.) The |
---|
433 | projective Bundle Adjustment is then performed in each step. It is |
---|
434 | really needed in only very special cases. It serves more as last |
---|
435 | rescue if everything else fails. Do not expect too much :-) |
---|