%---------------------------------------------------------
% 
% Copyright 1997 Marian Stewart Bartlett
% This may be copied for personal or academic use.
% For commercial use, please contact Marian Bartlett 
% (marni@salk.edu) for a commercial license.
%
% Image representations by Marian Bartlett. Revised 7/14/03
% ICA code by Tony Bell.
% The ICA method is patented by Bell & Sejnowski, at the Salk Institute. 
%
% Please cite Bartlett, M.S. (2001) Face Image Analysis by 
% Unsupervised Learning. Boston: Kluwer Academic Publishers. 
%
% --------------------------------------------------------


Based on Bartlett, Movellan, & Sejnowski (2002). Face Recognition by
Independent Component analysis. IEEE Transactions on Neural Networks
13(6) p. 1450-1464, and

Bartlett, M.S. (2001) Face Image Analysis by Unsupervised
Learning. Boston: Kluwer Academic Publishers.

This directory contains 2 matlab scripts for finding the ICA representation 
of a set of images for recognition:

1. Arch1.m: Gets representation of train and test images under architecture I
2. Arch2.m: Gets representation of train and test images under architecture II

Read through the comments of these scripts before attempting to run them. 

The above scripts call the following 6 MATLAB files for running infomax ica.
Written by Tony Bell http://www.cnl.salk.edu/~tony/

1. runica.m, the ica training script which calls the functions below.
2. sep96.m, the code for one learning pass thru the data
3. sepout.m, for optional text output
4. wchange.m, tracks size and direction of weight changes 
5. spherex.m, spheres the training matrix x.
6. zeroMn.m: Returns a zero-mean form of the matrix X, where each row has 
             zero-mean. (This one was added by Marian Bartlett) 

The following variables are used to calculate ica:

sweep:    how many times you've gone thru the data
P:        how many timepoints in the data
N:        how many input (mixed) sources there are
M:        how many outputs you have
L:        learning rate
B:        batch-block size (ie: how many presentations per weight update.)
t:        time index of data
sources:  NxP matrix of the N sources you read in
x:        NxP matrix of mixtures
u:        MxP matrix of hopefully unmixed sources
a:        NxN mixing matrix
w:        MxN unmixing matrix (actually w*wz is the full unmixing matrix
          in this case)
wz:       zero-phase whitening: a matrix used to remove 
          correlations from between the mixtures x. Useful as a 
          preprocessing step.
noblocks: how many blocks in a sweep;
oldw:     value of w before the last sweep
delta:    w-oldw
olddelta: value of delta before the last sweep
angle:    angle in degrees between delta and olddelta
change:   squared length of delta vector 
Id:       an identity matrix
permute:  a vector of length P used to scramble the time order of the
          sources for stationarity during learning.

INITIAL w ADVICE: identity matrix is a good choice, since, for prewhitened
data, there will be no distracting initial correlations, and the output
variances will be nicely scaled so <uu^T>=4I, right size to fit the 
logistic fn (more or less).

LEARNING RATE ADVICE: 
N=2: L=0.01 works
N=8-10: L=0.001 is stable. Run this till the 'change' report settles
down, then anneal a little. L=0.0005,0.0002,0.0001 etc, a few passes
(= a few 10,000's of data vectors) each pass.
N>100: L=0.001 works well on sphered image data.