1 | % sep96.m implements the learning rule described in Bell \& Sejnowski, Vision |
---|
2 | % Research, in press for 1997, that contained the natural gradient (w'w). |
---|
3 | % |
---|
4 | % Bell & Sejnowski hold the patent for this learning rule. |
---|
5 | % |
---|
6 | % SEP goes once through the mixed signals, x |
---|
7 | % (which is of length M), in batch blocks of size B, adjusting weights, |
---|
8 | % w, at the end of each block. |
---|
9 | % sepout is called every F counts. |
---|
10 | % |
---|
11 | % I suggest a learning rate (lrate) of 0.006, and a blocksize (B) of |
---|
12 | % 300, at least for 2->2 separation. |
---|
13 | % When annealing to the right solution for 10->10, however, lrate of |
---|
14 | % less than 0.0001 and B of 10 were most successful. |
---|
15 | % |
---|
16 | % Copyright 1996 Tony Bell |
---|
17 | % This may be copied for personal or academic use. |
---|
18 | % For commercial use, please contact Tony Bell |
---|
19 | % (tony@salk.edu) for a commercial license. |
---|
20 | |
---|
21 | x=x(:,perm); |
---|
22 | sweep=sweep+1; t=1; |
---|
23 | noblocks=fix(P/B); |
---|
24 | BI=B*Id; |
---|
25 | for t=t:B:t-1+noblocks*B, |
---|
26 | count=count+B; |
---|
27 | u=w*x(:,t:t+B-1); |
---|
28 | w=w+L*(BI+(1-2*(1./(1+exp(-u))))*u')*w; |
---|
29 | if count>F, sepout; count=count-F; end; |
---|
30 | end; |
---|