[94] | 1 | % sep96.m implements the learning rule described in Bell \& Sejnowski, Vision |
---|
| 2 | % Research, in press for 1997, that contained the natural gradient (w'w). |
---|
| 3 | % |
---|
| 4 | % Bell & Sejnowski hold the patent for this learning rule. |
---|
| 5 | % |
---|
| 6 | % SEP goes once through the mixed signals, x |
---|
| 7 | % (which is of length M), in batch blocks of size B, adjusting weights, |
---|
| 8 | % w, at the end of each block. |
---|
| 9 | % sepout is called every F counts. |
---|
| 10 | % |
---|
| 11 | % I suggest a learning rate (lrate) of 0.006, and a blocksize (B) of |
---|
| 12 | % 300, at least for 2->2 separation. |
---|
| 13 | % When annealing to the right solution for 10->10, however, lrate of |
---|
| 14 | % less than 0.0001 and B of 10 were most successful. |
---|
| 15 | % |
---|
| 16 | % Copyright 1996 Tony Bell |
---|
| 17 | % This may be copied for personal or academic use. |
---|
| 18 | % For commercial use, please contact Tony Bell |
---|
| 19 | % (tony@salk.edu) for a commercial license. |
---|
| 20 | |
---|
| 21 | x=x(:,perm); |
---|
| 22 | sweep=sweep+1; t=1; |
---|
| 23 | noblocks=fix(P/B); |
---|
| 24 | BI=B*Id; |
---|
| 25 | for t=t:B:t-1+noblocks*B, |
---|
| 26 | count=count+B; |
---|
| 27 | u=w*x(:,t:t+B-1); |
---|
| 28 | w=w+L*(BI+(1-2*(1./(1+exp(-u))))*u')*w; |
---|
| 29 | if count>F, sepout; count=count-F; end; |
---|
| 30 | end; |
---|