2001-05-01

Date: May 2, 2001

License: GPL

Back-propagation is the most widely used supervised learning
algorithm for artificial neural networks. A typical
back-propagation network consists of three fully connected
layers: input, hidden, and output, with each processing unit
connected to every unit in the layers above and below. I assume
you are already familiar with the back-propagation algorithm,
and therefore I will not go into further details of it. If you
are not familiar with back-propagation (or neural networks in
general), Chapter 19 of Russell
and Norvig's (1995) artificial intelligence textbook
provides a brief but excellent introduction. You may also want
to read Rumelhart, Hinton, and Williams (1986), which is one of
the chapters in the famous book *Parallel Distributed
Processing* (Rumelhart & McClelland, 1986) that
popularized back-propagation in artificial intelligence,
cognitive science, as well as many other fields.

IML-BKP is an implementation of the standard back-propagation network in SAS/IML (interactive matrix language). All other programs I have made available on the web were written in C. IML-BKP was original written in 1995 to process a huge set of data. I was using SAS to perform statistical analyses on the set of data. Instead of writing a separate C program, I decided to implement the network in IML so I could run it within SAS.

The program is very small, very straightforward, and
commented. If you are familiar with SAS/IML (and
back-propagation) you should have no difficulty understanding
it. I have also implemented the **Encoding
problem** as described in Rumelhart, Hinton, and
Williams (1986; pp. 335-339) as a sample problem for IML-BKP.
In an encoding problem a set of orthogonal input patterns to a
set of orthogonal output patterns through a small set of hidden
units. As a result, the hidden units must be very flexible and
efficient to ensure successful learning.

Copyright (c) 1995-2001 Chih-Hao Tsai.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

Download: iml-bkp.zip

Rumelhart, D. E., & McClelland, J. L. (Eds.). (1986).
*Parallel distributed processing: Explorations in the
microstructure of cognition. Volume 1: Foundations*.
Cambridge, MA: MIT.

Rumelhart, D. E., Hinton, G. E., & Williams, R. J.
(1986). Learning internal representations by error propagation.
In D. E. Rumelhart & J. L. McClelland (Eds.), *Parallel
distributed processing: Explorations in the microstructure of
cognition. Volume 1: Foundations* (pp. 318-364). Cambridge,
MA: MIT.

Russell, S., & Norvig, P. (1995). *Artificial
intelligence: A modern approach*. Upper Saddle River, NJ:
Prentice-Hall.