NNCREAT

Description

Parameters

nncreat(file, dbic, niunit, nhunit, dbib, valu, nsample, dbsn, dbsd)

Parameter descriptions

FILE

The name of the PCIDSK file in which to create the neural-network segment.

DBIC

The input image channel or channels to be used to train the neural network.

At least one channel must be specified. Duplicate channels are not allowed.

NIUNIT

The number of input units to process for each input channel.

It is recommended to use one to three input units per input channel. If no value is specified for this parameter, one input unit is used for each input channel.

If the number of values specified is less than the number of input channels after the range expansion, the last value specifies the number of units for the remaining input channels.

This parameter is optional.

NHUNIT

The number of units for one to five hidden layers.

If no value is specified for this parameter, one hidden layer is created, having the greater of the total number of input units (NIUNIT) or the total number of bitmaps.

It is recommended to use one or two hidden layers, each containing somewhere between the total number of input units and the total number of bitmaps.

DBIB

The input bitmap segments that define the training sites from which to sample data (from the input channels). Each bitmap specifies an output class on which to train the neural network.

Up to 254 bitmap segments can be handled using ranges, or up to 16 bitmap segments may be specified directly.

At least one segment must be specified. Duplicate segments are not allowed.

VALU

The gray-level values of the output class (0 - 255).

Duplicate values cannot be specified. A value of zero cannot be specified, because it is reserved for the null class (by NNCLASS) for pixels that are not in any of the training classes.

If one or more values are specified, the number of values must equal the number of input bitmaps specified. If no value is specified for this parameter, the output classes are assigned sequentially, from one to the number of classes.

Up to 254 values can be handled using ranges, or up to 16 values can be specified directly.

This parameter is optional.

NSAMPLE

The total number of samples to read from all input bitmap segments. If no value is specified for this parameter, NNCREAT processes as many samples as can be read.

If the value specified for this parameter is greater than the total number of ON bits, NNCREAT displays a warning message, and then resets the value of NSAMPLE to the number of ON bits. If the value specified for NSAMPLE is less than the number of bitmaps, NNCREAT displays a warning message, and then resets the value of NSAMPLE to the number of bitmaps.

If the value specified for NSAMPLE is greater than the number of samples that can fit in memory when NNTRAIN runs, the function displays a warning message, and then resets the value of NSAMPLE to the number of samples that fit in memory.

This parameter is optional.

DBSN

The name (up to eight characters) for the output neural-network segment.

DBSD

A description (of up to 64 characters) of the contents or origins of the output neural-network segment.

Return Value

Returns: 

Type:  PCI_INT

When running NNCREAT is complete, the segment number of the new neural-network segment is saved in DBNNS for future use.

Details

NNCREAT creates and initializes a new neural-network segment (type 180, BIN) for back-propagation neural-network processing. This neural-network segment can be trained with NNTRAIN to recognize classes. After the neural network has been trained, it can be used to classify imagery with NNCLASS. With NNREP, a report can be produced on a neural-network segment. For an overview of neural-network processing, see Background later in this topic.

The specified input image file (FILE) contains the input image channels (DBIC) and training site bitmaps (DBIB) that specify the sample data on which to train the neural network. The number of samples to read can be specified or defaulted (NSAMPLE).

The order in which the bitmaps are specified may produce different results. For example, if a neural-network segment is created and trained using one set of bitmaps, and a second segment is created and trained using the same set of bitmaps, but specified in a different order, the classifications produced by the two neural networks will be similar, yet may differ. If the two neural networks are trained enough and have stabilized, the classification results should be very similar.

The dimensions of the neural network are specified with the number of units per input channel (NIUNIT) for the input layer, the number of units per hidden layer (NHUNIT), and the number of bitmaps for the output layer. The NIUNIT and NHUNIT parameters can be defaulted.

The output values encoded for each class can be specified or defaulted (VALU).

The output neural-network segment is appended to the end of FILE. Its number is returned in the DBNNS parameter.

The name for the output neural-network segment (DBSN) and a description (DBSD) can be specified.

Background

The neural-network programs NNCREAT, NNTRAIN, and NNCLASS produce multispectral image classification based on training sites. The neural-network classifier provides an alternative to using the maximum likelihood classifier (MLC) program and other classifiers from the Multispectral Analysis component. Neural-network classifiers process imagery by pattern recognition. NNCREAT, NNTRAIN, and NNCLASS use a back-propagation network that learns using the Generalized Delta Rule.

Neural networks are computational systems, either hardware or software, that mimic the computational abilities of biological systems by using simple, interconnected artificial neurons. A neuron is the fundamental building block of the nervous system. Artificial neurons are simple emulations of biological neurons: they take input information, perform very simple operations on it, and then selectively pass the results to other artificial neurons.

The back-propagation network (or back-propagating Perceptron) is probably the most well-known and widely used of neural-network systems. The term "back-propagation" refers to the training method by which the connection weights of the network are adjusted. The back-propagation network is a type of multilayer feed-forward network. Other types of feed-forward networks are the Perceptron, the ADALINE and MADALINE networks, the Boltzmann machine, and the Cauchy Machine.

For more information on back-propagation neural networks, see one or more of the books and papers cited in References later in this topic. In particular, a general discussion of back-propagation neural networks can be found in section 7.4 (page 94) of the book "Handbook of Neural Computing Applications".

Example

The following neural-network segment is used in the Example section of each corresponding Help topic for NNTRAIN, NNCLASS, and NNREP.

The input data is five channels of Landsat TM data (8-bit unsigned). Since NHUNIT is defaulted, one hidden layer is created with the maximum number of units for the input and output layers (that is, eight units). VALU is set with the same values used for the classes in the MLC example.

from pci.nncreat import nncreat

file	=	"irvine.pix"
dbic	=	list(range(1, 5 + 1))
niunit	=	[]	# default, one unit per channel
nhunit	=	[]	# default, eight units in one hidden layer
dbib	=	list(range(9, 16 + 1))
valu	=	[10,20,30,40,50,60,70,80]
nsample	=	[]	# default, maximum number of samples
dbsn	=	""	# default, "neural"
dbsd	=	""	# default, blank line

dbnns = nncreat( file, dbic, niunit, nhunit, dbib, valu, nsample, dbsn, dbsd )

  Input Layer              O   O   O   O   O

  Hidden Layer           O  O  O  O  O  O  O  O

  Output Layer           O  O  O  O  O  O  O  O

Algorithm

A neural network consists of interconnected processing elements, known as "units" ("nodes" or "neurons"). Units are organized in two or more layers. An input layer of units is activated by the input image data. The output layer of units represents the output classes for which to train. In between, there is usually one or more hidden layers of units (units that are neither input units nor output units).

The following diagram illustrates the layers of units and their order in a neural network:

 1. Input Layer              O   O   O   O   O

 2. Hidden Layer           O  O  O  O  O  O  O  O

 3. Output Layer           O  O  O  O  O  O  O  O

where "O" represents the units for the layer. Each layer unit has a link to each unit in the next layer.

The input image pixel values are scaled linearly to a value between 0.0 and 1.0 for input to the neural network with the image channel minimum and maximum. Other types of scaling can be obtained by scaling the image channel with ARI or SCALE to a range of 0.0 to 1.0, and then saving the result in a 32-bit real-image channel.

If there is more than one input unit allocated for an image channel, linear scaling is performed with as many subranges as units allocated. For example, if two units are used for an image channel with pixel values in the range of 0 to 20, the first unit would be activated for image pixels in the range of 0 to 10, and the second unit would be activated for image pixels in the range of 11 to 20.

A pixel value of 7 is represented as 0.7 for Unit 1; for example:

(7 - 0)/(10 - 0))

and 0.0 for Unit 2; a data value of 13 would be represented as 0.0 for Unit 1 and 0.3 for Unit 2

(13 - 10)/(20 - 10)).

A unit in one layer is connected to all units in the next layer. A unit in a hidden or output layer receives input from all units in the previous layer and produces one output value. Each link from a unit to the next layer's units has a weight, which suppresses or allows the output value from the unit. To calculate the output from a hidden layer unit or output layer unit (output_j), the net input (net_i) to that unit is calculated first:

net_i = sum of (weight_i * output_i)

After the net input to the unit is added up, this value is put through an activation function (in this case the sigmoid function) to produce the unit's output:

output_j = 1.0 / (1.0 + exp( -(net_i) )

The sigmoid function has a range of one (when net_i goes to infinity) to zero (when net_i goes to negative infinity).

This is what is done going forward through the network (from the input layer through to the output layer). After the network has been trained, NNCLASS classifies the imagery by first using the input image data to activate the input layer, and then moving forward through the network, using the activation of the output layer units to produce the output imagery.

The second phase in training is a backward pass through the network to reduce the error between the actual and the desired output. This involves determining the errors, and then calculating and adding weight adjustments to the neural-network weights to reduce the number of errors.

The errors reported are calculated as follows. The individual error is the sum of the errors of the output layer units. It is this value that is tested against MAXIERR by NNTRAIN.

individual error = sum of ( abs( target - output ) )

Normalized total error is the value tested against MAXTERR by NNTRAIN and displayed in the error plot.

normalized total error = 0.5*(sum of(individual error**2))/NSAMPLE

References

The following papers and books provide more information on the back-propagation neural-network algorithm used by NNCREAT, NNTRAIN, and NNCLASS:

Benediktsson, J.A., P.H. Swain, and O.K. Ersoy (1990). "Neural Network Approaches Versus Statistical Methods in Classification of Multisource Remote Sensing Data", IEEE Transactions on Geoscience and Remote Sensing, Vol. 28, No. 4, July 1990.

Bischof, H., W. Schneider, and A.J. Pinz (1992). "Multispectral Classification of Landsat-Images Using Neural Networks", IEEE Transactions on Geoscience and Remote Sensing, Vol. 30, No. 3, May 1992.

Hertz, J.A., A.S. Krogh, and R.G. Palmer (1991). "Introduction to the Theory of Neural Computation", Addison-Wesley Publishing Company.

Maren, A.J., C.T. Harston, and P.M. Pap (1990). "Handbook of Neural Computing Applications", Academic Press Inc., San Diego.

Pao, Y.H. (1989). "Adaptive Pattern Recognition & Neural Networks", Addison-Wesley Publishing Company.