Wing Sheung Chan

Appendix A. Introduction to neural network classification This appendix provides a minimal introduction to neural network (NN) classification, with the aim of providing the readers with the necessary vocabulary. Generally speaking, an artificial neural network (or simply called neural network) is any simulated collection of connected units known as (artificial) neurons, or nodes, where each neuron outputs a certain signal when given a set of input signals. Inputs to a neuron can either be an external signal, or an output signal from another neuron in the network. Similarly, the output from a neuron can either be an input to another neuron, or an external output. A neural network is therefore ultimately a mapping from a set of external input signals to a set of external output signals. Such a mapping is determined by the way each individual neuron responds to its inputs and the way that the neurons are connected to each other. The mapping can be highly non-linear and, with enough neurons in a network, can approximate any continuous functions to an arbitrary accuracy [135] . In other words, NNs are universal approximators. The function that defines the response of a neuron when given an input or a set of inputs is called the activation of the neuron. Common activations for neurons with a single input ( x ) include the standard logistic sigmoid function f ( x ) = 1 1 + e − x , (A.1) and the rectified linear unit (ReLU) function f ( x ) = ( 0 if x < 0 , x if x ≥ 0 . (A.2) For neurons with multiple inputs ( x ), a linear combination ( L ) of the inputs can be used instead as the input to the above functions: x = L ( x ) = X x i ∈ x w i x i + b, (A.3) 125