Unit 6: Understanding Neural Networks

A neural network is inspired by interconnected neurons in the human brain. Its artificial neurons are linked, and the weights of these connections are adjusted during learning to recognize patterns. CPU instruction cycles, database relationships, and internet routing are different computing concepts, not the primary biological inspiration.

The input layer receives data, hidden layers transform it and learn useful features or patterns, and the output layer produces a prediction or classification. Thus, the layers process information through successive stages. They do not merely provide permanent storage, control hardware, or automatically remove errors from the data.

Human supervision allows people to review AI outputs and identify errors, bias, or unacceptable risks. They can then correct the system, limit its use, request further review, or reject an unsafe result. The other options remove oversight or weaken privacy and safety.

A weight is a numerical value that determines how strongly an input contributes to a neuron's calculation. During training, the network adjusts these values to improve its outputs. A weight does not count inputs, specify an image colour, or name a training file.

In a neural network, a bias is a learnable constant added to the weighted sum before activation: z = w₁x₁ + w₂x₂ + b. Changing b shifts the value at which a neuron activates, so the model can move its decision boundary and fit data more flexibly. Choosing an activation function, initializing weights, and setting the learning rate are separate tasks; therefore option A is correct.

An activation function transforms the weighted input of a neuron and introduces nonlinearity. Without it, several stacked layers would behave like one linear transformation and could not represent many complex patterns. Functions such as ReLU, sigmoid, and tanh serve this role. Normalization prepares data, backpropagation updates weights, and a loss function measures error, so option A is correct.

The loss value measures how far the model’s predictions are from the expected or actual values. During training, optimization usually aims to reduce this value, although the exact loss measure depends on the task. The number of layers describes the model’s architecture, not its prediction error.

The learning rate controls the size of each parameter update. If it is too high, updates may overshoot a minimum-loss point, causing oscillation, divergence, or unstable training. A high learning rate does not guarantee perfect accuracy, and it does not remove the need for data.

With a very low learning rate, parameter updates are extremely small. The model may therefore need many more iterations to reduce the loss and approach a suitable solution, making training slow. A low learning rate does not eliminate the need for data or make errors disappear automatically.

During training, a model compares its predictions with the actual target values. A loss function measures the difference between them, and an optimization algorithm changes the model parameters to reduce that loss. The learning rate controls the step size of these changes, so it is not usually the main quantity being minimized.

Hidden layers are positioned between the input and output layers. Their neurons combine inputs through weights, apply activation functions, and transform representations step by step. With training, these layers learn useful features and increasingly complex patterns. Hence A is correct. The input layer receives raw data, the output layer generally gives the final prediction, and weights are not a permanent data store.

Option A is correct. Deep learning generally refers to machine-learning methods that use neural networks with multiple processing layers. These layers can learn increasingly complex representations from data. Single-layer networks, manually written rules, and fixed non-learning algorithms represent different approaches.

Forward propagation is the computation phase in which an input moves from the input layer through successive hidden layers to the output layer. Each layer performs weighted sums and usually applies an activation function, producing the next layer’s input and finally a prediction. Thus A is correct. Sending gradients backward is backpropagation, not forward propagation.

Backpropagation uses the chain rule to calculate how the loss changes with respect to each trainable weight. It sends these gradients backward through the network, after which an optimizer such as gradient descent updates the weights to reduce prediction error. Therefore A is correct. Resizing images, deleting data, and adding layers are separate operations, not the purpose of backpropagation.

The input layer is the network’s entry point: it receives feature values from external data and makes them available to the following computational layers. For example, pixels may represent an image and measurements may represent a table. Thus option A is correct. The output layer produces the final prediction; setting weights to zero or stopping training is not the input layer’s role.

The output layer is the final layer of a neural network. It uses information processed by earlier layers to produce the final prediction or result. For example, in a classification task, it may produce class probabilities or a predicted class. Extracting intermediate features is generally the role of hidden layers, so option C is not correct.

An activation function introduces nonlinearity into a neuron's output. This allows a neural network to learn complex patterns instead of being limited to simple linear relationships. Deleting data, shutting down the network, or hiding images is not the role of an activation function. Exam tip: associate activation functions primarily with nonlinearity and learning complex patterns.

During training, the network's prediction is compared with the correct target to calculate loss. Weights and biases are updated using this loss so that error decreases and the model makes better predictions. Option C is incorrect because the number of input features is normally determined by the data or network structure, not by updating weights and biases.

A loss function measures the error between a model's prediction and the true target value as a numerical quantity. During training, an optimization method changes the model parameters to reduce this loss. Option B concerns data creation, whereas a loss function evaluates prediction error.

The learning rate determines the size of the change made to the model’s weights at each update. A learning rate that is too high can overshoot the minimum-error point and make training unstable, while one that is too low makes training very slow. Therefore, it must balance learning speed and stability. Option B concerns only changing a name and has no role in model optimization.