What are the parameters within nodes that allow extrapolation in the hidden layers?

What are the parameters within nodes that allow extrapolation in the hidden layers?

• A. Bias terms
• B. Weights
• C. Activation functions
• D. Thresholds

Answer: (B)

Weights are the parameters that control how much influence each input feature has in a neuron. Each neuron calculates a weighted sum of its inputs, plus a bias, and then applies an activation function. The values of these weights are learned during training, shaping how information is transformed as it moves through hidden layers. By adjusting the weights, the network forms increasingly rich representations, enabling it to make inferences about inputs it hasn’t seen before—this is how extrapolation is enabled in deep learning. Bias terms also play a role by shifting the activation, which helps determine when a neuron should fire, but they don’t by themselves establish the relative influence of inputs. Activation functions introduce nonlinearity, which is essential for capturing complex relationships, but the learnable adjustments that drive how inputs contribute come from the weights. Threshold concepts are more aligned with older, simpler models, whereas in modern networks the learned weights are the primary parameters that enable these transformations.

Weights are the parameters that control how much influence each input feature has in a neuron. Each neuron calculates a weighted sum of its inputs, plus a bias, and then applies an activation function. The values of these weights are learned during training, shaping how information is transformed as it moves through hidden layers. By adjusting the weights, the network forms increasingly rich representations, enabling it to make inferences about inputs it hasn’t seen before—this is how extrapolation is enabled in deep learning.

Bias terms also play a role by shifting the activation, which helps determine when a neuron should fire, but they don’t by themselves establish the relative influence of inputs. Activation functions introduce nonlinearity, which is essential for capturing complex relationships, but the learnable adjustments that drive how inputs contribute come from the weights. Threshold concepts are more aligned with older, simpler models, whereas in modern networks the learned weights are the primary parameters that enable these transformations.