Activation functions and weights are intrinsically related to Neural Networks.

We have a variety of activation functions such as ReLU, Tanh, etc.

src:- https://images.app.goo.gl/6XpjYSLSRyKQ9vTk8

These activation functions are used to scale down sequential values to a fixed range, and sometimes to eliminate those parameters which do not meet the required Threshold.

Do you know that we living beings also have activation functions built into our neurons? Didn't understand? lemme explain.

So, we all are familiar with the fact that neurons pass-on information in the form of electrical signals. That is how we sense touch, pain, temperature, taste, etc. Now, imagine yourself sitting close facing a tabletop fan. What do you feel? Gusts of winds blowing towards you. Now, consider that you walk away from it to a calm place. Do you still feel air moving around you? No. Does that mean the air around you is still? not at all, air is dynamically moving and there is no such thing as still air. If that is the case, why can't you feel it?

Now that you are pondering this, let me tell you my view on this. Imagine that you could feel the seemingly still air even in a calm place. This would mean that you would be feeling the air around you in almost every part of your body. Doesn't it seem like a waste of energy to process all this information in the brain? Now let us add the feel of your clothes, the feel of every surface in contact with you. All of this would be generating a lot of processing burden on the brain. Doesn't it seem a waste of brainpower?

The reason this is not the case is that we have something called activation potential in our neurons, which prohibits any signals below a voltage of ~ -50mV from passing them. This would eliminate all the weak signals from reaching the brain, and only the ones that supposedly matter will. Neurons have very high resistance and very high capacitance, analogous to RC receivers. So, even signals just above the threshold will eventually weaken and dissipate before moving to the next neuron.

src:- https://images.app.goo.gl/rPumt7NMV6Czf2sFA

This also explains reflex arcs. Reflex arcs are actions that are directed by our spine rather than our brain. Imagine you lay your finger on something sharp, or touch something very hot, in those cases, time is of the essence because prolonged contact may cause damage. So, before the signals reach the brain, the spine itself will direct motor neurons in your hands to move away. What do you infer from this??

src :- https://images.app.goo.gl/BKXGZo7ozatjrYkx5

Yes, you are right. Certain signals of high voltage levels such as sensing a hot or sharp object can activate the motor neurons originating from the spine and enable us to reflex in such conditions. Sounds fair?

What do you think about this? drop your opinions in the comments and feel free to point out any scope of improvement in my article.

Happy Learning!!!!

references for further reading:-

https://faculty.washington.edu/chudler/ap.html

https://www.sciencedirect.com/topics/neuroscience/reflex-arc#:~:text=The%20reflex%20arc%20is%20a,e.g.%2C%20a%20skeletal%20muscle).