Artificial Neural Networks Also Need Rest To Prevent Catastrophic Forgetting, Improve Generalization

Artificial neural networks are like human brains as they work similarly and require the same thing - rest. Without rest, they are prone to glitches.

Artificial Neural Networks Need a Break Too

Machine brains can work non-stop to learn new tasks day and night. However, they also tend to forget what they already know when introducing a new task.

The apparent network overload led Maxim Bazhenov, a professor who studies computational neuroscience and sleep at the University of California San Diego School of Medicine that the spiking neural networks - which mimic the pattern of neurons firing in the brains - probably just needed a rest, Nautilus reported.

He observed the same basic processes among humans and honeybees working over information accumulated during waking moments. He thought the machinery was doing something worthwhile.

He thought of giving artificial neural networks a break from external stimuli to instruct them to go into a rest state. The networks remained active, but they mulled the old stuff instead of taking on new information.

They noticed that when neural networks are in a sleep-like state, they mitigate catastrophic forgetting and improve generalization. From cyber security to self-driving cars (which must remember and wisely apply the rules of the road and those of Asimov), both have consequences for these networks' genuine utility.

The downtime appears to reinforce new task-related connections made in the brain while awake and assist its transmission to parts of the brain for longer-term storage.

ALSO READ: Lights Off, Drawn Curtains and Closed Blinds: Sleep Expert Says These Will Help Achieve Better Health

Sleep, of course, is equally crucial for our memory and learning. Lack of sleep affects our capacity to learn new abilities and store memories, which has been known to researchers for decades.

A recent study reveals that we don't need to shut down to increase our procedural memory completely. Our brains benefit from simply relaxing without seeking new inputs, or as the researchers put it, "offline memory consolidation."

Yet Robert Stickgold urges caution in this case. Stickgold, a professor of psychiatry at the Brain Science Institute at Harvard Medical School's Brain Science Initiative, researches sleep and cognition. According to him, it's convenient to state that we're letting a network "sleep," but it's best not to overuse the term for our sake.

Benefits of Sleep to the Human Brain

According to the National Institute of Neurological Disorders and Stroke, sleep is essential in one's daily routine. We spend one-third of our time sleeping.

Quality sleep is as essential as food and water for one's survival. Without adequate sleep, it will be difficult for our brain to maintain pathways in the brain that lets us learn new things and create new memories. It's also harder to focus and react quickly.

Sleep is important to our brain functions, including how nerve cells communicate with each other. In fact, the brain and body remain active even when we are asleep.

A study also suggests that sleep removes toxins in the brain that build up while awake. Chronic lack of sleep or poor quality sleep will increase one's risk of high blood pressure, cardiovascular disease, diabetes, depression, and obesity.