Iranian Journal of Biomedical Engineering (IJBME)
نشریه علمی مهندسی پزشکی زیستی

Weight Synchronization in Feedforward Neural Networks and a Novel Method to Detect Synchrony Patterns

Document Type : Full Research Paper

Authors

Hossein Banki-Koshki 1
S. Ali Seyyedsalehi 2

1 Ph.D. Student, Bioelectric Department, Biomedical Engineering Faculty, Amirkabir University of Technology, Tehran, Iran

2 Associate Professor, Bioelectric Department, Biomedical Engineering Faculty, Amirkabir University of Technology, Tehran, Iran

https://doi.org/10.22041/ijbme.2023.2011702.1861
Abstract
Neuronal synchronization as a significant cognitive phenomenon of the human brain, has attracted the interest of neuroscience researchers in recent years. This phenomenon is generally investigated in discrete and continuous neuronal models or experimentally recorded signals of the brain. In this study, for the first time, we investigate the weight synchronization instead of neuronal synchrony, in the training step of the artificial feedforward neural networks. The findings of the study show that the generalized weight synchronization occurs both during the training mode and in the non-training mode. Furthermore, as the training is completed, the synchronization increases between the weights. In this study, a new method is introduced in order to detect synchrony patterns using signal derivative and hierarchical clustering. We have also presented a criterion to quantify weight synchronization in different layers of the neural network. Accordingly, the results demonstrate that the lower layers of the network have a significantly higher level of weight synchrony than the upper layers.

Keywords

Neuronal Synchronization
Weight Synchronization
Synaptic Plasticity
Artificial Neural Network
Neuronal Map
Cognition
Modeling
Chaos

Subjects

  1. Shafiei M, Jafari S, Parastesh F, Ozer M, Kapitaniak T, Perc M. Time delayed chemical synapses and synchronization in multilayer neuronal networks with ephaptic inter-layer coupling. Commun Nonlinear Sci Numer Simul 2020;84:105175.
  2. Timofeev I, Bazhenov M, Seigneur J, Sejnowski T. Neuronal synchronization and thalamocortical rhythms in sleep, wake and epilepsy. Jasper’s Basic Mech Epilepsies [Internet] 4th Ed 2012.
  3. Arenas A, Díaz-Guilera A, Kurths J, Moreno Y, Zhou C. Synchronization in complex networks. Phys Rep 2008;469:93–153.
  4. Kazemi S, Jamali Y. Phase synchronization and measure of criticality in a network of neural mass models. Sci Rep 2022;12:1319.
  5. Etémé AS, Tabi CB, Mohamadou A. Firing and synchronization modes in neural network under magnetic stimulation. Commun Nonlinear Sci Numer Simul 2019;72:432–40.
  6. Rakshit S, Ray A, Bera BK, Ghosh D. Synchronization and firing patterns of coupled Rulkov neuronal map. Nonlinear Dyn 2018;94:785–805.
  7. Sun H, Cao H. Synchronization of two identical and non-identical Rulkov models. Commun Nonlinear Sci Numer Simul 2016;40:15–27.
  8. Fell J, Axmacher N. The role of phase synchronization in memory processes. Nat Rev Neurosci 2011;12:105–18.
  9. Ahn S, Park C, Rubchinsky LL. Neural synchronization in parkinson’s disease on different time scales. Multiscale Model Brain Disord 2019:57–65.
  10. Arnulfo G, Wang SH, Myrov V, Toselli B, Hirvonen J, Fato MM, et al. Long-range phase synchronization of high-frequency oscillations in human cortex. Nat Commun 2020;11:5363.
  11. Amor F, Baillet S, Navarro V, Adam C, Martinerie J, Le Van Quyen M. Cortical local and long-range synchronization interplay in human absence seizure initiation. Neuroimage 2009;45:950–62.
  12. Bosman CA, Womelsdorf T, Desimone R, Fries P. A microsaccadic rhythm modulates gamma-band synchronization and behavior. J Neurosci 2009;29:9471–80.
  13. Womelsdorf T, Fries P, Mitra PP, Desimone R. Gamma-band synchronization in visual cortex predicts speed of change detection. Nature 2006;439:733–6.
  14. Börgers C, Kopell N. Synchronization in networks of excitatory and inhibitory neurons with sparse, random connectivity. Neural Comput 2003;15:509–38.
  15. Mahmoud GM, Mahmoud EE. Phase and antiphase synchronization of two identical hyperchaotic complex nonlinear systems. Nonlinear Dyn 2010;61:141–52.
  16. Koike T, Tanabe HC, Okazaki S, Nakagawa E, Sasaki AT, Shimada K, et al. Neural substrates of shared attention as social memory: a hyperscanning functional magnetic resonance imaging study. Neuroimage 2016;125:401–12.
  17. Malik SA, Mir AH. Synchronization of hindmarsh rose neurons. Neural Networks 2020;123:372–80.
  18. Wang J, Lu M, Li H. Synchronization of coupled equations of Morris–Lecar model. Commun Nonlinear Sci Numer Simul 2008;13:1169–79.
  19. Vivekanandhan G, Hamarash II, Ali Ali AM, He S, Sun K. Firing patterns of Izhikevich neuron model under electric field and its synchronization patterns. Eur Phys J Spec Top 2022;231:4017–23.
  20. Basheer IA, Hajmeer M. Artificial neural networks: fundamentals, computing, design, and application. J Microbiol Methods 2000;43:3–31.
  21. Dreyfus G. Neural networks: methodology and applications. Springer Science & Business Media; 2005.
  22. Citri A, Malenka RC. Synaptic plasticity: multiple forms, functions, and mechanisms. Neuropsychopharmacology 2008;33:18–41.
  23. Kennedy MB. Synaptic signaling in learning and memory. Cold Spring Harb Perspect Biol 2016;8:a016824.
  24. Abraham WC, Jones OD, Glanzman DL. Is plasticity of synapses the mechanism of long-term memory storage? NPJ Science of Learning 4: 9 2019.
  25. Ma S, Zuo Y. Synaptic modifications in learning and memory–a dendritic spine story. Semin. Cell Dev. Biol., vol. 125, Elsevier; 2022, p. 84–90.
  26. Quiroga RQ, Kraskov A, Kreuz T, Grassberger P. Performance of different synchronization measures in real data: a case study on electroencephalographic signals. Phys Rev E 2002;65:41903.
  27. Soriano MC, Van der Sande G, Fischer I, Mirasso CR. Synchronization in simple network motifs with negligible correlation and mutual information measures. Phys Rev Lett 2012;108:134101.
  28. Zhang Q, Lü J, Chen S. Coexistence of anti-phase and complete synchronization in the generalized Lorenz system. Commun Nonlinear Sci Numer Simul 2010;15:3067–72.
  29. Murtagh F, Contreras P. Algorithms for hierarchical clustering: an overview, II. Wiley Interdiscip Rev Data Min Knowl Discov 2017;7:e1219.
  30. Roodschild M, Gotay Sardiñas J, Will A. A new approach for the vanishing gradient problem on sigmoid activation. Prog Artif Intell 2020;9:351–60.
Iranian Journal of Biomedical Engineering (IJBME)
Volume 17, Issue 2
Summer 2023
Pages 97-108

  • Receive Date 16 September 2023
  • Revise Date 27 October 2023
  • Accept Date 02 November 2023

Home

Contact Us