Computational Biology
@ Comenius University in Bratislava

Martin Pašen, Vladimír Boža. Merging of neural networks. Technical Report 2204.09973, arXiv, 2022.

Download preprint: not available

Download from publisher: https://doi.org/10.48550/arXiv.2204.09973

Related web page: not available

Bibliography entry: BibTeX

Abstract:

We propose a simple scheme for merging two neural networks trained with 
different starting initialization into a single one with the same size as 
the original ones. We do this by carefully selecting channels from each 
input network. Our procedure might be used as a finalization step after one 
tries multiple starting seeds to avoid an unlucky one. We also show that 
training two networks and merging them leads to better performance than 
training a single network for an extended period of time.