Split Activation Networks for Neural Fields

Abstract

Neural field modeling is a developing area that improves state-of-the-art results in tasks such as 3D scene reconstruction, image manipulation, generative modeling, and other aspects of deep learning. In this work, we present SplitNet, a novel neural network architecture for neural field modeling that combines multiple activation functions in a single layer. We try different techniques to improve performance, such as proper weight initialization, and benchmark its performance on image representation, 3D scene reconstruction, and image classification tasks. As a part of the work, we found a way to improve the performance of previous work on implicit neural networks with sinusoidal activations in a limited setting and study how well this improvement generalizes to other tasks and data.