Artificial Neural Networks have traditionally been treated as black boxes, both in their development and in their use. We present a method to discover the internal structure of neural networks by visualizing activation properties of the network with respect to input data: co-activations of multiple neurons. Our method combines statistical analysis techniques with a modified Sankey Diagram to show flow of data through neural networks unlike previous visualizations methods. Implications for this technique beyond behavioral and structural visualization include the optimization of an artificial neural network through parameter reduction and further understanding of their processing.
While our visualization method runs in the browser, it depends on proper data collection. scripts/sankey.js does most of the heavily lifting of actual rendering, depeding on json data representing the nodes and links, along with some meta data.
The easiest way to generate this data is to run "nodejs tsne_xor.js". This script generates three neural networks (the XOR networks above), trians them, collects the activation data, processes it with tSNE, and generates the data structure needed by the rendering script in the browser. Other network data required external software support; the collected activation data has been included in the repository, but the neural network software was not.