During my second quarter as a computer science grad I took a course on neural networks. As mentioned in a previous post, I had previously taken a course in pattern recognition. Since I already knew quite a bit about neural nets I decided to research the topic of NeuroEvolution of Augmenting Topologies (NEAT) instead of work on the scheduled class project (comparing the performance of feed-forward, recurrent, and radial basis function neural nets). NEAT is a genetic algorithm for evolving artificial neural networks that protects innovation by dividing network types into species based on similar characteristics. During crossover new networks inherit nodes and node connections from their parent networks to form a new network structure. Node genes and connection genes are labeled with innovation numbers so that as those genes can be tracked as they are passed on to future generations. After crossover new networks undergo mutation where new nodes or new connections are added to the network and connections weights are adjusted by a random process.
Unlike traditional learning where connection weights are adjusted via a backpropagation technique, NEAT selects members from the spawned population, called genomes, for breeding. The evolved members are composed of nodes with sigmoidal activation functions that behave in the same way as traditional multi-layer perceptron networks.
Using jNEAT, a Java implementation of NEAT, I attempted to solve 2 different classification problems. The first problem was to classify 150 iris samples as 1 of 3 breeds based on 4 features. The second problem was to classify 178 wine samples as originating from 1 of 3 batches of wine based on 13 features. After conducting a number of different experiments using various configurations settings, I was able to generate a network with 96% correct classification for the iris test dataset and a network with 85% correct classification for the wine test dataset.
About a year later I went on to apply the concept of speciation to a genetic algorithm in my masters thesis.
Resources:
