More than the sum of its parts?
The implementation of "ensembles", "committee" and/or "team" solutions are frequently discussed in machine learning circles. The terms used merely refer to the act of taking a mean average from several AI generated predictions, and using this as the favourable option. There is proof that the resulting output will not just give a "mushy" middle-ground prediction (as intuition might suggest), but will in fact frequently give a more accurate result than any of the individual sources used, and will never be worse than the least accurate.
Being an individual who is all for the easy life, I always shunned the idea as increasing my data processing workload. Do I need to make life more complicated for the possibility little extra accuracy?
But a recent interesting observation on another newsgroup highlighted the fact that in a recent data mining competition (where machine learning solutions go head-to-head for top prize) combining the predictions of the top performers gave a better result than any one of the individuals. In many ways then, the whole, combined to a single solution outperformed any of the individual parts.
The data mining competition was hosted by the Pacific-Asia Knowledge Discovery and Data Mining conference (
PAKDD 2007) and was co-organized by the Singapore Institute of Statistics.
Even though some of the individual players employed "team" solutions themselves, a simple average compiled from the top 27 performers would have won the competition outright.
If the top 36 were averaged, this would have made runner-up spot.
Seems counter-intuitive somehow, that "average" can ever win the day, but in the realms of AI, two machine "heads" are obviously more often than not, better than one.
To cross-check, I compared two of my programs on some of my existing horse race data, then examined how an average of the two would fare.
The figures refer to cumulative absolute errors, so the lower the number the better.
NNet1 NNet2 "Team" combination
P1 5962 5956 5807
P2 6912 6957 6892
P3 11207 11558 11167
P4 1137 1124 1123
P5 5068 5110 5074
P6 7272 7499 7332
The rows, P1, P2 ... are the cases to which I'm seeking a solution
NNet1 & 2 are stand alone software programs, "Team" is the mean average of the outputs of the two programs, NNet1 & 2.
"Team" average scores top place in 4 of the 6 cases, and 2nd place in the remaining two cases (one of these a very close 2nd)
This I think is impressive. An area I have been ignoring for far too long perhaps?
DM