Sciences describing human beings have proven resistant to mathematical description. If we are unable to author elegant mathematical models, then we should embrace complexity and make use of data. The first lesson of web scale learning is to use available large scale data rather than hope for annotated data that isn’t available. Simple models with a lot of data inevitably trump complex models with less data. In practice, humans tend to care only about a finite number of distinctions. Once we have a large enough data set then that represents what we need, without generative rules.

As an example, the Google English language corpus is so large that it captures all kinds of edge cases of human behavior. That corpus could serve as a model for many cases if only we knew how to extract it from the data. Statistical speech recognition and statistical speech translation have been very successful because a large training set is available in the wild.

Alon Halevy, Peter Norvig, Fernando Pereira. The Unreasonable Effectiveness of Data. IEEE Intelligent Systems, vol. 24 (2009), pp. 8-12