Computational Cognitive Science 2010

"The theory of probability is at bottom only common sense reduced to calculation."
     -- Pierre Laplace, 1814.

"Behind the diverse behaviours of humans and animals, as behind the various motions of planets and stars, we may discern the operation of universal laws."
     -- Roger Shepard, 1987

Course description: This course provides an introduction to computational theories of human cognition. We use formal models from artificial intelligence and mathematical psychology to consider fundamental issues in human knowledge representation, inductive reasoning, learning, decision-making and language acquisition. What kind of informational structures describe the organisation of human knowledge, and what kinds of inferences do they license? How do humans make choices given time constraints, computational limitations, and external costs imposed by the world? What kinds of innate knowledge (if any) must people have? And how can formal models of human cognition inform our understanding of the design of intelligent machines?

Can you say that in the form of a sound bite? Yes, yes we can. This subject asks what kind of computer the mind might be, and what kinds of mathematics we need to describe it.

Can you link to a special issue of a journal that gives a nice overview of the general topic? Excellent question. How good of you to ask. Check out Trends in Cognitive Science (follow the links to Volume 10, Issue 7, July 2006).

Is this psychology, maths or computer science? In a way, it's "all of the above". The simplest description of "computational cognitive science" (CCS) that we've been able to come up with goes like this... it's just like artificial intelligence (AI), except that instead of trying to build superintelligent machines that could take over the world (according to a certain kind of paranoid mindset), we're trying to figure out how it is that nature created the human mind, a superintelligent machine that really did take over the world! AI involves trying to engineer new minds; CCS involves reverse engineering existing ones.

In fact, this idea has a fairly long history in psychology. It might sound strange if you've only ever thought of "psychology" as having something to do with counselling, but the search for formal mathematical laws underlying human thought and reason goes back at least as far as the 1860s, when the first psychophysicists began to uncover the mathematical laws that relate physical intensities (of light, sound etc) to the corresponding subjective perceptions (of brightness, loudness etc). It should come as no surprise that "the laws of thought" are harder to uncover. However, we also have new ideas about what computation is and new methods for investigating how the mind works. With that in mind, we thought it might be a good idea to have a subject based around current theories and techniques. And if that means that we have to talk about Kolmogorov complexity, then so be it...

How much programming/mathematics do I need to do? There will be a certain amount of flexibility, but you'll have to do a fair bit of at least one of them! You should expect to have to do at least some basic derivations and coding (in matlab, but we won't assume prior knowledge of matlab - although having programming experience will be very useful). We hope to set up the problem sets so you can emphasise coding or maths more, depending on your preference, but you should absolutely expect to do at least some of both. This is a maths/CS course, after all; if you want a non-technical subject, there are plenty of those in the psych curriculum already. We will be introducing psychological themes and content but will not be assuming you have a background in that already. Some understanding of statistics (not how to run SPSS, but actual statistical theory) will also be useful.

When is it? First semester 2010. One lecture will be on Tuesday 1.10-2.00pm in Plaza 2060 lecture room; one on Friday 2.10-4.00 in Napier LG23. There's also one tute per week. Marks will be part problem-set based and part exam-based.

Enough talk. Show me a syllabus. Okay, you need to take this with a grain of salt -- okay, many grains of salt -- because we're still arguing with each other over precisely what the lectures will look like, but here's what the last version of the syllabus was. At least you'll get a sense of what topics we care about and what keywords we'll be talking about...

Syllabus

  1. Introduction and overview course structure; the relevance of computation
  2. Theme 1: The problem of induction contrast with deduction; Goodman's problem of induction; relation to the "no free lunch" theorems
  3. Theme 2: Knowledge representation symbolic vs non-symbolic approaches; compositionality; structured vs no structure
  4. Theme 3: Types of theories in cognitive science Marr's levels; what computational models can and cannot tell us
  5. Basic Bayesian inference: Bayes' rule; conditional vs marginal probability; conjugacy; exchangeable random variables
  6. Complexity and Ockham's razor: relationship to prior probability; information theory; the minimum description length principle
  7. Introduction to decision making: definition of rationality; heuristics; biases (anchoring, confirmation bias, etc)
  8. Models of choice: MAP vs Maximum likelihood; Luce's choice axiom
  9. Sequential sampling 1: Accumulator / diffusion models: first-past-the-post reasoning; mathematical interpretation; applications
  10. Sequential sampling 2: More complex decision making: Markov models; ergodicity; application to decision problems
  11. Sequential sampling 3: Searching: Markov chain Monte Carlo; Metropolis-Hastings algorithm
  12. Applications to human reasoning: wrap-up of decision-making; Reaction time vs accuracy; neural interpretation
  13. Introduction to concepts: Prototype vs exemplar vs rule-based theories
  14. Simple models for concept learning and categorisation I: naive Bayes; beta-binomial model
  15. Simple models for concept learning and categorisation II: the generalised context model
  16. Connectionist models of category learning: ALCOVE; Rogers & McClelland
  17. Hierarchical learning I application to human cognition; hyperparameters; hierarchical beta-binomial model
  18. Non-parametric models I: Chinese restaurant process; Dirichlet process
  19. Non-parametric models II: Infinite mixture model; applications
  20. Non-parametric models III: Topic model; applications
  21. Learning structured concepts: Bayesian structure discovery
  22. Applications to human concept learning; wrap-up of concepts; Concept learning over development; complexities not accounted for by models
  23. Introduction to language: Great debates- Nature vs. nurture; poverty of the stimulus; representation
  24. Bottom-up statistical information: n-gram models; word segmentation
  25. Hierarchical Markov models I: description; application to identifying parts of speech
  26. Hierarchical Markov models II: relationship to regular grammars; limitations in capturing human linguistic knowledge
  27. More complicated grammars: Chomsky hierarchy; context-free grammars; dependency grammars
  28. Parsing: human parsing; ambiguity; inside-outside algorithm
  29. Grammar induction: unsupervised vs supervised; difficulties; search space problem
  30. Conclusion/wrap-up: reprise of main themes; common modeling approaches; open questions