Swarm Intelligence: A Reading Note
Chapter 2: Symbolism and Connectionism
How can we imbue machines with intelligence? The question turns on how humans think. We do not process raw stimuli directly; we first convert them into mental representations, then operate on those representations. Cognitive science splits the mind's machinery into two accounts (computationalism and connectionism), and artificial intelligence mirrors this division (symbolism and connectionism).
Computationalism treats representations as discrete symbols and the brain as a formal engine that manipulates them. Traditional symbolic AI encodes problems as tree structures and solves them by decomposition. To open a door: check whether it is locked; if so, find a key; if not, turn the knob. Each branch leads to a smaller subproblem. This architecture handles sequential decisions well but carries two limits. Trees lack feedback loops, and connections encode only order, not strength or direction. Symbolism also fails to explain certain cognitive phenomena. Richens's Semantic Network links concepts in a graph, yet it cannot account for the representativeness effect: people take longer to confirm "a penguin is a bird" than "a canary is a bird." Under the network's assumptions both traversals should cost the same, since both link to the concept bird. They do not.
Connectionism descends from empiricism, the claim that knowledge is induced from experience, and its successor associationism, the claim that knowledge is constructed through learned associations. Hume argued that the most fundamental mental operation is connection: simple ideas combine into complex ones through linkage. Modern connectionism in AI originates with Rumelhart and McClelland's (1986) parallel distributed processing model, which proposed that the mind could be represented by a network modeled on the neural system. Neural networks map units to neurons and connections to synapses. Each connection carries a weight encoding the relevance between linked units, and a mental representation corresponds to the activation state of the network's output. Units obey a threshold rule: when input exceeds a critical value the unit fires, otherwise it remains silent (outputs 0), mirroring the all-or-nothing law of biological neurons. The Hebbian rule adds that innate connections between neurons are necessary for learning; experience strengthens them. The engineer's task is to establish initial connectivity in a machine and let acquired data reshape it. The difference between semantic and neural networks reduces to granularity: in a semantic network each node is a concept; in a neural network a concept is a distributed state across many nodes.
Neural network design often borrows from psychological theory. Meta-learning draws on human metacognition (Jackson, 2004). It teaches machines how to learn by autonomously adjusting their own parameters or structures The attention mechanism introduced in "Attention Is All You Need" (Vaswani et al., 2017, cited over twenty thousand times) enabled networks to selectively weight specific tokens in long sequences, achieving precise processing of extended input. DeepMind's "Machine Theory of Mind" (Rabinowitz et al., 2018) trained a neural network to infer the goals and beliefs of other agents; though the experiments remained within a grid world, equipping machines with such inference could transform human-machine interaction. More recently, neural network technology has advanced past the point of human interpretability. Meta-learning allows networks to generate their own architectures, and we lack the theoretical foundations to explain what individual parameters encode. Ali Rahimi, accepting the 2017 NeurIPS Test of Time award, compared deep learning training to alchemy. Recall the earlier argument: the mind's apparent randomness stems from our ignorance of its mechanisms. By that criterion, networks whose internal operations elude their creators may already satisfy one condition we associate with having a mind.
Symbolism and connectionism each solve problems the other cannot, and modern systems combine them. AlphaGo opens with Monte Carlo tree search (a symbolic method where raw computational power yields direct advantage) but switches to a deep neural network when the branching factor exceeds what even supercomputers can enumerate, obtaining rapid evaluations from a model pretrained on human game records (Silver et al., 2016).
References:
Rumelhart, D. E., & McClelland, J. L. (Eds.). (1986). Parallel distributed processing: Explorations in the microstructure of cognition, vol. 1: Foundations. MIT Press.
Jackson, N. (2004). Developing the concept of metalearning. Innovations in Education and Teaching International, 41, 1470-32971470.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... Polo- sukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 5999-6009.
Rabinowitz, N., Perbet, F., Song, F., Zhang, C., Eslami, S.M.A. & Botvinick, M.. (2018). Machine Theory of Mind. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:4218-4227.
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., ... & Hassabis, D. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484-489.