Fragments: October 4

In response to my last fragments (probably the bit about us worrying if LLMs have consciousness when we when we should be wondering why they don’t have a conscience) “Metalanguage” replied:

we shipped the id and forgot the superego. classic software lifecycle.

I don’t know what was on their mind, but their post immediately made me think of the classic 1956 movie Forbidden Planet. Plenty of sci-fi, and other literature, have explored humans creating technology with unintended behavior, going back at least to Mary Shelly. But that movie was particularly influential on sci-fi film-making and in the heart of its story is what happens when we nurture a thinking machine.

I use the term “nurture” here deliberately. We talk of building software, but building implies a degree of determinism. When we build a bridge, or a locomotive, we expect it to behave in a controlled and well-understood manner. That’s a difference in degree to how we cultivate plants in our garden, or nurture young children. One of the challenges of working with these systems is understanding what has changed in this shift from building a computational system to nurturing an inferential one, and how our processes need to change in response.