Hollis Robbins on training LLMs on poetry and the concept of judging “greatness”:

Poetry is a kind of test case, a training ground for understanding what “expert knowledge” brings to the table. Mercor’s bet is that the same process that trains a model to write better poetry can train it to do better legal drafts, better medical diagnoses, better financial analyses. The core assumption is that professional judgment (a lawyer deciding how to frame an argument) and aesthetic judgment (a poet deciding how to break a line) are computationally similar problems. Both require the model to navigate an “unbounded” decision space where there is no single “correct” answer, only “better” or “worse” ones based on expert consensus.

Suddenly the “they should have sent a poet” line from Contact takes on a whole new meaning.

This is a victory for the humanities I guess? Perhaps a Pyrrhic one though.