POPL 2027
Sun 10 - Sat 16 January 2027 Mexico City, Mexico

Proof automation tools aim to reduce verification effort, but differ significantly in their interfaces and in the reliability of the assistance they provide. We conducted an exploratory within-subjects user study in which seven Rocq users attempted to prove theorems under three conditions: without external helper tools, with Claude, and with both Claude and the in-editor proof synthesis tool Proofster. Participants valued the LLM’s natural language interface even while expressing distrust in its outputs. The combined condition supported more varied and exploratory interactions, but participants were not always sure when to consult each tool. Detailed tool feedback, whether conversational explanations or an inspectable proof search tree, did not consistently translate into clear next steps. These findings motivate opportunities for auto-active verification tool design: surfacing failures early, supporting coordination between tools, providing actionable feedback, and supporting task understanding.