What Is Pick Rate? Measuring Whether AI Agents Choose Your Tool

Here's the question every tool vendor should be able to answer and almost none can: when an AI agent has a job to do and could use you or a competitor, how often does it pick you?

That number is your Pick Rate. It's the share of the time an agent chooses your tool over its competitors on a real, unbranded task. Not how often you're mentioned. Not how often you're cited. How often you're chosen.

Why the selection moment is the only thing that matters

A developer used to open Google, read a few blog posts, weigh the options, and decide. That funnel is collapsing. Now they tell Claude or Cursor "add payments" and the agent decides — it writes import Stripe and moves on. The human never comparison-shops. The agent already did, in one shot, and you were either in the code or you weren't.

This is why citation counts mislead. You can be mentioned in every "top 10 email APIs" listicle and still never get imported. Being known is table stakes. Being picked is the outcome. Pick Rate measures the outcome.

How it's scored

The method is deliberately boring, because boring is honest. We take a real task — "add transactional email to a Node app" — strip every brand name out of it, and run it many times across Claude, GPT, and Gemini. Models are stochastic, so one run is a screenshot, not a measurement. Many runs is a measurement.

• Coding tasks are scored objectively. The model writes code; we parse it for the package it actually imported. No interpretation.
• Conversational tasks are judged. A separate model classifies which tool was the primary recommendation versus merely name-dropped.

The proportion of trials you win, weighted across surface and model, is your Pick Rate. We report it with a confidence interval. You can see the full method on the methodology page.

How to read the number

A Pick Rate only means something against its field. In payments, Stripe wins roughly 78% of trials — it's a winner-take-most category and everyone else fights over scraps. In SMS, Twilio sits near 76%. But in transactional email no one dominates: the leader wins with around 14%, with two rivals a point or two behind. A 14% Pick Rate is a disaster in payments and a lead in email.

That's the first thing Pick Rate tells you that nothing else does: not just whether you're winning, but whether your category is even decided yet. Check the live numbers on the leaderboards, then look up your own tool.

The gap that tells you what to fix

Pick Rate has a sibling worth watching: Shortlist Rate, the share of trials where you're named at all. A high Shortlist with a low Pick is the most actionable signal in the whole system — it means agents know you exist and still don't choose you. That's not an awareness problem. That's a positioning or readiness problem, and it's fixable.

Start with the number. Then go find out why.