2. Set the success criteria

Before you start

• A chosen hypothesis from step 1.
• The person who will make the decision. They should agree the criteria now, not after the results.

Steps

1. Compare the experiment designs

   In your AI toolIn the app

   Show me the SwiftCNS experiment options with their scores and success criteria. I'll choose.

   The card Select an experiment lists the designs. Open Approach and measurements on each to see how it runs and what it measures.

   Each design is scored 1 to 5 for setup, run time, cost and data reliability (lower is simpler, faster, cheaper, more reliable) and comes with success criteria and key metrics.

2. Check the criteria before you choose

   Would a miss change your mind? If not, raise the criteria or pick another design. Check the sample size and how long it will take. Don't choose on cost alone: a cheap design that measures what people say is weaker than one that measures what they do.

   The experiment's success criteria are what you'll call the result against. They can differ from the provisional criteria in the hypothesis; make sure the two agree before you approve. Provisional criteria are a starting point. To change them, choose Edit experiment below the thread (or tell your AI tool) and describe the change.

3. Assign and approve

   Choose an owner and, if you like, a due date, then approve. Approving commits you to this design and these success criteria. They can't be edited afterwards.

What you get

The running example offered two designs for the same hypothesis:

The team chose the paid offer, because only real payment answers the question. The criterion they approved:

At least 3 of the first 15 trial teams that see the offer pay $79 within 7 days of seeing it.

Two more metrics explain the result without deciding it: where teams drop off, and why they don't pay.

Done when

• The criterion is a number to reach, with a sample and a time window.
• The decision-maker agreed it before the experiment started.
• The experiment is approved and has an owner.

Common mistakes

• No number. "We'll see what people say" can't be missed.
• Criteria you'd ignore. If 1 in 15 would still make you build it, 3 in 15 isn't your real criterion.
• Measuring interest when you need commitment. Clicks and "I'd pay" are weaker than payment.

For agents: MCP tools and gotchas

• experiment_card_selection via select_experiment (selectedExperimentCardId).
• Then assign_experiment (assigneeUserPublicId, optional dueAt in RFC 3339 with an offset, in the future). Find eligible people with list_eligible_experiment_assignees.
• Then approve_experiment (empty submission). All three need cns:decide and return continuationMode: NONE.
• Edits before approval: send_message describing the change, then apply_field_edit_proposal with a toolCallId from get_message → fieldEditProposals. Edits are refused once the experiment leaves draft.

Next

Step 3: Run the smallest experiment.