Skip to main content

Creating a Test

The test wizard walks you through four steps. This page covers each step in detail. For a quick walkthrough, see Create Your First Test.

Step 1: Basic Info​

Test name and description​

Give your test a clear name that describes what you're experimenting with. The description is optional but helpful for teammates.

Primary metric​

Select the metric you're trying to move. The dropdown only shows metrics available for your organization based on your connected data sources. Available metrics include:

  • Revenue — total sales
  • Transactions — completed purchases
  • Visits — location visitors
  • Units Sold — total products sold
  • Basket Size — average items per transaction
  • Conversion Rate — percentage of visitors who purchase
  • Enrollments — new member sign-ups
  • Enrollment Revenue — revenue from new enrollments
  • Churned Members — members who terminated or expired
  • Lost MRR — monthly recurring revenue lost to churn

Minimum Detectable Effect (MDE)​

The MDE is the smallest percentage change you want to be able to detect. For example, an MDE of 5% means you want to know if your experiment moved the metric by at least 5%.

warning

Setting MDE below 3% requires very large sample sizes and long test durations. ProofPod displays a warning if your MDE is too aggressive for your data.

Date range​

Pick start and end dates. ProofPod displays the calculated duration and recommends a minimum of 2–3 weeks. If you pick a start date in the past, you'll see a warning—tests work best when started prospectively.

Step 2: Data Source​

Choose one of three data paths:

Integration data​

If you have an active integration (Mindbody, Square, or ClubReady), this option uses your synced transaction data. ProofPod checks your connection status and auto-advances if a healthy connection is found.

Synced events​

Select an event type from your warehouse data. You can optionally filter by category dimensions (e.g., product category, service type) to narrow the data used for analysis.

CSV upload​

Drag and drop a CSV file (up to 50 MB). ProofPod:

  1. Parses the file and detects columns
  2. Shows a mapping preview with confidence scores for each column
  3. Lets you adjust mappings before confirming
  4. Creates a dataset and extracts location IDs automatically

Step 3: Location Assignment​

Divide your locations into treatment (running the experiment) and control (business as usual).

Selection modes​

  • Smart — ProofPod's algorithm picks optimal groups based on historical data similarity, stability, and donor availability. Best for most users.
  • Hybrid — starts with Smart's recommendation, then lets you drag locations between groups to customize.
  • Manual — you assign every location yourself via drag-and-drop.

Matching diagnostics​

After assignment, ProofPod shows matching quality metrics including R² (fit quality) and any warnings about group balance. See Location Matching for details on the algorithm. All quoted MDEs use your actual planned test duration.

Design Validation (placebo tests)​

Once matching completes, ProofPod automatically runs placebo tests against your design: backdated windows of your test's exact length where nothing happened (any "effect" is noise) and fake-treatment permutations across your control pool. The card shows pass/warn/fail badges—advisory only, never blocking—and everything is stored server-side as a permanent, auditable record linked to the test on creation. You can add secondary or falsification metrics from the card. See Design Validation.

Overlap detection​

If any selected location is already in another running test, ProofPod flags the overlap so you can avoid contamination.

Step 4: Review​

The review step shows a summary of your entire test configuration:

  • Test name, description, and dates
  • Primary metric and MDE
  • Data source details
  • Location assignments with counts (treatment vs. control)

Use the Edit buttons to jump back to any previous step. When satisfied, click Create Test to launch your experiment.