A/B Split Tests

A/B split tests let you simultaneously send different push notification variants to the same audience and measure which variant converts better by CTR. This helps make data-driven decisions rather than relying on intuition.

How It Works

When sending, the system randomly distributes recipients across variants according to the specified weights. Each user receives exactly one variant. After sending, analytics will show for each variant: how many pushes were sent, delivered, clicked and the resulting CTR.

Maximum number of variants per push — 4 (A, B, C, D).

Creating Variants

Variants are created inside a Push Pack at the level of a specific push (event-based or scheduled).

1. Open Push Packs → select the pack → open the language package → select the push.
2. In the "Variants" section click "Add variant".
3. Select the variant name: A, B, C or D.
4. Fill in title and body for this variant.
5. Set a weight from 1 to 100.

How Weights Work

The weight determines the percentage of traffic each variant receives. The system normalizes weights automatically.

Example:

• Variant A = 70, Variant B = 30 → 70% receive A, 30% receive B.
• Variant A = 1, Variant B = 1, Variant C = 1 → each receives approximately 33%.

For equal distribution, use the same weight for all variants.

6. Repeat steps 2–5 for each additional variant (maximum 4).
7. Save the push.

Viewing Results

After sending, results are available in Analytics:

1. Go to Analytics → Send History.
2. Find the required send and open details.
3. Click the "Variants" button or expand the variants tab.

The results table shows:

Alternative: Push Rotation Operator

If you don't want to use Push Packs, you can apply the {Push=...} operator directly in the push template. It rotates variants in a round-robin fashion between consecutive sends (A → B → C → A), not randomly by weights.

{Push={Welcome to {{app_name}}!|Hi {{external_id}}, we missed you!}}

This is convenient for simple A/B without precise weight control. More about operators — in the Push Templates section.

For statistically significant results, a minimum of 500 sends per variant is recommended.