Paystack Production Validation

Use this runbook before enabling Paystack in production and during the first launch window.

Required environment

Set these on the backend deployment:

PAYMENT_ENABLED=true
PAYMENT_PROVIDER=paystack
PAYSTACK_SECRET_KEY=<live-or-test-secret-key>
PAYMENT_WEBHOOK_RETENTION_DAYS=90
FRONTEND_BASE_URL=https://<app-domain>

Store PAYSTACK_SECRET_KEY in a secrets manager such as AWS Secrets Manager, HashiCorp Vault, or Kubernetes Secrets. Never commit secrets to version control. Use Paystack test keys for staging and live keys only in production; rotate the key immediately if it is exposed.

PAYMENT_WEBHOOK_RETENTION_DAYS controls raw Paystack webhook payload retention. The default is 90 days and the application caps it at 365 days.

Keep PAYMENT_PLAN_SYNC_ENABLED=false in production unless the release is intentionally creating missing Paystack plans from the DB plan catalog. The normal production model is:

1. Create or review plans in Paystack.
2. Store the corresponding paystack_plan_code on each paid lookup.subscription_plan row.
3. Deploy with plan sync disabled.

Billing terms and provider prices

A plan tier is one row. Each term a tier can be sold at is a row in lookup.subscription_plan_provider_prices, keyed by (subscription_plan_id, provider, billing_interval) and carrying the external_price_id (the PLN_... code), the currency, and the amount_minor that term actually costs. Migration 074 moves any legacy *_ANNUAL plan row's Paystack plan onto its tier as an annually price row, so an upgraded database keeps its annual offering without manual work. A database that never had annual plans will only offer monthly until an annually price row is added.

Paystack dashboard

Configure the webhook URL:

https://<api-domain>/api/v1/core/webhooks/paystack

Dashboard checklist:

1. Confirm live mode is enabled only for the production Paystack business.
2. Confirm all paid Ovahol plans exist in Paystack and their plan codes are stored in lookup.subscription_plan_provider_prices.external_price_id for the matching billing_interval (the legacy lookup.subscription_plan.paystack_plan_code is only a monthly fallback).
3. Confirm the webhook URL points to the production API domain over HTTPS.
4. Confirm the webhook secret matches PAYSTACK_SECRET_KEY.
5. Confirm payment channels are intentionally enabled for the target market.

Test-mode validation

Run these checks in Paystack test mode before using live keys:

1. Start a paid-plan checkout from the app.
2. Complete the Paystack test payment.
3. Confirm the frontend returns to /billing/callback?reference=....
4. Confirm the organization becomes ACTIVE.
5. Confirm core_public.billing_transactions has a SUCCESS row for the ovh_... reference with a non-null user_id.
6. Confirm Paystack sends subscription.create, and the organization stores subscription_code and email_token.
7. Trigger or simulate a renewal charge.success for the subscription.
8. Confirm the renewal transaction has plan_code = 'RENEWAL', status = 'SUCCESS', and user_id IS NULL.
9. Re-deliver the same renewal webhook.
10. Confirm there is still only one transaction row for that Paystack reference.
11. Trigger or simulate a paid invoice.update (Paystack's renewal signal). Confirm the organization is ACTIVE and that no extra billing_transactions row is created — charge.success owns the revenue trail, and the invoice path only restores entitlement.
12. Trigger or simulate invoice.payment_failed.
13. Confirm the organization moves ACTIVE -> GRACE; send it again and confirm GRACE -> OVERDUE.
14. Trigger or simulate subscription.not_renew.
15. Confirm the organization remains ACTIVE and non_renewing = true.
16. Trigger or simulate subscription.disable.
17. Confirm the organization becomes CANCELLED and clears subscription_code, email_token, grace state, and non_renewing.

Use Paystack's current test payment details from the official docs when validating cards, failures, and pending payment states: https://paystack.com/docs/payments/test-payments/

Rollout checklist

1. Apply database migrations through 036_payment_webhook_events.
2. Deploy backend and frontend with PAYMENT_ENABLED=true, PAYMENT_PROVIDER=paystack, and live PAYSTACK_SECRET_KEY.
3. Keep PAYMENT_PLAN_SYNC_ENABLED=false unless intentionally creating missing provider plans.
4. Confirm /api/v1/core/plans returns the expected paid and free plans.
5. Complete one live low-value paid checkout with an internal organization.
6. Confirm the transaction appears in organization billing history with both ovh_... and Paystack references.
7. Confirm /admin/billing/subscriptions reconciliation has no unexpected issues.
8. Confirm recent webhook events appear under Recent Paystack webhooks and signatures are valid.
9. Confirm expired core_public.payment_webhook_events rows are purged by PaymentWebhookRetentionScheduler.

Post-launch monitoring

After enabling Paystack, review billing health daily during the first launch week:

1. Run the admin billing reconciliation check from /admin/billing/subscriptions.
2. Investigate any organization where local subscription state conflicts with provider subscription metadata.
3. Watch backend logs for rejected webhook signatures, unrecognized webhook events, missing subscription codes, and failed renewal transaction writes.
4. Confirm duplicate webhook deliveries remain idempotent and do not create duplicate core_public.billing_transactions rows.
5. Confirm org admins receive billing emails for payment success, renewal success, failed payment, overdue status, cancellation, and plan changes when mail is enabled.

Data retention and exports

core_public.payment_webhook_events.raw_body may contain personal or payment-provider data. Treat this table as restricted operational evidence:

1. Do not include core_public.payment_webhook_events in general analytics exports, customer exports, or long-lived support extracts.
2. If database backups are retained longer than PAYMENT_WEBHOOK_RETENTION_DAYS, either exclude this table from long-term backup sets or apply the same retention policy to restored/exported copies.
3. Full raw payload access is restricted to super-admin billing diagnostics. Non-elevated service access must use sanitized payloads.
4. The purge job deletes rows after expires_at; do not bypass it for routine reporting.

Manual repair

Use manual repair only after checking Paystack and the local billing transaction record:

1. If a Paystack payment succeeded but the organization did not activate, use Replay payment on /admin/billing/subscriptions with the ovh_... reference.
2. Re-run reconciliation and confirm the organization status, plan, provider subscription code, and transaction status now match.
3. If the issue came from a webhook payload rather than a known payment reference, inspect Recent Paystack webhooks on /admin/billing/subscriptions and compare the stored raw payload with Paystack's dashboard event.
4. If manual DB repair is unavoidable, preserve financial records. Do not delete core_public.billing_transactions; append notes to the incident record and re-run reconciliation after repair.

Rollback procedure

1. Disable new checkouts by setting PAYMENT_ENABLED=false or routing users away from checkout.
2. Keep the webhook endpoint online while investigating so renewal/failure events are still recorded.
3. If rolling back migrations, do not run the 035 down migration while core_public.billing_transactions.user_id IS NULL rows exist; it intentionally fails for manual review.
4. Reconcile Paystack dashboard state against /admin/billing/subscriptions before re-enabling checkout.
5. Rotate PAYSTACK_SECRET_KEY if any secret exposure is suspected during rollback.

Expected webhook fields

The backend currently expects these Paystack webhook fields:

Headers:

• x-paystack-signature (for webhook verification)

Payload:

• event
• data.reference
• data.status
• data.amount
• data.currency
• data.paid_at
• data.subscription_code, or data.subscription.subscription_code
• data.email_token
• data.plan.plan_code

The x-paystack-signature header is required and must be validated before processing the webhook payload or trusting any event or data.* value.

If Paystack test-mode payloads differ, update backend/internal/infra/payment/paystack.go and the parser tests before production.