Insights — QA Plan
Auth
| Step | Expected |
|---|---|
Non-member calls GET /insights | 403 — requireMembership throws. |
| Coach role calls | 403 — role check. |
| Member role calls | 403. |
| Owner / admin calls | 200 with insight array (possibly empty). |
Empty state
| Step | Expected |
|---|---|
| Org with no cancellation requests, no debt, no past-due members, no overdue tasks | Response is { data: [] }. |
Rule — finance.cancellation_requests
| Step | Expected |
|---|---|
2 pending cancellation requests, 0 scheduled | Insight returned, count = 2, severity warning, meta.pending = 2, meta.scheduled = 0. |
0 pending, 5 scheduled (subscriptions with cancel_at_period_end = true) | count = 5, severity info. |
| 3 pending + 5 scheduled | count = 8, severity warning. |
Deleted subscriptions with cancel_at_period_end = true | Excluded from the count. |
Rule — finance.outstanding_debt
| Step | Expected |
|---|---|
| One member with debt 1200 cents | count = 1, meta.totalDebtInCents = 1200, severity warning. |
| Five members totaling 100k cents debt | count = 5, meta.totalDebtInCents = 100000. |
Subscription with debt = 0 | Excluded. |
| Deleted subscription with debt > 0 | Excluded. |
Rule — members.at_risk_billing
| Step | Expected |
|---|---|
| 0 past-due members | Rule returns null; insight not in array. |
| 1 past-due member | count = 1, severity info, sample contains that member. |
| 5 past-due members | severity warning. |
| 15 past-due members | severity urgent. |
| Sample size | Capped at 3, ordered by name ascending. |
| Member with NULL name | Sample entry has name: null, email always present. |
Rule — operations.tasks_overdue
| Step | Expected |
|---|---|
Task with due_date < today, not completed | Insight returned. |
Task due_date = today | Not overdue (strict less-than). |
| Completed task with past due date | Not counted. |
Ordering
| Step | Expected |
|---|---|
Org has urgent at-risk billing + warning debt + info cancellations | Order: at-risk → debt → cancellations. |
| Tied severities | Higher count comes first. |
Agent surface
| Step | Expected |
|---|---|
| Prompt: “anything I should look at?” | Spotter calls analytics.org_insights, returns the same payload, summarizes in prose. |
| Prompt as a coach | Tool returns 403; agent surfaces “Sorry, that’s owner/admin only.” (verify via integration test). |
Performance
- 500-member org with realistic data → response < 150ms p95.
- Re-running the same query 100 times in a loop → no rule slowdown (indexes cold-hit warm-hit ratio acceptable).
Negative
- Force one rule to throw (mock the DB) → API returns 500. Document this as a current limitation; rules are not per-rule isolated.
- Org id that does not exist → returns
[](empty arrays from each rule).