QA Documentation
Platform Overview
What is this?The CMP Overlay Platform is a centralized abstraction and orchestration layer that sits above multiple Operator CMPs (e.g., Airtel, Vodafone/VI). It provides a Single Pane of Glass (SPoG) for managing telecom SIM inventory and lifecycle without having to log into each operator's portal separately.
As QA on this project your primary concerns are: async pipeline correctness, RBAC + parent context scoping, SIM state transition integrity, and inventory upload/validation edge cases. These are the highest-risk areas with the most potential for silent failures.
Modules in Scope
FeaturesAuthentication & Access Control governs all entry points and feature visibility.
- Login with email/password; invalid credential handling with proper error messaging
- Forgot/Reset password flows — link must point to correct env (not localhost)
- Parent context selector (top-right dropdown) — changes accessible data and features for the session
- Org Admin RBAC: can have multiple roles from different parents; effective access = intersection per selected parent
- Non-admin RBAC: single role; effective access = role features ∩ selected parent's permissions
- Dashboard access is mandatory for all roles — no user should be locked out of dashboard
Reset password link was found pointing to localhost in certain environments. Always verify link targets match the deployed environment under test.
The Dashboard is read-only and shows aggregated SIM state summaries and activation trends.
- SIM state summary counts (Ready, Active, Suspended, etc.) must update when parent context changes
- Activation trend charts are informational — validate data accuracy vs DB/API
- No write actions on this screen — test for any unintended mutation paths
Account / Organisation Management manages the parent-child account tree.
- View full account hierarchy tree (parent → child chain)
- Create new child account: requires admin user, unique admin email, role assignment
- Link existing account as child: manual account ID entry; must block circular links
- Manage users within child accounts: view, add, edit user records
Test circular link prevention (A→B→A), duplicate admin email on child creation, and what happens when a parent account is removed while children exist.
Manage SIMs is the primary operational screen for SIM lifecycle actions.
- Search/filter by: ICCID, MSISDN, IMSI, date/range, profile, organisation, plan status/name
- Bulk Change State: select SIMs from table OR upload CSV list of ICCIDs
- Request type selection (Activate, Suspend, etc.) with optional M2M plan selection
- Submit → triggers async request; status must be trackable
- SIM details view via arrow action → shows full metadata and device details
- Unverified SIMs must NOT appear in Manage SIM (only in Inventory)
Unverified SIMs appearing in Manage SIM is a critical data integrity failure. Add explicit tests to confirm visibility rules are enforced after inventory upload at all pipeline stages.
Inventory Management is the most complex module with the highest QA surface area.
- Inventory summary by child account with counts and verification status
- View Inventory (drill-down), Download Inventory (async export → available in Reports)
- Inventory Upload: CSV/XLSX; single-profile and multi-profile (up to 3 profiles)
- Three ICCID variants per SIM: Operator Portal ICCID, ICCID with device checksum, ICCID without checksum
- Re-upload pre-verification: corrections and adding MSISDN/profiles allowed
- Re-upload post-verification: must go through Inventory Correction flow
- Inventory Correction: correct ICCID variants + SIM swap (move profile to new ICCID; old record end-dated, not deleted)
| Rule | Detail | QA Action |
|---|---|---|
| Field length | ICCID/IMSI/MSISDN must meet exact length specs | Test boundary values: correct, -1, +1 |
| No scientific notation | Large numbers must not be stored in exp format | Upload with Excel-auto-formatted cells |
| Forbidden characters | Special chars not allowed in SIM ID fields | Test injection chars: #,%,',< |
| No duplicates within file | Same ICCID/IMSI/MSISDN cannot appear twice in same upload | Upload file with intentional dups |
| ICCID invariant | ICCID1 and ICCID-with-checksum must contain ICCID-without-checksum as substring | Verify substring check logic |
| MSISDN optional | Upload allowed without MSISDN; verification blocked until MSISDN exists | Upload without MSISDN, confirm verification blocked |
Request Management tracks all async state-change requests raised from Manage SIMs.
- View raised requests with status tracking (pending, in-progress, completed, failed)
- Retry failed requests — verify idempotency; retrying must not create duplicate operations
- Parent visibility into child account requests
- Verify requests generated from both table-selection and CSV-upload paths appear correctly
Reports provides async exports at account and SIM level.
- Generate account-level and SIM-level export reports
- Async download: job completes → user notified → user downloads from Reports section
- Validate export data accuracy against UI-shown counts and details
- Enforce permission-based access to reports (child accounts cannot see parent reports)
Cross-validate export file column values against what the UI displays for same filters. Test permission boundary: log in as child org user and attempt to access parent's report URL directly.
Data Flow Diagrams
ArchitectureAsync Pipeline (Kafka)
High RiskThe Kafka-based pipeline is the most critical QA area. Two primary flows run through it: Inventory Upload Verification and SIM Lifecycle/State Change. Failures here are often silent and require explicit observability.
Async pipeline issues are often silent. A job can appear to submit successfully but fail silently downstream. QA must have visibility into Kafka consumer logs, DB job tables, and operator API call records to verify end-to-end correctness.
| Pipeline Concern | What to Test | Severity |
|---|---|---|
| Idempotency | Submit same inventory twice; confirm no duplicate SIM records created | Critical |
| Ordering / GCID | GCID/identifier sequence enforcement — out-of-order events must be handled | Critical |
| Retries | When operator API fails, retry must happen; verify backoff behavior | High |
| Circuit breaker | Sustained operator API failures should trip circuit breaker; verify fallback state | High |
| DLQ handling | Confirm dead-letter queue exists and messages route there on repeated failure | High |
| Partial corruption | If consumer fails mid-batch, verify DB is not left in inconsistent state | Critical |
| Job table accuracy | UI status must match DB job table state at all times | High |
Data Model & Invariants
EntitiesUnderstanding the data model is essential for QA to write accurate test assertions and catch data integrity violations.
| Invariant | Rule | Test Verification |
|---|---|---|
| ICCID substring rule | ICCID1 and ICCID-with-checksum must contain ICCID-without-checksum as substring | Submit ICCID where substring fails; expect rejection |
| No cross-field dups | No duplicate ICCID/IMSI/MSISDN within same upload file | Upload file with intentional cross-field duplicates |
| MSISDN & verification gate | MSISDN is optional at upload, but verification is blocked until MSISDN exists | Upload without MSISDN → confirm verification status stays blocked |
| Re-upload restrictions | Pre-verification: corrections OK. Post-verification: only via Inventory Correction | Test both paths explicitly; confirm post-verification direct re-upload is blocked |
| SIM swap end-dating | When SIM is swapped to new ICCID, old record is end-dated (not deleted) | Perform swap; verify old ICCID record exists with end_date set |
| Unverified SIM exclusion | Unverified SIMs must NOT appear in Manage SIM screen | Upload SIM without MSISDN → check Manage SIM; should not be listed |
QA Approach & Test Planning
StrategyMap requirements from user stories + inventory spec + existing test cases into a traceability matrix. Each row should link: Story/Feature → Test Cases → Execution Status → Defects. This is your single source of truth for coverage.
Clarify whether operator APIs in QA are real sandbox or mocks/stubs. Confirm Kafka availability and whether you have DB read access to observe async job states. If only UI is accessible, many async tests will be inconclusive.
Functional Testing Areas
Test Cases| Module | Key Test Scenarios | Priority |
|---|---|---|
| Auth | Valid/invalid login, forgot password link target, password rules, session expiry, token refresh, logout | P1 |
| RBAC + Parent Context | Feature visibility changes per parent, data scoping per parent, org-admin multi-role intersection, non-admin single role | P1 |
| Dashboard | SIM counts update on parent switch, chart accuracy, no write actions possible | P2 |
| Account Management | Create child (unique email), link child (no circular links), view hierarchy, user management in child | P1 |
| Manage SIMs | All filter combos, table select + CSV upload path, all state transitions, plan selection required/optional, SIM detail view | P1 |
| Inventory Upload | Valid file, all file validation failures (negative), multi-profile (1/2/3 profiles), re-upload pre/post verification, ICCID invariants | P1 |
| Inventory Correction | Correct ICCID variants post-verification, SIM swap (verify end-dating of old record) | P1 |
| Requests | Status lifecycle, retry idempotency, parent sees child requests, child cannot see parent requests | P1 |
| Reports | Async export job, notification on completion, data accuracy vs UI, permission enforcement (direct URL access) | P2 |
Integration & Async Testing
Kafka · Operator APIsThese tests are the hardest to execute but carry the highest impact. They require infrastructure-level visibility.
- Message production: Confirm events are produced to correct Kafka topic on upload submit and state-change request
- Message consumption: Consumer picks up events in order; verify GCID sequence enforcement
- Retry behavior: Simulate consumer failure mid-process; confirm retry with backoff and no duplicate records
- Duplicate handling: Publish same event twice; confirm idempotent processing (same result, no second DB record)
- DLQ: Force repeated consumer failure; confirm message lands in Dead Letter Queue and does not block healthy messages
- Kafka lag monitoring: Under load, verify consumer lag stays within acceptable bounds (performance target: 200–300 TPS)
- API success path: Verify ICCID/IMSI/MSISDN verified correctly and SIM status updated in DB
- API failure path: Simulate operator API timeout/500; confirm retry is triggered with correct backoff
- Circuit breaker: Sustained failures should trip circuit breaker; verify CMP enters fallback state gracefully
- Partial batch failure: If some SIMs verify and some fail, confirm no cross-contamination; each SIM status accurately reflects its own outcome
- API contract: Verify request format sent to operator API matches expected schema; test with real sandbox if available
- CMP vs Operator mismatch: Manually alter a SIM state in operator portal; verify reconciliation job detects and corrects the mismatch in CMP
- Alert on mismatch: Confirm alerting/notification is triggered when reconciliation finds discrepancy
- Reconciliation frequency: Understand schedule; verify it runs as expected and logs are available
Confirm with the team whether reconciliation jobs exist and what their trigger mechanism and frequency are. This directly affects test environment setup requirements.
Non-Functional Testing
Performance · Security| Area | Target / Approach | Tools / Notes |
|---|---|---|
| Throughput | 200–300 TPS with response time targets | Load test with realistic SIM upload + state change mix |
| Kafka lag | Consumer lag must stay within SLA under load | Monitor with Kafka consumer metrics / Grafana |
| Soak / stress test | Sustained load over hours; check memory, CPU, DB connections | Monitor for connection pool exhaustion, OOM |
| Auth security | Brute-force lockout, token expiry, refresh behavior | Test rate limiting on login endpoint |
| Input injection | Injection testing on CSV fields and search inputs | SQLi, XSS, special char uploads |
| RBAC bypass | Direct API calls without proper parent context; privilege escalation attempts | Test all permission boundary API endpoints directly |
| Session handling | Session persistence across refresh, token expiry, concurrent session behavior | Known existing bugs in bug sheet — prioritise |
Risk Registry
QA RisksOpen Points & Assumptions
Needs ConfirmationThese items must be confirmed before QA execution can begin in those areas. Each open point blocks a specific set of tests.
| # | Open Point | Why It Matters for QA | Status |
|---|---|---|---|
| 1 | Operator API in QA environment — real sandbox or mocks? | Determines whether integration tests give real verification results or need mock assertions | Pending |
| 2 | Test SIM data — which ICCIDs are safe to use in QA? | Prevents accidentally activating real SIMs; need a defined safe test set | Pending |
| 3 | Async observability — access to DB job tables, Kafka consumer metrics, application logs? | Without this, async tests can only verify UI state, not actual pipeline correctness | Pending |
| 4 | Request Type options — full list for Change State and state transition rules per operator? | Cannot write complete Manage SIM test cases without knowing all valid request types and allowed transitions | Pending |
| 5 | Reconciliation jobs — do they exist? What is trigger/frequency? | Required to plan reconciliation test scenarios | Pending |
| 6 | DLQ handling — is a DLQ configured in Kafka? What is the alert/recovery flow? | Needed to test what happens to messages that fail repeatedly | Pending |
| 7 | Bug sheet cleanup — ownership and process for completing incomplete bug entries? | Current bugs are not actionable without required fields (severity, env, repro, status) | Blocking |
Before starting execution: confirm environment → get test SIM dataset → get Kafka/DB access → get full state transition rules → clean up bug sheet. Only then can test cases be fully written and execution begin.