# P2/P3 Scaling Architecture & DB/API Sync

**Scope:** CBU bureau API, BullMQ at scale, vault deployment, GPU pool, immutable scoring audit  
**Repos:** `Agnet` (BFF/UI) + `mpgis-farmer-platform` (core-api, worker, DB)  
**Date:** 2026-06-26

---

## Recommended skills

**Primary:** `@senior-architect` — system design for P2/P3 scale, DB/API contract alignment, deployment topology  
**Autonomous prompt:** [P2P3_SENIOR_ARCHITECT_PROMPT.md](./P2P3_SENIOR_ARCHITECT_PROMPT.md)  
**Also consider:**
- `@playwright-skill` — browser E2E wizard flows alongside API suite
- `@docker-expert` — GPU node pool, worker HPA, vault sidecars

---

## System functional flow

```mermaid
sequenceDiagram
    participant Browser
    participant BFF as Agnet BFF
    participant Core as core-api
    participant Score as AgriAccessAdapter
    participant Worker as agri-access-worker
    participant DB as PostgreSQL
    participant CBU as CBU Bureau stub
    participant Vault as Secrets Vault

    Browser->>BFF: POST /api/onboarding/{id}/submit
    BFF->>Core: POST /onboarding/{id}/submit
    Core->>DB: status=scoring
    Core->>CBU: queryCbuBureau (consent-gated)
    Core->>Score: score(stepData)
    Score->>Vault: GEMINI_API_KEY, WORKER_TOKEN
    Score->>Worker: POST /api/analyze
    Worker-->>Score: basel_iii + SHAP + cbuClass
    Score-->>Core: normalized 0-1000
    Core->>DB: ScoringRun + FinanceCase
    Core->>DB: ScoringAudit (append-only)
    Browser->>BFF: GET /api/onboarding/{id}/status
    BFF-->>Browser: scoringPhase, score, scoringMeta
```

---

## DB ↔ API sync matrix

| Prisma model / column | OpenAPI (BFF) | OpenAPI (core) | Notes |
|----------------------|-----------------|----------------|-------|
| `OnboardingSession.stepData` | `OnboardingStepData` | session JSON | Wizard PATCH merges per step |
| `OnboardingStepRecord.payload` | step PATCH body | same | Normalized to `FarmerProfile`, `LandParcel`, etc. |
| `ScoringRun.totalScore` | `OnboardingStatusResponse.score` | `OnboardingScoringStatus.score` | 0–1000 integer |
| `ScoringRun.decision` | `eligibilityOutcome` | same | Maps from raw 300–850 thresholds |
| `ScoringRun.rulesResult` | `scoringMeta`, `reasonCodes` | adapter output | JSON blob; dashboard reads `riskPd` |
| `FinanceCase.publicCaseId` | `financeCaseId` | same | `AGN-FIN-*` |
| `FinanceCase.notes` | `FinanceModule.riskPd` | — | JSON: PD, rating, IFRS9, CBU |
| `ScoringAudit.*` | — (internal) | — | Immutable; SEC-05 redacted snapshots |
| `AdapterCredential.encryptedRef` | — | vault path | T-006 / AA-011 |

Full machine-readable matrix: `lib/e2e/api-flow-registry.ts` → `DB_API_SYNC_MATRIX`.

---

## P2/P3 implementation status

| Task | ID | Status | Location |
|------|-----|--------|----------|
| CBU bureau API stub | AA-025 | **Done** | `packages/scoring/src/cbu-bureau.stub.ts` — live when URL+key |
| Immutable scoring audit | AA-029 | **Done** | `ScoringAudit` model + `scoring-audit.service.ts` |
| BullMQ at scale | AA-033 | **Done** | `@mpgis/scoring` pipeline + `jobs-worker/scoring-worker.ts` |
| Vault deployment | AA-011 | **Done** | `vault-loader.ts` + `AdapterCredential` seed |
| GPU node pool | AA-031 | **Done** | `infra/k8s/agri-access-worker-hpa.yaml` |
| Gemini rate limit | AA-035 | **Done** | `redis-scoring.ts` — 1 explain/session |
| Redis idempotency | AA-033 | **Done** | `acquireScoringLock` + existing `ScoringRun` check |
| Model version pinning | AA-034 | Partial | `MODEL_VERSION` env on worker |

---

## P2/P3 deployment topology (target)

```
Internet → ALB → Agnet BFF
              → core-api (public)
core-api → Redis (BullMQ scoring queue)
jobs-worker → scoring-service adapter → agri-access-worker (GPU optional)
Vault → GEMINI_API_KEY, OPENWEATHER_API_KEY, AGRI_ACCESS_WORKER_TOKEN
CBU API → core-api (consent-gated, circuit breaker)
```

### Vault paths (AA-011)

| Secret | Vault path | Consumers |
|--------|------------|-----------|
| `GEMINI_API_KEY` | `secret/agnet/prod/gemini` | agri-access-worker |
| `AGRI_ACCESS_WORKER_TOKEN` | `secret/agnet/prod/worker-token` | scoring adapter + worker |
| `OPENWEATHER_API_KEY` | `secret/agnet/prod/openweather` | worker (optional) |
| `CBU_BUREAU_API_KEY` | `secret/agnet/prod/cbu` | core-api (AA-025 live) |

### BullMQ scaling (AA-033)

- Queue: `scoring` (existing in `jobs-worker`)
- Concurrency: 10 workers per pod, HPA on queue depth
- Idempotency: `onboardingSessionId` unique index on `ScoringRun`
- Job payload: `{ sessionId, stepDataHash, correlationId }`

### GPU pool (AA-031)

- Optional node pool: `nodeSelector: workload=gpu`
- Worker env: `PRITHVI_DEVICE=cuda`, `PRITHVI_MODEL_CACHE_DIR=/models`
- CPU fallback: documented in worker README when GPU unavailable

---

## E2E test coverage

| Suite | Command | Scope |
|-------|---------|-------|
| Contract | `npm run test:contract` | OpenAPI shapes + dev-mock |
| API full flow | `npm run test:e2e:api` | All BFF flows F01–F15 + report |
| Wizard UI | `npm run test:e2e:wizard` | Playwright 13-step onboarding |
| Worker (optional) | `AGRI_ACCESS_WORKER_URL=... npm run test:e2e:api` | Live worker F13–F14 |

Reports written to `reports/e2e-api-flow-latest.{md,json}`.

---

## Next implementation sprint (P2/P3)

1. Wire `jobs-worker` scoring worker to `AgriAccessScoringAdapter` (replace stub score=72)
2. Deploy Vault agent + `AdapterCredential` loader in core-api bootstrap
3. Add Redis idempotency + Gemini rate limit middleware
4. CBU live adapter behind feature flag `CBU_BUREAU_LIVE=true`
5. GPU node pool Terraform + worker HPA manifests