Project spec

PROJECT.md

@@ -0,0 +1,214 @@
+Project: REmodel
+A Python calculation engine + FastAPI backend + Next.js frontend for Indian renewable energy (Solar + Wind + BESS) project finance modeling. Replaces an Excel-macro workflow currently used by senior finance professionals at ReNew Power for bid preparation.
+Owner: Manohar (senior finance professional, self-taught dev, India). Builds in evening/weekend hours.
+Goal: Compute optimal flat tariff and full 25-year project financials for hybrid RTC RE projects in <30 seconds per scenario, with 50-scenario sensitivity sweeps in <10 minutes.
+Vision
+Three layers of architecture, in order of foundation to UI:
+
+Calculation engine (packages/engine) — pip-installable, UI-agnostic Python library. Takes a Pydantic ScenarioInput, returns a Pydantic ScenarioResult. Can be driven by CLI, Jupyter, or the API. This is the durable asset.
+API layer (packages/api) — FastAPI + Arq async workers. Schedules scenario runs, persists results to SQLite + Parquet. Exposes REST + SSE for the UI.
+Web UI (packages/web) — Next.js 14 App Router + shadcn/ui + Tailwind. Configurable KPI dashboard, scenario wizard, results viewer, sensitivity tornado.
+
+Domain context (essential reading)
+
+Indian RE bidding workflow: developer wins a PPA (power purchase agreement) at a tariff (₹/kWh) bid into auctions (SECI, state DISCOMs). Tariff must clear target equity IRR (typically 15-20%) given debt sizing constraints (D:E ≤ 75:25, DSCR ≥ 1.20).
+Hybrid RTC project: delivers ~24×7 power from a combination of solar + wind + BESS (battery firming). Contracted at a "RTC CUF" — the developer commits to a specific load factor, e.g., 57.64%. Shortfall triggers DSM (deviation settlement mechanism) penalties.
+Construction takes 18-36 months. During this time, debt is drawn progressively, and interest accrues — this is IDC (Interest During Construction). IDC is a major capex item and a primary lever for tariff optimization.
+Indian tax regime: Section 115BAA applies → 22% + cess = 25.17%, no MAT, no carry-forward. Book and tax depreciation differ (SLM book vs WDV 40% tax for solar/wind plant), creating deferred tax.
+Currency: INR Crore (Cr) = 10 million; Lakh = 100 thousand. Capex typically quoted in Cr/MW or INR/Wp.
+
+Key architectural decisions (don't relitigate)
+
+Engine is UI-agnostic. Importable Python package. Web UI is one of three drivers (web, CLI, notebook). No web logic in engine.
+CostItem table model, not flat fields. Every capex line item is a record with basis (INR/Wp, Cr/MW, Lakh/Acre, etc.), depreciation class, attribution (solar/wind/BESS/common), and a phasing curve.
+Three nested iterations in the solver: tariff (brentq) → debt sizing (fixed-point) → IDC (fixed-point). Each converges in 5-20 iters.
+IDC has independent equity and debt drawdown schedules. Equity can bridge (>100% momentarily), debt can replace equity later (negative cum equity delta). Both editable by user.
+Two compliance knobs for DSCR: raise tariff OR reshape debt schedule. User chooses; system supports both.
+Sync-async hybrid: all scenario runs go through Arq queue (Redis-backed). UI polls/subscribes via SSE. Even single runs use the queue to avoid refactor later.
+Storage: SQLite (v0) + Parquet for timeseries. Migrate to Postgres in v2 when multi-user.
+DataGrid component is a shared React component used in 5+ places (CostItem table, phasing matrix, equity/debt schedules, opex table, debt repayment %). Build once, well.
+Excel parity is the trust gate. Three parity gates at Sprints 2, 3, 4, 7. We don't ship a sprint if its parity gate fails.
+
+Tech stack (locked)
+
+Backend: Python 3.12, Poetry, Pydantic v2, FastAPI, Arq, Redis, SQLAlchemy + SQLite, Pandas, NumPy, scipy, numpy_financial, pyarrow, openpyxl
+Frontend: Node 20, Next.js 14 App Router, TypeScript strict, shadcn/ui, Tailwind, Recharts, AG Grid Community (for DataGrid), TanStack Query, Zustand (lightweight state)
+Tooling: Ruff, mypy strict, pytest, pytest-cov, pre-commit, GitHub Actions, Docker Compose
+Deployment target (later): Hetzner. Local-first for v0.
+
+Repo structure
+remodel/
+├── packages/
+│   ├── engine/                 Python package
+│   │   ├── pyproject.toml
+│   │   ├── src/remodel_engine/
+│   │   │   ├── __init__.py
+│   │   │   ├── schemas/        Pydantic models (single source of truth)
+│   │   │   ├── catalog/        Default values, CostItem catalog, profiles
+│   │   │   ├── generation/     solar.py, wind.py, bess_state.py
+│   │   │   ├── dispatch/       hybrid_rtc.py, mcp_settlement.py
+│   │   │   ├── commercial/     ppa.py, dsm.py, charges.py, losses.py
+│   │   │   ├── capex/          cost_items.py, phasing.py, idc.py
+│   │   │   ├── financial/      pnl.py, cfs.py, bs.py, depreciation.py, tax.py, working_capital.py
+│   │   │   ├── debt/           sizing.py, sculpting.py, schedule.py, compliance.py
+│   │   │   ├── irr/            metrics.py
+│   │   │   ├── solver/         tariff.py
+│   │   │   ├── scenarios/      runner.py, sweep.py
+│   │   │   ├── io/             parquet_io.py, excel_export.py
+│   │   │   └── cli.py          Typer-based CLI
+│   │   └── tests/
+│   │       ├── fixtures/       Gold scenarios from user's Excel
+│   │       ├── unit/
+│   │       └── integration/
+│   ├── api/
+│   │   ├── pyproject.toml
+│   │   └── src/remodel_api/
+│   │       ├── main.py
+│   │       ├── routers/        scenarios, sensitivities, dashboard, templates
+│   │       ├── workers/        Arq tasks
+│   │       ├── db/             SQLAlchemy models, migrations
+│   │       └── deps.py
+│   └── web/
+│       ├── package.json
+│       ├── next.config.mjs
+│       └── src/
+│           ├── app/
+│           │   ├── (dashboard)/
+│           │   ├── scenarios/
+│           │   └── api-types/  Auto-generated from OpenAPI
+│           ├── components/
+│           │   ├── DataGrid/
+│           │   ├── KpiCard/
+│           │   ├── Wizard/
+│           │   └── Charts/
+│           └── lib/
+├── docker-compose.yml
+├── .github/workflows/ci.yml
+├── PROJECT.md                  This file
+├── sprints/
+│   ├── SPRINT_00.md
+│   ├── SPRINT_01.md
+│   └── ...
+└── README.md
+Module specs (summary — full detail in sprint files)
+Generation
+
+Solar 25y: 8760 profile × scaling × DC losses × inverter × AC losses → clipping at MW_AC → availability → soiling → degradation Y1..Y25
+Wind 25y: power curve lookup at hub-height-corrected wind speed → wakes → electrical → availability → degradation
+BESS state: nameplate × capacity_factor(year) where capacity_factor = degradation_curve(cum_cycles) + augmentation_steps
+
+Dispatch (Hybrid RTC)
+
+Per timestamp: target = ppa_capacity (flat 24×7), gen = solar+wind, surplus → charge BESS or curtail/MCP, deficit → discharge BESS or shortfall (DSM)
+SOC update with RTE on charge, 1/RTE on discharge, aux load
+User toggle: surplus when BESS full → curtail (zero rev) OR MCP merchant sale (uses MCP forecast input)
+
+Commercial
+
+Revenue = injection × tariff × (1 - aux_loss) × (1 - tx_loss) × (1 - dsm_loss)
+DSM loss% absorbs RTC penalty exposure (per user requirement)
+Receivables, bad debt provision
+
+Capex (CostItem catalog + IDC)
+
+~30 standard CostItem records, user-extensible
+Tier 1 fields exposed prominently (module INR/Wp, BOS INR/Wp, WTG Cr/MW, WTG BOP, land Lakh/Acre + lease/yr); Tier 2/3 collapsed
+Phasing matrix: items × construction months, % per cell, rows sum to 100%
+Equity drawdown: cumulative %, allowed >100% temporarily (bridge), settles at 100% by COD
+Debt drawdown: cumulative %, monotonic, settles at 100% by COD
+IDC: monthly waterfall, fixed-point converging IDC + total cost + debt = consistent
+
+Financial (3-statement)
+
+P&L: Revenue → OpEx → EBITDA → Depreciation → EBIT → Interest → PBT → Tax → PAT
+CFS: indirect method, CFO/CFI/CFF
+BS: assets = liab + equity, reconciled within ₹1 each year
+Depreciation by asset class (book SLM + tax WDV 40% for plant)
+Tax: 115BAA hardcoded v0
+WC: receivable_days, payable_days, inventory_days
+
+Debt
+
+Sizing: 3 constraints (D:E cap, min DSCR, avg DSCR), take binding
+Schedule shapes: equal_principal, equal_installment, dscr_sculpted, custom_pct_vector, balloon
+Compliance: try sculpting first; if equity_irr below target, raise tariff; if infeasible, warn user
+
+Solver
+
+brentq on tariff in [2.0, 8.0] ₹/kWh
+Inner: debt sizing fixed-point
+Innermost: IDC fixed-point
+
+Sensitivities (the "frequent 7")
+
+CAPEX, Module price, WTG price, Interest rate, CUF, Tariff, DSCR target
+
+Pydantic schema sketch (canonical types)
+pythonclass CostItem(BaseModel):
+    id: str
+    name: str
+    category: Literal["BOS","HardCost","SoftCost","EPCOverhead","EPCMargin","FinancingCost"]
+    basis: Literal["PER_WP_DC","PER_MW_AC","PER_MWP_DC","PER_ACRE","PER_KWH_USD","PCT_OF_HARDCOST","ABS_INR_CR","PER_MW_SOLAR","PER_MW_WIND"]
+    value: float
+    fx_rate: Optional[float] = None
+    depr_class: Literal["Plant","BESS","Building","Land_NoDepr","LandLease_Amortized","Intangible","Capitalized_NoDepr","Expensed"]
+    escalation_pct: float = 0.0
+    phasing_id: str
+    attribution: Literal["SolarOnly","WindOnly","BESSOnly","Common"]
+
+class ScenarioInput(BaseModel):
+    project: ProjectInfo
+    solar: Optional[SolarConfig]
+    wind: Optional[WindConfig]
+    bess: Optional[BessConfig]
+    commercial: CommercialConfig
+    capex: CapexConfig                  # contains list[CostItem] + phasing curves + equity/debt curves
+    opex: OpexConfig
+    debt: DebtConfig                    # rate, tenor, target_dscr, schedule_shape, ...
+    tax: TaxConfig
+    solver: SolverConfig                # target_equity_irr OR fixed_tariff
+
+class ScenarioResult(BaseModel):
+    inputs: ScenarioInput
+    status: Literal["queued","running","success","failed"]
+    solved_tariff: Optional[float]
+    kpis: KpiSummary                    # equity_irr, project_irr, min_dscr, avg_dscr, rtc_cuf, capex, idc, ...
+    financials: Financials              # pnl_25y, cfs_25y, bs_25y as DataFrames
+    timeseries_uri: str                 # parquet path
+    debt_schedule: list[DebtYearRow]
+    warnings: list[str]
+    runtime_s: float
+Working agreements
+
+Always read PROJECT.md and the active SPRINT_XX.md at session start. The sprint file lists tasks; pick one, complete it, mark it done, commit.
+One task per commit. Commit message format: [S2-T03] Implement IDC fixed-point solver.
+Tests before features. Every public function gets a unit test. Every module gets an integration test. Coverage target ≥85%.
+Type strict. mypy strict mode. No Any except at JSON boundaries.
+Lint clean. Ruff pre-commit hook; CI fails on lint errors.
+Pydantic for all I/O. No raw dicts crossing module boundaries.
+No magic numbers. Defaults live in catalog/defaults.py.
+Comments explain WHY, not WHAT. Code shows what; comments show why a non-obvious decision was made.
+When in doubt, ask the user. Don't silently make architectural choices. Add a # DECISION NEEDED: comment with options.
+Excel parity is sacred. When a parity test fails, stop adding features. Debug the diff. Don't paper over with tolerance bumps.
+Don't optimize prematurely. Vectorize when slow. Pure-Python loops are fine for v0 if they pass tests.
+Frontend types come from backend. Run openapi-typescript in CI. Never hand-write API types.
+One DataGrid component. Don't create a second grid implementation, no matter how convenient.
+Reference scenarios live in tests/fixtures/. When the user supplies new ones, add them as new fixtures, never modify existing.
+README in every package. Setup steps, common commands, troubleshooting.
+
+Definition of Done (per sprint)
+
+All sprint tasks ticked off in SPRINT_XX.md
+All tests pass locally and in CI
+Coverage ≥85% on new code
+Parity gate (if applicable) passes
+Sprint demo runnable (CLI command or UI flow)
+Sprint retro added to SPRINT_XX.md (what worked, what didn't, carryover)
+
+When to bring back Opus
+Stop and ask the user to consult Opus when:
+
+A parity test fails by >0.5% and the cause isn't obvious in 30 minutes
+An architectural decision arises that wasn't covered in PROJECT.md or sprint specs
+A module is shaping up to be >800 LOC — likely needs decomposition
+Performance is >2× the budget for a sprint deliverable

sprints/SPRINT_00.md

@@ -0,0 +1,19 @@
+Goal: Empty end-to-end skeleton. User can submit a dummy scenario via UI, see it queued, see a dummy result. No real engine logic yet.
+Tasks:
+
+ S0-T01 Initialize monorepo with packages/engine, packages/api, packages/web directories.
+ S0-T02 Set up packages/engine with Poetry, Python 3.12, Ruff, mypy strict, pytest, pre-commit. Empty remodel_engine package with __init__.py exposing version.
+ S0-T03 Set up packages/api similarly. FastAPI app with /healthz returning {"status":"ok","version":"..."}.
+ S0-T04 Set up packages/web with create-next-app@latest --typescript --tailwind --app. Add shadcn/ui (pnpm dlx shadcn@latest init). Add Recharts, TanStack Query, Zustand.
+ S0-T05 Create docker-compose.yml with Redis service. Add docs to README.md for docker compose up.
+ S0-T06 Add Arq worker stub in packages/api/src/remodel_api/workers/. One dummy task run_dummy_scenario that sleeps 3s and returns {"id": ..., "result": "dummy"}.
+ S0-T07 Add SQLAlchemy + SQLite to API. One model: Scenario (id, name, status, inputs_json, kpis_json, created_at). Alembic migration for initial schema.
+ S0-T08 API endpoints: POST /api/scenarios (creates DB row, enqueues dummy job), GET /api/scenarios/{id} (returns row), GET /api/scenarios (list).
+ S0-T09 SSE endpoint: GET /api/scenarios/{id}/events streaming progress from Redis pub/sub. Worker publishes {stage, pct} events.
+ S0-T10 Frontend: home page with "New Dummy Scenario" button. Posts to API, redirects to /scenarios/[id]. Page shows status + SSE progress bar + final result.
+ S0-T11 GitHub Actions CI: install, lint (ruff), typecheck (mypy + tsc), test (pytest + jest if any), build web. Fail on any error.
+ S0-T12 Pre-commit hook: ruff, mypy, prettier, eslint.
+ S0-T13 Top-level README.md with setup instructions: clone, make setup, make dev. Makefile targets for common tasks.
+ S0-T14 OpenAPI → TypeScript types: add openapi-typescript to web package, run on dev start, output to src/app/api-types/.
+
+Definition of Done: make dev brings up Redis, API on :8000, web on :3000, worker. Click "New Dummy Scenario" on web, see status update, see "dummy" result. CI green.

sprints/SPRINT_01.md

@@ -0,0 +1,16 @@
+Goal: Solar + Wind 25-year generation simulation working as a pure Python module with CLI driver. Validated against one Excel scenario.
+Tasks:
+
+ S1-T01 Schema: SolarConfig, WindConfig, GenerationResult in engine/schemas/generation.py.
+ S1-T02 Catalog: 3 solar 8760 profiles (RJ, KA, GJ) as CSV in engine/catalog/profiles/solar/. 3 wind profiles. Add loader.
+ S1-T03 generation/solar.py: function simulate_solar(config: SolarConfig) -> pd.DataFrame returning 25y × 8760 rows with columns year, hour, dc_power_mw, ac_power_mw_pre_clip, ac_power_mw, degradation_factor.
+ S1-T04 Solar logic: scaling, DC losses, inverter η, AC losses, clipping at MW_AC, availability, soiling (monthly profile or flat), Y1 + 0.7%, Y2-25 + 0.5% degradation.
+ S1-T05 Tests: CUF reasonable (15-25% range for solar), clipping captures DC excess correctly, degradation factors monotonic.
+ S1-T06 generation/wind.py: power curve lookup, wind shear correction, wake losses, electrical, availability, degradation.
+ S1-T07 Wind tests: PLF reasonable (25-40%), edge cases (zero wind, gale wind = power=0).
+ S1-T08 generation/bess_state.py: function simulate_bess_capacity(config: BessConfig, cum_cycles_per_year: list[float]) -> pd.Series returning usable MWh per year. Note: dispatch happens elsewhere; this module only models physical capacity. Augmentation as input list of (year, additional_mwh).
+ S1-T09 CLI: remodel simulate-gen --input scenario.json --output gen.parquet.
+ S1-T10 Parity test: load tests/fixtures/nagasamudra_inputs.json, run, compare year-1 generation against Excel value (within 0.1%). User to provide expected output.
+ S1-T11 Documentation: packages/engine/docs/generation.md explaining the module and how to extend.
+
+Definition of Done: CLI generates 25y output for the user's reference scenario. Year-1 CUF matches Excel within 0.1%. All tests pass.

sprints/SPRINT_02.md

@@ -0,0 +1,18 @@
+Goal: Capex modeling including the full CostItem catalog and the IDC fixed-point solver. Critical sprint — IDC parity gate.
+Tasks:
+
+ S2-T01 Schema: CostItem, PhasingCurve, EquityCurve, DebtCurve, CapexConfig in engine/schemas/capex.py.
+ S2-T02 Catalog: ~30 default CostItems in engine/catalog/cost_items.py matching the user's Excel structure (BOS subcomponents, hard cost items, soft cost items). Each with realistic 2026-level defaults.
+ S2-T03 Catalog: phasing templates solar_standard_18mo, wind_standard_24mo, hybrid_rtc_36mo with reasonable distributions per item.
+ S2-T04 capex/cost_items.py: function to compute total capex in INR Cr from a list of CostItems given project capacity, FX rate, etc. Each basis (PER_WP_DC, PER_MW_AC, etc.) has its own conversion.
+ S2-T05 capex/phasing.py: validation (rows sum to 100%), template loader, custom override.
+ S2-T06 capex/idc.py: the fixed-point IDC solver. Algorithm in PROJECT.md. Iterates until debt converges within ₹0.01 Cr.
+ S2-T07 Test: deterministic 2-month construction with hand-computed expected IDC. Solver matches.
+ S2-T08 Test: 36-month hybrid RTC scenario with equity bridge pattern (cum equity going to 110% then back to 100%). Solver matches.
+ S2-T09 Test: zero debt → zero IDC. Edge case.
+ S2-T10 CLI: remodel compute-idc --input scenario.json --output idc.json.
+ S2-T11 PARITY GATE: load tests/fixtures/nagasamudra_inputs.json. Run IDC. Total IDC must match user's Excel within 0.5%. User to walk Sonnet through the Apr-Jun-27 negative cum equity rows if mismatch.
+ S2-T12 Documentation: packages/engine/docs/capex.md covering CostItem model, phasing, IDC algorithm.
+
+Definition of Done: IDC parity gate passed. Catalog has 30 items with sensible defaults. Phasing templates work. CLI runs end-to-end.
+ESCALATION: If IDC parity fails by >0.5% after 4 hours of debug, stop and consult Opus with the diff.

sprints/SPRINT_03.md

@@ -0,0 +1,15 @@
+Goal: Full 3-statement model (P&L, CFS, BS) for 25 years with depreciation, tax, working capital. Parity gate on statements.
+Tasks:
+
+ S3-T01 Schema: OpexConfig, TaxConfig, Financials, PnLRow, CFSRow, BSRow.
+ S3-T02 financial/depreciation.py: book SLM by asset class (Plant 25y, BESS 12y, Building 30y, Intangible 25y, Land never), tax WDV 40% for plant. IDC capitalized proportionally.
+ S3-T03 financial/tax.py: 115BAA computation. PBT × 25.17%. Deferred tax from book/tax depreciation diff.
+ S3-T04 financial/working_capital.py: receivables, payables, inventory based on days inputs.
+ S3-T05 financial/pnl.py: 25-row P&L from revenue + opex + depreciation + interest schedule (passed in) + tax.
+ S3-T06 financial/cfs.py: CFO from PAT + depr - ΔWC; CFI from capex; CFF from debt + equity flows.
+ S3-T07 financial/bs.py: gross block, accumulated depr, net block, cash, receivables, equity, reserves, debt, payables, deferred tax. Reconciliation assertion.
+ S3-T08 Tests: each module unit tested. Integration test: dummy scenario, statements reconcile.
+ S3-T09 PARITY GATE: load nagasamudra_inputs.json, run with user's actual tariff. Each P&L line for years 1, 5, 10, 15, 25 within 0.5% of Excel. BS reconciles each year.
+ S3-T10 Documentation.
+
+ESCALATION: Parity miss > 0.5% on any statement line → Opus.