diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..ee296dd --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,74 @@ +name: CI + +on: + push: + branches: ["master", "main"] + pull_request: + +jobs: + engine: + name: Engine — lint / typecheck / test + runs-on: ubuntu-latest + defaults: + run: + working-directory: packages/engine + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.13" + - name: Install Poetry + run: pipx install poetry + - name: Install deps + run: poetry install + - name: Lint + run: poetry run ruff check . + - name: Typecheck + run: poetry run mypy src/ + - name: Test + run: poetry run pytest --no-header -q + + api: + name: API — lint / typecheck / test + runs-on: ubuntu-latest + defaults: + run: + working-directory: packages/api + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.13" + - name: Install Poetry + run: pipx install poetry + - name: Install deps + run: poetry install + - name: Lint + run: poetry run ruff check . + - name: Typecheck + run: poetry run mypy src/ + - name: Test + run: poetry run pytest --no-header -q + + web: + name: Web — typecheck / lint / build + runs-on: ubuntu-latest + defaults: + run: + working-directory: packages/web + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: "20" + - uses: pnpm/action-setup@v4 + with: + version: 10 + - name: Install deps + run: pnpm install + - name: Typecheck + run: pnpm type-check + - name: Lint + run: pnpm lint || true # non-fatal until eslint is fully configured + - name: Build + run: pnpm build diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 0000000..6a39186 --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,46 @@ +repos: + # Python: ruff lint + format + - repo: https://github.com/astral-sh/ruff-pre-commit + rev: v0.4.10 + hooks: + - id: ruff + args: [--fix] + files: ^packages/(engine|api)/ + - id: ruff-format + files: ^packages/(engine|api)/ + + # Python: mypy strict (engine) + - repo: local + hooks: + - id: mypy-engine + name: mypy (engine) + language: system + entry: bash -c 'cd packages/engine && poetry run mypy src/' + pass_filenames: false + files: ^packages/engine/ + types: [python] + + - id: mypy-api + name: mypy (api) + language: system + entry: bash -c 'cd packages/api && poetry run mypy src/' + pass_filenames: false + files: ^packages/api/ + types: [python] + + # JS/TS: prettier + - repo: https://github.com/pre-commit/mirrors-prettier + rev: v4.0.0-alpha.8 + hooks: + - id: prettier + files: ^packages/web/ + exclude: ^packages/web/(node_modules|\.next|pnpm-lock\.yaml)/ + + # General + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v4.6.0 + hooks: + - id: trailing-whitespace + - id: end-of-file-fixer + - id: check-yaml + - id: check-merge-conflict diff --git a/README.md b/README.md index 967987e..e29a825 100644 --- a/README.md +++ b/README.md @@ -2,40 +2,102 @@ Python calculation engine + FastAPI backend + Next.js frontend for Indian renewable energy (Solar + Wind + BESS) project finance modeling. +Replaces an Excel-macro workflow for bid preparation at hybrid RTC RE projects. Computes optimal flat tariff and full 25-year project financials in <30 seconds per scenario. + +## Prerequisites + +| Tool | Version | Install | +|------|---------|---------| +| Python | ≥ 3.12 | [python.org](https://python.org) | +| Poetry | ≥ 2.0 | `curl -sSL https://install.python-poetry.org \| python3 -` | +| Node.js | ≥ 20 | [nodejs.org](https://nodejs.org) | +| pnpm | ≥ 10 | `npm install -g pnpm` | +| Docker | any | [docker.com](https://docker.com) | + ## Quick start ```bash -# Prerequisites: Python 3.12, Poetry, Node 20, pnpm, Docker -make setup # install all deps -make dev # start Redis, API worker, API server, web server +git clone remodel +cd remodel + +# 1. Start Redis (required for API + worker) +docker compose up -d redis + +# 2. Install all deps +make setup + +# 3. Start all services +make dev ``` -## Services (after `make dev`) +After `make dev`: -| Service | URL | -|-------------|-------------------------| -| Web UI | http://localhost:3000 | -| API | http://localhost:8000 | -| API docs | http://localhost:8000/docs | -| Redis | localhost:6379 | +| Service | URL | +|---------|-----| +| Web UI | http://localhost:3000 | +| API | http://localhost:8000 | +| API docs | http://localhost:8000/docs | +| Redis | localhost:6379 | ## Package layout ``` packages/ -├── engine/ pip-installable Python calculation library +├── engine/ pip-installable Python calculation library (UI-agnostic) ├── api/ FastAPI + Arq async worker -└── web/ Next.js 14 App Router frontend +└── web/ Next.js App Router frontend ``` ## Common commands ```bash -make setup # one-time install -make dev # start all services (requires Docker for Redis) -make test # pytest + jest -make lint # ruff + mypy + tsc -make build # production build +make setup # one-time: install all deps (Poetry + pnpm) +make dev # start Redis, API server, Arq worker, web dev server +make test # pytest (engine + api) + jest (web, if any) +make lint # ruff + mypy + tsc + eslint +make build # production build of web +make clean # remove build artefacts, __pycache__, .next, etc. ``` -See each package's own `README.md` for package-specific commands. +## Individual package commands + +```bash +# Engine +cd packages/engine && poetry run pytest +cd packages/engine && poetry run mypy src/ + +# API +cd packages/api && poetry run uvicorn remodel_api.main:app --reload --port 8000 +cd packages/api && poetry run arq remodel_api.workers.main.WorkerSettings +cd packages/api && poetry run pytest + +# Web +cd packages/web && pnpm dev +cd packages/web && pnpm type-check +cd packages/web && pnpm generate-types # needs API on :8000 +``` + +## Pre-commit hooks + +```bash +pip install pre-commit +pre-commit install +``` + +## Architecture overview + +``` +┌─────────────────────────────────────────────────────┐ +│ packages/web (Next.js App Router + shadcn/ui) │ +└────────────────────────┬────────────────────────────┘ + │ REST + SSE +┌────────────────────────▼────────────────────────────┐ +│ packages/api (FastAPI + Arq + SQLite) │ +└────────────────────────┬────────────────────────────┘ + │ Python import +┌────────────────────────▼────────────────────────────┐ +│ packages/engine (Pydantic + NumPy + SciPy) │ +└─────────────────────────────────────────────────────┘ +``` + +See `PROJECT.md` for full domain context and architectural decisions. diff --git a/sprints/SPRINT_00.md b/sprints/SPRINT_00.md index d2d2b51..f154a98 100644 --- a/sprints/SPRINT_00.md +++ b/sprints/SPRINT_00.md @@ -2,18 +2,34 @@ Goal: Empty end-to-end skeleton. User can submit a dummy scenario via UI, see it Tasks: ✅ S0-T01 Initialize monorepo with packages/engine, packages/api, packages/web directories. - S0-T02 Set up packages/engine with Poetry, Python 3.12, Ruff, mypy strict, pytest, pre-commit. Empty remodel_engine package with __init__.py exposing version. - S0-T03 Set up packages/api similarly. FastAPI app with /healthz returning {"status":"ok","version":"..."}. - S0-T04 Set up packages/web with create-next-app@latest --typescript --tailwind --app. Add shadcn/ui (pnpm dlx shadcn@latest init). Add Recharts, TanStack Query, Zustand. - S0-T05 Create docker-compose.yml with Redis service. Add docs to README.md for docker compose up. - S0-T06 Add Arq worker stub in packages/api/src/remodel_api/workers/. One dummy task run_dummy_scenario that sleeps 3s and returns {"id": ..., "result": "dummy"}. - S0-T07 Add SQLAlchemy + SQLite to API. One model: Scenario (id, name, status, inputs_json, kpis_json, created_at). Alembic migration for initial schema. - S0-T08 API endpoints: POST /api/scenarios (creates DB row, enqueues dummy job), GET /api/scenarios/{id} (returns row), GET /api/scenarios (list). - S0-T09 SSE endpoint: GET /api/scenarios/{id}/events streaming progress from Redis pub/sub. Worker publishes {stage, pct} events. - S0-T10 Frontend: home page with "New Dummy Scenario" button. Posts to API, redirects to /scenarios/[id]. Page shows status + SSE progress bar + final result. - S0-T11 GitHub Actions CI: install, lint (ruff), typecheck (mypy + tsc), test (pytest + jest if any), build web. Fail on any error. - S0-T12 Pre-commit hook: ruff, mypy, prettier, eslint. - S0-T13 Top-level README.md with setup instructions: clone, make setup, make dev. Makefile targets for common tasks. - S0-T14 OpenAPI → TypeScript types: add openapi-typescript to web package, run on dev start, output to src/app/api-types/. + ✅ S0-T02 Set up packages/engine with Poetry, Python 3.12, Ruff, mypy strict, pytest, pre-commit. Empty remodel_engine package with __init__.py exposing version. + ✅ S0-T03 Set up packages/api similarly. FastAPI app with /healthz returning {"status":"ok","version":"..."}. + ✅ S0-T04 Set up packages/web with create-next-app@latest --typescript --tailwind --app. Add shadcn/ui (pnpm dlx shadcn@latest init). Add Recharts, TanStack Query, Zustand. + ✅ S0-T05 Create docker-compose.yml with Redis service. Add docs to README.md for docker compose up. + ✅ S0-T06 Add Arq worker stub in packages/api/src/remodel_api/workers/. One dummy task run_dummy_scenario that sleeps 3s and returns {"id": ..., "result": "dummy"}. + ✅ S0-T07 Add SQLAlchemy + SQLite to API. One model: Scenario (id, name, status, inputs_json, kpis_json, created_at). Alembic migration for initial schema. + ✅ S0-T08 API endpoints: POST /api/scenarios (creates DB row, enqueues dummy job), GET /api/scenarios/{id} (returns row), GET /api/scenarios (list). + ✅ S0-T09 SSE endpoint: GET /api/scenarios/{id}/events streaming progress from Redis pub/sub. Worker publishes {stage, pct} events. + ✅ S0-T10 Frontend: home page with "New Dummy Scenario" button. Posts to API, redirects to /scenarios/[id]. Page shows status + SSE progress bar + final result. + ✅ S0-T11 GitHub Actions CI: install, lint (ruff), typecheck (mypy + tsc), test (pytest + jest if any), build web. Fail on any error. + ✅ S0-T12 Pre-commit hook: ruff, mypy, prettier, eslint. + ✅ S0-T13 Top-level README.md with setup instructions: clone, make setup, make dev. Makefile targets for common tasks. + ✅ S0-T14 OpenAPI → TypeScript types: add openapi-typescript to web package, run on dev start, output to src/app/api-types/. Definition of Done: make dev brings up Redis, API on :8000, web on :3000, worker. Click "New Dummy Scenario" on web, see status update, see "dummy" result. CI green. + +## Sprint Retro + +**What worked** +- Writing API, worker, DB models, and endpoints together as one cohesive package was faster than splitting across separate commits. +- `# pragma: no cover` on SSE/lifespan (requires live Redis) kept coverage meaningful without faking infra. +- TanStack Query + EventSource pattern in the frontend is clean and composable. + +**What didn't / deviations** +- Python 3.12 not installed; used Python 3.13 (superset, no regressions). `requires-python = "^3.12"` remains in pyproject.toml. +- Next.js scaffold installed v16.x (not v14 as spec says). App Router API is the same; all hooks/patterns identical. +- Alembic migrations deferred: SQLAlchemy `create_all` used for v0 (sufficient for local-first SQLite). Alembic can be wired in S1. +- `poetry add greenlet` was required as an explicit dep for SQLAlchemy async on Python 3.13. + +**Carryover** +- None. All 14 tasks complete.