How LearnixSai is built.
LearnixSai is a curation-first learning platform: type a topic and it assembles a structured course out of real videos, articles, docs and Q&A — not AI-generated lesson text. This is the full engineering reference: architecture, algorithms, the API, AI integration, and how it all deploys. Diagrams and flows are interactive — click around.
System topology
Four independently-hosted pieces, deliberately decoupled so no single failure takes everything down. Click a box to see what it is and why it's there.
Click any box to see what it is, where it runs, and why it's there. Animated lines show live request paths.
Decouple by failure domain. The app, the search engine, and the status page each run on separate infrastructure. SearXNG going down degrades one source, not the app. The status page lives on Vercel so it can report an outage even while Heroku is on fire.
One database, many roles. Postgres holds application data, the pg-boss job queue (its own schema), and the rate-limit table. No Redis, no extra managed services — fewer moving parts to pay for and monitor.
Architecture decisions & why
The choices that shaped the system — each with the context that forced it and the trade-off it carries. These are the load-bearing decisions.
Curate real content, don't generate lesson text
USPGeneration is a background job, not an HTTP request
pg-bossPOST enqueues a durable pg-boss job and returns in ~50ms; the worker runs the pipeline and the frontend polls a job row.Run the worker in-process, not on a second dyno
costinstrumentation.ts starts the pg-boss worker inside the web dyno at boot, capped by GENERATION_CONCURRENCY.Hybrid models — free to classify, paid to build
latencySelf-host SearXNG on a free VM instead of a paid search API
$0SEARXNG_URL.Keyless structured APIs over web scraping
reliabilityPostgres-backed rate limiter, not in-memory
correctnessINSERT … ON CONFLICT upsert on a RateLimit table enforces limits globally.Keep the status page on separate infrastructure
opsCourse generation pipeline
Generation is a background job, not part of the HTTP request. The old design ran the whole multi-minute pipeline inside an SSE request — slow, and it broke under load. Step through the current flow:
Enqueue & return
The request does the bare minimum: auth, a Postgres rate-limit check, create a Course (status generating) and a 1:1 GenerationJob, enqueue to pg-boss, and return { courseId }. No work happens in the request — it returns in milliseconds.
POST /api/courses/generate
auth + DB rate-limit
create Course(status:"generating") + GenerationJob
boss.send("course-generation", { courseId, topic, level, userId })
→ 202 { courseId }Why it's durable: pg-boss coordinates through Postgres with SKIP LOCKED. If the dyno restarts mid-job (Heroku cycles daily), the job is retried on next boot — nothing is lost, and it's safe even if the app scales to multiple web dynos.
Feature workflows
How each user-facing feature is actually built, end to end. Pick a feature:
The flagship loop. Covered in depth under Generation pipeline — in short: enqueue → classify → structure → parallel curation → persist, with the frontend polling a job row. The important design move is that nothing generative happens in the request; it's a durable background job with a global concurrency cap.
Algorithms
The non-obvious logic that makes curation actually good — routing, filtering, reranking, balancing, and racing.
lib/source-router.ts maps the topic class to a set of keyless APIs, so a cooking course doesn't get Stack Overflow and a Docker course doesn't get arXiv.
switch (topicClass) {
developer_tech: dev.to(tag) + HackerNews + StackOverflow + GitHub
business_founder: HackerNews + dev.to
academic_science: arXiv + Stack Exchange (physics/stats/…)
history_humanities: Stack Exchange (history/philosophy/…)
practical_skill: Stack Exchange topic site (guitar→music.se,
cooking→cooking.se … keyword map)
}
// Wikipedia + SearXNG web search run for every classRemoved after live testing: wikiHow (search API returns 500) and Reddit (403 from datacenter IPs). Brave Search was rejected — it requires a credit card even on the free tier.
API system
The routes that matter, from generation to health. Click to expand request/response shapes.
{ topic, level, mode?, pathNodeId? }202 { courseId } — returned in ~50ms; work happens in the background worker.AI integration
Every model call, why it's on the tier it's on, and the prompts behind them.
Cheap, simple, tolerant of latency
- Topic classification — one label + a module estimate.
- Article summaries — lazy, cached, off the critical path.
Fast, not globally rate-limited
- Structure — course title + ordered modules.
- Rerank — the per-module editor pass.
Why hybrid
Free OpenRouter models share a global account rate limit — running the whole pipeline on them meant 429 storms the moment two users generated at once, and long queues. Moving the latency-critical, on-the-hot-path calls (structure + rerank) to a paid fast model (Gemini Flash, ~$0.001–0.01/course) killed both the latency and the 429s, while classify + summaries stay free. Models are env-overridable via PIPELINE_PAID_MODELS, and the client cycles through a fallback list on error.
The prompts
classifier.tsTopic → class + module estimate
curator.ts · structureClass + topic → title + modules
curator.ts · buildRerankPromptCandidates → { keep: [indices] }
summary.tsArticle text → { summary, keyPoints }
quiz promptModule resources → questions
Keys never touch the client. All model calls are server-side; the OpenRouter key lives only in Heroku config vars. No tokens in cookies or the frontend bundle.
Data model
17 Prisma models on PostgreSQL, ID strategy cuid() throughout. Filter by domain and click any entity for its real fields, types and relations.
Relationship map
Everything hangs off User and Course. A course owns its modules and resources; a module optionally owns a quiz; progress and XP are per-user.
Entity explorer
User
CoreThe learner and their progression state. NextAuth credentials + gamification live on the same row.
| id | String | PK |
| String? | UNIQUE | |
| username | String? | UNIQUE |
| password | String? | |
| xp | Int @default(0) | |
| level | Int @default(1) | |
| streakCount | Int @default(0) | |
| longestStreak | Int @default(0) | |
| lastActive | DateTime? | |
| avatar | String @default('default') | |
| avatarFrame | String @default('none') | |
| title | String @default('Novice') | |
| characterData | Json? | |
| createdAt | DateTime |
Status & enum values
These are string columns (no DB enums), so the valid values live in code. The ones that matter:
generating · active · completed · failed — listings hide generating + failed.queued · running · done · failedclassifying · structure · module · doneyoutube · article · wikipedia · paper · ai_generatednot_started · in_progress · completeddeveloper_tech · academic_science · business_founder · history_humanities · practical_skilllocked · active · completedbeginner · intermediate · advancedOne database, three roles. These models live alongside the pg-boss queue (its own pgboss schema) and the RateLimit table on a single Heroku Postgres. Prisma 7 with the @prisma/adapter-pg driver adapter; migrations applied via prisma migrate deploy on release.
Deployment · CI/CD
The app ships through GitHub Actions on every push to main — no manual deploys. Step through the pipeline:
Push to main
A merge/push to main triggers the deploy workflow in .github/workflows/deploy.yml. Feature branches don't deploy.
This ops site deploys separately. It's its own repo on Vercel — pushing auto-deploys it. It was split out of the main repo because the parent's Tailwind/PostCSS config leaked into the subfolder build.
Deployment · servers
Where everything physically runs, and the constraints that shaped each host.
Next.js App + Worker
Heroku · Docker- Runtime
- Standalone Docker image, App Router
- Worker
- pg-boss, in-process (no 2nd dyno)
- Why not Vercel
- Serverless can't host a long-lived worker
- Deploy
- GitHub Actions → migrate → release
PostgreSQL
Heroku Postgres- App data
- All core + gamification models
- Queue
- pg-boss
pgbossschema - Limiter
- RateLimit table (cross-dyno)
- Migrations
- prisma migrate deploy on release
SearXNG
Oracle Cloud · always-free VM- Shape
- VM.Standard.E2.1.Micro, Ubuntu
- Cost
- $0 / month (always-free tier)
- Run
- Docker, restart: unless-stopped
- Caveat
- Ephemeral public IP — reserve or update env if it changes
Status + Docs
Vercel- Role
- Live status board + these docs
- Isolation
- Separate repo, off Heroku on purpose
- Checks
- /api/health + frontend + SearXNG
- Alerts
- UptimeRobot can watch /api/health