Architecture · LikeC4
Code Intelligence Digest
An Inoreader-fed curation pipeline that scores, ranks, and packages code-intelligence content — code search, agents, retrieval, dev tools — into weekly & monthly digests, GTM agent reports, newsletters, and podcasts, served by a Next.js app and a read-only MCP copilot.
production
evolving — built, still moving
research — speculative track
risk — open question
The whole picture
System landscape
The digest in context of its content sources and the inference, delivery, and hosting services it depends on — then the same system opened up into its containers.
In context
indexThe system against Inoreader and NASA ADS (sources), the LLM / embedding / TTS / web-search services, Resend (email), LangSmith (traces & evals), and Render (hosting).
Containers
digestSystemEight pieces: ingestion & sync, the scoring & ranking pipeline, the Postgres store, the LLM provider gateway, the GTM agent layer, the media renderers, the Next.js web/API, and the MCP copilot.
Inside each box
Containers, decomposed
Each container opened to the components that implement it. Every box links to its source in the interactive explorer.
Ingestion & sync
TypeScript / NodePulls feed items and research papers in with quota and resumability guards — OAuth2 client, the daily-sync orchestrator, a sync-state circuit breaker, ADS research sync, and an API budget guard.
Scoring & ranking
TypeScriptThe hybrid LLM + BM25 + recency scorer and the diversity-aware ranker that turn normalized, categorized items into a digest.
Postgres store
PostgreSQL 16 · pgvectorThe connection pool plus repositories over items & precomputed scores (tsvector full-text), pgvector embeddings, agent runs & generated media, operator relevance tuning, and OAuth tokens & API budget.
LLM provider gateway
TypeScriptRoutes completions to Anthropic or OpenAI by model id, with timeouts, usage accounting, and a quality-model resolver.
GTM agent layer
evolvingRetrieves from the corpus plus live web search, ranks for a goal, and writes competitive-intel, market-brief, and content-ideas reports — traced to LangSmith.
Media renderers
TypeScriptTurn a digest into newsletters, podcasts/audio (via the TTS providers), and PDF exports.
Next.js web + API
Next.js 16The React reader/admin UI plus the REST API — auth via next-auth + middleware, with admin/debug routes blocked in production.
MCP copilot
evolvingRead-only MCP tools over the local production mirror, so a coding agent can query the digest corpus.
How it runs
Walkthrough flows
Dynamic views — the narrative spine of the system, step by step.
Daily sync & scoring
The Render cron job, tuned for a 512 MB container. A throttled run pauses on the sync_state row rather than failing, and resumes from the continuation token.
Serving a ranked digest
Ranking is on-demand and stateless: load precomputed scores, apply the subtle recency term, filter by min-relevance, then diversity-sample a per-source-capped digest.
A GTM agent report
On cron or from the Reports UI (direct POST is blocked in production). The curator trace is persisted with the run so /reports/[id] can replay the config, retrieval, and ranking.
MCP copilot query
Read-only tools over the hourly mirror. The agent calls mirror_status before any "today / this week" claim, so it can warn when the mirror is stale.
What to probe
Risks, planned work & deployment
The open questions a reviewer should surface, the speculative tracks, and where each piece actually runs.
Risks & open questions
riskInoreader token-refresh fragility, no cross-provider LLM fallback, curator-trace schema-version compatibility, and the free-tier single-instance Postgres.
Planned & research
researchEvolving and speculative tracks — engagement/citation scoring, the MCP Phase B tools, and research deep-dives — with built dependencies dimmed.
Deployment
deploymentRender web service + cron jobs + managed Postgres, the local production mirror feeding the MCP copilot, and the external inference, search, and delivery services.
Explore it live
The figures above are static exports. The interactive explorer lets you pan, zoom, follow relationships, and jump from any box to the source.