We Built an Engineering & Security News Wire — Here's How (and Why It Costs $0/Month)

We just shipped a curated engineering & security news feed built directly into Clawship. It aggregates the top stories from Hacker News, Lobsters, Krebs on Security, the GitHub Blog, and six other high-signal sources — updated every six hours, with zero manual curation.

You can read it right now at clawship.app/news. No login required. Here’s why we built it, how it works under the hood, and what comes next.

Why Build a News Feed Into a SaaS?

We read a lot of engineering and security content. Hacker News, Lobsters, various security blogs — the usual rotation. But keeping up means juggling RSS readers, bookmarks, and browser tabs across ten different sites.

We wanted a single page that answered: “What happened in engineering and security today?” — without the noise of general tech news. No product launches. No gadget reviews. Just software engineering, infrastructure, and security.

Building it inside Clawship made sense for three reasons:

1. Eat our own cooking. A real-time aggregation pipeline is a great stress test for our own stack.
2. Give back something useful. Every developer we know reads this content. Making it freely accessible — no login, no paywall — is a good way to be useful.
3. Feed the AI. Our assistants can reference this data for daily briefings, security alerts, and trend summaries. (That part comes next.)

10 Sources, 3 Categories

We evaluated dozens of feeds and settled on ten that consistently produce high-signal content. Each article is auto-categorized into one of three buckets:

Every source was chosen for its signal-to-noise ratio. We deliberately excluded general tech outlets (TechCrunch, The Verge, Wired) because they dilute the feed with content our audience doesn’t need. If an important story breaks, it will surface through one of the engineering or security sources anyway.

How It Works: Architecture

The system has three components: an aggregator, a scheduler, and an API. Here’s the high-level flow:

The Aggregator

The aggregator service fetches all RSS feeds in parallel using rss-parser. For Hacker News, we use the Firebase API instead — it’s faster and gives us the top 20 stories directly.

Each article is deduplicated by hashing the URL. If the same story appears on both Hacker News and Lobsters (which happens constantly), only the first one is stored. This eliminates ~30% of raw feed volume.

Category assignment is source-based: Krebs and NIST are always security, OpenAI and Anthropic are ai, and the rest default to programming. Simple, deterministic, no ML required.

The Scheduler

A lightweight in-process scheduler ticks every 6 hours. No cron jobs, no external queue — just a simple timer that fires the aggregator. It runs an initial tick on startup so fresh deployments populate immediately.

Why not a cron job? Because this is a single-instance service. There’s no distributed coordination problem to solve. Adding a task queue would be engineering theater.

The API

A single route serves the feed with pagination, category filtering, and cache headers:

{
  "ok": true,
  "data": {
    "articles": [...],
    "pagination": {
      "total": 847,
      "hasMore": true
    }
  }
}

Articles are sorted newest-first and filtered to the last 7 days. The response includes CDN-friendly cache headers so the page stays fast without serving stale content.

The Frontend: Terminal-Editorial Design

We wanted the page to feel like a terminal dashboard crossed with a newspaper wire. Key design decisions:

• No images. Every article is a compact row: index number, category tag (ENG/SEC/AI), title, source with icon, domain, and time ago. Dense information, no decoration.
• Sticky date headers. Articles are grouped by publication date with sticky headers that persist while scrolling. You always know which day you’re looking at.
• 3-digit index. Each article has a padded number (001, 002, ...) that serves as a visual anchor. It’s a small detail borrowed from wire services.
• Live Feed indicator. A pulsing emerald dot with “Live Feed” text signals that this is an auto-updating source, not a static page.
• Stats strip. Quick counts for ENG, SEC, AI, and TOTAL articles in the current page. Gives you a category breakdown at a glance.

All links point to the original source — we don’t republish content or wrap URLs in tracking redirects. Every click goes directly to the author’s site.

Cost: $0/month

The entire pipeline runs at zero marginal cost:

• RSS feeds: Free, public, no API keys
• Hacker News API: Free, no authentication, no rate limits
• Database: Already running for the main app — one extra table
• Compute: The aggregator runs in the existing backend process

We evaluated NewsAPI.org ($449/mo) and GNews ($50/mo) before realizing RSS + the HN Firebase API covers our use case better — with zero ongoing cost and no vendor dependency.

What Comes Next

This is v1. Here’s what we’re building next:

Try It

The news wire is live at clawship.app/news. No account needed. Bookmark it, add it to your daily rotation, or just check it when Hacker News is down.

If you want the AI assistant features — daily briefings, security alerts, multi-channel deployment — check out our plans starting at $12/month. Or deploy a free Telegram bot by joining our Discord.