How to Export Hacker News Threads and Comments for Analysis

Quick Answer: The fastest way to export Hacker News comments is with the Comment Exporter Chrome extension. Navigate to any HN thread, click the extension, and download comments as CSV or JSON — including usernames, timestamps, points, and reply depth. No coding, no API wrangling, no recursive fetch scripts.

Why Hacker News Data Matters

Hacker News is the tech industry's watercooler. Discussions span hiring trends, technology adoption, startup launches, funding rounds, and product feedback. The commenters are not anonymous hobbyists — they are engineers at FAANG companies, startup founders, venture capitalists, and open-source maintainers.

That makes HN comment data unusually high-signal. A thread about a new developer tool might contain feedback from 200 engineers who have actually used similar tools. A "Who is hiring?" post contains thousands of real job listings with salary ranges and tech stacks. An "Ask HN" thread about burnout might surface patterns across dozens of companies.

But Hacker News does not offer any built-in export. You can read comments in your browser, one thread at a time. No download button, no CSV option, no bulk access from the UI. This guide covers three methods for exporting HN threads into structured data — a Chrome extension, the HN Firebase API, and Google BigQuery.

Who Needs Hacker News Data?

• ✓Developer tool companies: Track how engineers discuss your product — and your competitors. A single HN thread about "best CI/CD tools" can contain more honest feedback than a dozen customer interviews.
• ✓Recruiters and HR teams: The monthly "Who is hiring?" threads are the largest public dataset of tech job postings — internal culture descriptions, actual tech stacks, and candid comp ranges that job boards do not surface.
• ✓Venture capitalists: Monitor what technologies founders and engineers are excited about — or abandoning. HN sentiment often leads broader market adoption by 6-12 months.
• ✓Tech journalists: When a major outage, layoff, or product launch happens, HN threads fill with insider perspectives within hours.
• ✓Academic researchers: Study tech industry discourse, developer sentiment, and online community behavior with a dataset spanning over 15 years.

What Data Can You Extract from Hacker News?

Each Hacker News comment carries structured metadata beyond the text itself. Here is what is available:

The combination of points and author data is what makes HN comments worth exporting. A 150-point comment from a known YC founder carries different weight than a 1-point reply from a new account. Structured data lets you filter for signal.

Method 1: Chrome Extension (Comment Exporter)

The Comment Exporter Chrome extension supports Hacker News alongside 10 other platforms — including Reddit, YouTube, Amazon, Steam, and more. It works directly in your browser with no coding and no API setup.

Step-by-Step Walkthrough

1. ✓Install the extension: Visit the Comment Exporter page on the Chrome Web Store and click "Add to Chrome." Installation takes about 10 seconds.
2. ✓Navigate to any HN thread: Open a Hacker News discussion page — for example, news.ycombinator.com/item?id=.... This works on regular link posts, Ask HN threads, Show HN threads, and any other post type with comments.
3. ✓Open the extension and start scraping: Click the Comment Exporter icon in your browser toolbar. The extension detects the HN page and begins collecting comments — capturing text, usernames, timestamps, points, and reply depth.
4. ✓Choose your format and download: Select CSV or JSON and save the file. CSV opens directly in Excel or Google Sheets. JSON preserves the nested thread structure if you need parent-child relationships for analysis.

Nested comment threads get exported as flat CSV — one row per comment with a depth column — or as structured JSON that preserves the reply hierarchy. A typical HN thread with 300 comments exports in under 30 seconds.

Pros

• ✓No coding or API setup required
• ✓Captures all metadata — username, timestamp, points, reply depth
• ✓Exports nested threads as flat CSV or structured JSON
• ✓Works on any HN thread URL, including Ask HN and Show HN
• ✓One-click export to CSV or JSON
• ✓Also works on 10 other platforms with the same extension

Cons

• ✓Requires the Chrome browser (not available on Firefox or Safari)
• ✓HN scraping requires the All Access plan ($49.99/mo)

Best for: Researchers, marketers, recruiters, and analysts who need structured HN comment data without writing code. The Hacker News Scraper page has more details on what the extension captures.

Method 2: Hacker News API (Firebase)

Hacker News has a free public API built on Firebase. No authentication required — no API keys, no OAuth, no registration. You hit an endpoint and get JSON back.

How It Works

The API exposes every item on HN — stories, comments, polls, jobs — through a single endpoint pattern:

GET https://hacker-news.firebaseio.com/v0/item/{id}.json

Each item has a numeric ID. A story item contains a kids array listing the IDs of its direct child comments. Each comment item also has a kids array for its replies. To fetch a full thread, you start with the story ID, get its children, then recursively fetch each child's children.

Sample Python Code

import requests
import csv
import time

BASE_URL = "https://hacker-news.firebaseio.com/v0/item/{}.json"

def fetch_item(item_id):
    """Fetch a single HN item by ID."""
    resp = requests.get(BASE_URL.format(item_id))
    return resp.json()

def fetch_comments(item_id, depth=0):
    """Recursively fetch all comments in a thread."""
    item = fetch_item(item_id)
    if item is None or item.get("deleted") or item.get("dead"):
        return []

    comments = []
    if item.get("type") == "comment":
        comments.append({
            "id": item.get("id"),
            "author": item.get("by", "[deleted]"),
            "text": item.get("text", ""),
            "time": item.get("time"),
            "depth": depth
        })

    for kid_id in item.get("kids", []):
        time.sleep(0.1)  # Rate limiting
        comments.extend(fetch_comments(kid_id, depth + 1))

    return comments

# Fetch all comments from a thread
story_id = 38009291  # Replace with any HN story ID
comments = fetch_comments(story_id)

# Write to CSV
with open("hn_comments.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=["id", "author", "text", "time", "depth"])
    writer.writeheader()
    writer.writerows(comments)

print(f"Exported {len(comments)} comments to hn_comments.csv")

This script works, but there is a catch. Each comment is a separate HTTP request. A thread with 500 comments means 500+ API calls — one for the story, one per comment, plus recursive calls for nested replies. With rate-limiting delays, that takes several minutes.

Pros

• ✓Completely free — no API key, no authentication, no rate limit tokens
• ✓Returns structured JSON with all item metadata
• ✓Fully programmable — schedule exports, pipe into databases, build dashboards
• ✓Access to every item ever posted on HN (stories, comments, polls, jobs)

Cons

• ✓Each comment is a separate API call — no bulk endpoint
• ✓Requires recursive fetching logic (the kids array only contains IDs, not full objects)
• ✓A 500-comment thread takes 500+ HTTP requests
• ✓Requires Python or similar coding knowledge
• ✓No built-in CSV output — you build your own export pipeline
• ✓Points (karma) for comments are not exposed in the API for most items

Best for: Developers building automated data pipelines or integrating HN data into internal tools and research workflows.

Method 3: BigQuery HN Dataset

Google maintains a public BigQuery dataset containing the full history of Hacker News — every story, comment, poll, and job posting since 2006. It is updated regularly and free to query within BigQuery's free tier limits.

How It Works

1. ✓

   Open BigQuery: Go to console.cloud.google.com/bigquery and sign in with a Google account. If you do not have a GCP project, create one — the free tier gives you 1 TB of query processing per month at no cost.

2. ✓

   Find the dataset: The HN dataset lives at bigquery-public-data.hacker_news. The main tables are full (all items) and stories / comments (filtered views).

3. ✓

   Write a query: Use standard SQL to filter, aggregate, and export. For example, to get all comments on a specific story:

   SELECT author, text, time_ts, ranking
   FROM `bigquery-public-data.hacker_news.comments`
   WHERE parent = 38009291
   ORDER BY ranking DESC
   

4. ✓

   Export results: BigQuery lets you save results as CSV, JSON, or directly to Google Sheets. For large result sets, export to a Google Cloud Storage bucket.

Pros

• ✓Access to the entire HN archive — every comment since 2006
• ✓SQL-based querying — filter by date, author, keyword, points, or any field
• ✓Free tier covers most research use cases (1 TB of queries per month)
• ✓Can aggregate across thousands of threads in a single query
• ✓No rate limiting — queries run against a pre-loaded dataset

Cons

• ✓Requires a Google Cloud account and BigQuery setup
• ✓SQL knowledge is mandatory — no point-and-click interface for filtering
• ✓Dataset updates are not instant — there can be a delay of hours or days
• ✓Nested thread structure is not preserved — comments are stored as flat rows
• ✓Exceeding the free tier incurs charges ($5 per TB of query data processed)

Best for: Researchers and data analysts who need to query across thousands of threads, filter by date ranges, or analyze historical trends across the full HN archive.

Comparison Table

Use Cases for Hacker News Data

Once you have HN comments in structured format, here is what you can do with them.

Tech Industry Research

Export comments from threads about a specific technology — "Rust vs Go" or "Is Kubernetes worth it?" — and you have hundreds of firsthand opinions from working engineers. No survey bias, no marketing filter.

Developer Tool Marketing

Every "Show HN" launch generates unfiltered feedback. Export the comments from your own launch — or a competitor's — and categorize them: feature requests, complaints, praise, comparisons to alternatives. Market research that would cost thousands through traditional channels.

Startup Competitive Intelligence

When a competitor announces funding, a pivot, or a new product, the HN thread is where the candid reactions live. Export those comments to track what engineers and founders actually think — not what the press release says.

Technology Trend Analysis

Export comments from "Who is hiring?" threads across 12 months and count mentions of specific technologies — TypeScript, Rust, Kubernetes, LLMs. You now have a data-backed trend line that predicts adoption better than most industry reports.

Academic Research

HN's 15+ year archive is one of the longest-running records of tech industry discourse. Researchers use it to study community dynamics, sentiment evolution, and the spread of technical ideas. The BigQuery dataset makes longitudinal studies practical.

Recruiting Intelligence

The monthly "Who is hiring?" and "Who wants to be hired?" threads contain thousands of data points — remote work preferences, tech stack preferences, compensation expectations, and company culture signals. Export and analyze these to inform your recruiting strategy.

Frequently Asked Questions

Is the Hacker News API free?

Yes. The HN API is powered by Firebase and is completely free to use. No authentication, API keys, or rate limit tokens are required. However, each comment is a separate API call, and there is no bulk endpoint — so fetching an entire thread with hundreds of comments requires hundreds of individual requests.

Does Comment Exporter handle nested threads on Hacker News?

Yes. Comment Exporter captures the full nested thread structure from Hacker News, including reply depth. You can export as a flat CSV where each row is one comment with a depth indicator, or as structured JSON that preserves the parent-child reply hierarchy.

What format does Hacker News data export in?

With Comment Exporter, you can export HN threads as CSV or JSON. The HN Firebase API returns JSON natively but requires you to build your own export pipeline. The BigQuery dataset can be queried and exported as CSV, JSON, or saved directly to Google Sheets.

Can I export Ask HN and Show HN posts?

Yes. Ask HN and Show HN are standard Hacker News post types. Comment Exporter works on any HN thread URL, including Ask HN, Show HN, and regular link submissions. The HN API and BigQuery dataset also include all post types with a type field you can filter on.

How many comments can I export per Hacker News thread?

Comment Exporter captures all comments loaded on the HN thread page. Most HN threads contain between 50 and 500 comments, though viral posts can exceed 1,000. The HN API can fetch every comment on any thread, but requires recursive calls for each one. BigQuery contains the full historical archive — every comment ever posted on Hacker News.