Building an HTML to Markdown Converter: A Field Guide

This article is based on building the HTML-to-Markdown converter that powers Clean Copy — a cross-platform tool (browser extensions, CLI, GitHub Action, Homebrew, Obsidian plugin) that converts selected text or URLs to clean Markdown. The converter is a single JavaScript file, ~400 lines, zero dependencies, shared verbatim across all platforms.

Related: Preserving table column alignment in HTML to Markdown — how v1.4.1 carries text-align into the separator row.

1. The naive approach: regex

Every HTML-to-Markdown converter starts the same way:

html.replace(/<h1>(.*?)<\/h1>/gi, '# $1')
    .replace(/<b>(.*?)<\/b>/gi, '**$1**')
    .replace(/<img.*?src="(.*?)".*?>/gi, '![]($1)')

This works for controlled input — a README template, a note-taking app's limited HTML subset. But throw a real web page at it — Wikipedia, MDN docs, a news article — and the cracks appear immediately.

A production converter needs to handle:

• Nested elements (a list inside a list inside a blockquote)
• HTML attributes that contain > (inline JSON, event handlers)
• Self-closing tags, malformed tags, and unclosed tags
• Tables with pipes in cell content, colspan, rowspan
• CDATA sections, SVG/MathML subtrees, form controls
• Smart quotes, em-dashes, zero-width characters, invisible Unicode
• HTML entities that look like tags (<b> in code blocks)

Let's walk through each challenge and how the Clean Copy converter addresses it.

2. Architectural approach: iterative refinement

The converter doesn't use a DOM parser or recursive descent. It uses iterative regex substitution — a series of passes that transform HTML into progressively cleaner Markdown. Each pass handles one category of element, and some passes loop until no more substitutions can be made (required for nested structures like lists and blockquotes).

The processing pipeline:

1. Pre-strip: Remove script, style, noscript, template, and head content entirely — before any other transformation runs
2. Preserve CDATA: Extract CDATA content before tag stripping consumes it
3. Strip SVG/MathML: Remove entire subtrees (preserving MathML alt text)
4. Process form controls: Extract select option labels, input values, textarea content
5. Complex block elements: details/summary, dl/dt/dd, figure/figcaption, iframe/object
6. Nested elements: blockquotes → tables → lists (innermost first, iteratively)
7. Inline elements: headings, bold, italic, links, images, code, pre
8. Simple block elements: paragraphs, line breaks, horizontal rules
9. Safe tag stripping: Remove any remaining HTML tags, respecting quotes
10. Entity decode: Decode HTML entities after all tags are gone
11. Text cleaning: Normalize smart quotes, dashes, whitespace, zero-width characters

This ordering is critical. Run entity decoding before tag stripping, and <b> in a code block becomes a real <b> tag, which the tag stripper consumes — eating the rest of the block's content. That was a real bug.

3. The big challenge: nested structures

Lists

A simple regex can flatten one level of list: /<li>(.*?)<\/li>/g. But real HTML has:

<ul>
  <li>Item 1
    <ul>
      <li>Nested A</li>
      <li>Nested B</li>
    </ul>
  </li>
  <li>Item 2</li>
</ul>

The naive match <li>(.*?)<\/li> stops at the first </li> it finds, which is the inner </li> after "Nested A". The tagged content for "Item 1" becomes Item 1\n <ul>\n <li>Nested A — splitting the inner list across multiple captures and producing broken Markdown.

Solution: Convert lists innermost-first. Match <ul|ol>...<\/ul|ol> using a pattern that does not cross another <ul|ol> tag, then loop until no more conversions happen. Each pass converts only the deepest list level, producing correctly indented nested Markdown.

do {
  prev = md;
  md = md.replace(
    /(<(?:ul|ol)[^>]*>)((?:(?!<\/?(?:ul|ol)[^>]*>)[\s\S])*)<\/(?:ul|ol)>/gi,
    convertList
  );
} while (md !== prev);

The (?:(?!...)...)* pattern is a tempered greedy token — it matches any character that doesn't start another list tag. This guarantees we always operate on the innermost list first. After converting that level to Markdown, the outer list becomes convertible because its children are now text.

This same approach is used for blockquotes and tables.

Tables with pipes

Markdown pipe tables use | as the cell separator. If a cell contains a literal pipe — from inline code, or just text like "conditions apply*" — the naive converter produces broken table alignment.

Solution: Escape pipes inside cell content: .replace(/\|/g, '\\|'). Also escape newlines in cells by replacing them with <br>. And handle colspan by padding with empty cells.

Blockquotes

Nested blockquotes need to prefix every line of inner content with > , and nested blockquotes multiply the prefix (> > text). Same innermost-first approach: convert the deepest blockquote first, then the outer one sees already-prefixed lines.

4. The attribute parsing trap

Many converters use /<[^>]*>/g to strip remaining tags. This stops at the first > character — even if it sits inside a quoted attribute value.

Consider Wikipedia's data-mw attribute, which contains JSON:

<span data-mw="{&quot;parts&quot;:[{&quot;template&quot;:{&quot;target&quot;:{&quot;wt&quot;:&quot;cite web&quot;}}} ] }">text</span>

A naive regex sees the > in the JSON's => entity (or unencoded > in JavaScript inline handlers) and stops there — eating the rest of the attribute text and leaking raw JSON into the output.

Solution: A stateful scan that tracks whether we're inside a quoted string:

function stripTagsSafe(html) {
  let out = '', i = 0;
  while (i < html.length) {
    const lt = html.indexOf('<', i);
    if (lt === -1) { out += html.slice(i); break; }
    out += html.slice(i, lt);
    let j = lt + 1, quote = null;
    while (j < html.length) {
      const ch = html[j];
      if (quote) {
        if (ch === quote) quote = null;
      } else if (ch === '"' || ch === "'") {
        quote = ch;
      } else if (ch === '>') {
        break;
      }
      j++;
    }
    if (j >= html.length) { out += html.slice(lt); break; }
    i = j + 1;
  }
  return out;
}

This is slower than a regex — scanning character-by-character — but it's the difference between correct and broken output on real web pages.

5. Entity decoding: the ordering trap

HTML entities like &, <, > need to be decoded to their character equivalents. The obvious approach — decode early — is wrong.

Consider a code block containing <b>bold<\/b>. If you decode entities before stripping tags, the <b> becomes a real <b> tag, which the tag stripper will consume — turning the example into plain "bold" instead of preserving it as code content.

Solution: Decode entities as the last processing step, after all tags have been stripped or converted. The only entity decoded earlier is & (so that & doesn't double-decode to &).

6. Unicode normalization: invisible characters

Real pages contain invisible Unicode that leaks into copied text:

• Zero-width space (U+200B) — used for line-break hints in long words
• Zero-width non-joiner (U+200C) — same purpose
• Word joiner (U+2060) — The Guardian embeds these mid-word
• Invisible times operator (U+2062) — same
• Byte order mark (U+FEFF) — BOM from copy-paste
• Non-breaking space (U+00A0) — should become a regular space

These don't appear in test cases that use clean example HTML. They only surface when you run the converter against hundreds of real web pages and inspect the output. Most converter projects never discover them because nobody checks the output of "The Guardian, article #57".

7. Table handling: beyond the basics

Markdown tables are limited — no colspan, no rowspan, no block elements in cells. Real HTML tables have all of these. The converter flattens them:

• colspan → pad the row with empty cells so column alignment holds
• rowspan → ignore (Markdown has no equivalent; content appears in the first row only)
• Nested tables → innermost-first conversion (same pattern as lists)
• Cell content → recursively run through the full converter pipeline

Cell content is run through htmlToMarkdown() recursively, so inline formatting, links, lists, and even nested tables inside cells are converted correctly. Whitespace in cells is collapsed to single spaces so multi-line cell HTML doesn't produce broken table rows.

8. Readability extraction: the URL mode

When the converter is given a URL (via --url flag or the browser extension's URL input), it needs to extract the page's main content before conversion. The approach:

1. Fetch the page, apply a max size limit (5 MB), follow redirects
2. Strip scripts, styles, nav, header, footer markup
3. Score each block-level element by text density and class/id heuristics
4. Return the highest-scoring block as the "main content"
5. Run that through the standard HTML→Markdown pipeline

This is a simplified version of Mozilla's Readability algorithm — good enough for articles, blog posts, and documentation pages. It fails on single-page apps and pages that render content through JavaScript (which is why the browser extension mode works on selected text instead).

9. Testing strategy

The converter has 25 permanent tests covering:

• Basic formatting: headings, bold, italic, links, images
• Nested structures: 3-level nested lists, nested blockquotes, lists in blockquotes
• Tables: with pipes in cells, empty cells, colspan, nested tables
• Entities: < in code blocks, & double-decode prevention
• Edge cases: CDATA, empty input, whitespace-only, very long words (5K chars), unclosed tags
• HTML5: details/summary, figure/figcaption, dl/dt/dd
• Unicode: smart quotes, em-dashes, zero-width characters, invisible operators
• Real-world regressions: Every bug found against a real page becomes a permanent test

The "real-world regression" rule is the most important: when a converter processes 100 real pages and someone reports that page #101 produces garbage output, the fix gets a test. Over time, the test suite becomes a map of the web's actual HTML surface area.

10. Lessons for building your own

1. Don't parse the DOM. In a browser extension, you could use DOMParser — but that gives you a full browser-grade DOM with normalized tags, which hides the exact markup the user is seeing. Regex-based conversion preserves the raw HTML structure.
2. Process innermost-first. Nested structures (lists, tables, blockquotes) must be converted from the inside out. A single-pass regex will break nesting.
3. Decode entities last. Early entity decoding turns code examples into real tags and eats content.
4. Respect quotes in attributes. A naive <[^>]*> strip will break on attributes containing >.
5. Test against real pages. Clean example HTML hides every edge case. Run the converter against Wikipedia, MDN, the Guardian, and a dozen random blogs. Every broken output is a test you didn't know you needed.
6. Zero dependencies is a goal, not a default. A ~400-line converter with no external dependencies is possible — but it requires deliberate attention to Unicode, entities, and recursive nesting.

Try the converter

Clean Copy is available as a browser extension (Chrome + Firefox), CLI (Homebrew), GitHub Action, and online web tool — all running the same converter core. No analytics, no telemetry, no network requests (except when you explicitly fetch a URL).

Related: the complete terminal guide — See it used end-to-end in the terminal guide for the Clean Copy CLI.