No LLM Artifacts in Prose

Text drafted with a language model and pasted into a repository often carries residue that no human types: citation tokens from the model's browsing tool (`oaicite`, `turn0search0`, `[cite: 3]`, `g...

generalenforced by 1 rule

Context

Text drafted with a language model and pasted into a repository often carries residue that no human types: citation tokens from the model's browsing tool (oaicite, turn0search0, [cite: 3], grok_card, ppl-ai-file-upload), chat-turn framing (Certainly! Here is, I hope this helps, As an AI language model), unfilled template slots ([Insert project name]), and marketing tracking parameters copied from a model's link output (?utm_source=chatgpt.com). The same residue turns up in code comments, where a pasted explanation keeps its I hope this helps! sign-off or its [cite: 2] marker.

Wikipedia's Signs of AI writing treats these as the highest-confidence indicators because they are deterministic strings, not stylistic tendencies. A reader who spots one stops trusting the rest of the document, and a search engine that indexes oaicite in a README is indexing a paste error.

These strings are also the easiest class to check by machine: an exact-token match has essentially no false positives in ordinary prose, so a rule can block on them without slowing anyone down. This ADR is the strict tier of the pack; the structural and wording tiers (GEN-002, GEN-003) are advisory because they measure tendencies rather than tokens.

Decision

Prose in the repository, whether a Markdown document or a comment in source code, MUST NOT contain machine artifacts left behind by a language model or its tools. Concretely:

No citation-tool residue. Tokens emitted by ChatGPT (contentReference[oaicite, oaicite, oai_citation, turn0search0-style markers, attributableIndex, 【4†source】 lenticular citations), Gemini ([cite: 1], [span_1](start_span)), Grok (grok_card, grok_render_citation_card_json), Perplexity (ppl-ai-file-upload), and the unattributed :::writing fence are removed before commit. Tokens that are also plausible identifiers in code (attached_file, a bare contentReference) are deliberately not matched, since the rule reads code comments too.
No chat residue. Turn-opening and turn-closing phrases addressed to a chat user (Certainly!, Great question, I hope this helps, Let me know if you'd like, Would you like me to, Feel free to reach out) and capability disclaimers (As an AI language model, as of my last knowledge update, I cannot browse the internet) do not belong in a document.
No unfilled placeholders. Bracketed template slots such as [Insert description here], [Your Name], or [Add link] are filled in or deleted.
No tracking parameters in links. URLs carrying utm_source, utm_medium, utm_campaign, utm_term, or utm_content are stripped to the canonical address.

In Markdown, fenced code blocks, inline code spans, and HTML comments are exempt because they may legitimately document these tokens (this ADR does). In source files only comments are read: // and /* */ in C-family languages, # in script languages and YAML/TOML, -- in SQL and Lua. String literals, docstrings and heredocs are code, not prose, and are left alone. Files that declare @generated in their first lines are skipped entirely; nobody authored their prose.

Do's and Don'ts

Do

Read the pasted text once before committing; artifacts cluster at the ends of sentences and paragraphs where citations and sign-offs were.
Strip query strings down to what the target site needs; a link to https://example.com/docs does not need ?utm_source=chatgpt.com.
Replace a template slot with real content, or remove the sentence that needed it.
Wrap a token in backticks when a document must mention it, for example when explaining what to look for.

Don't

Commit oaicite, turn0search, [cite: N], grok_card, or ppl-ai-file-upload anywhere in prose.
Leave Certainly!, I hope this helps, or As an AI in a README, changelog, ADR, or code comment.
Ship [Insert X here] or [Your Name] in a published document.
Suppress a finding from this ADR; the fix is a deletion and takes seconds.

Consequences

Positive

Readers do not lose confidence in a document over a paste error.
The check is deterministic, so it runs at error severity without slowing anyone down.
Search indexes and package registries never see citation-tool tokens in a project's public text.

Negative

The token list tracks specific vendors' current tooling and needs updating as they change; a new artifact goes uncaught until someone adds it.
A README that intentionally documents these tokens must put them in code spans, which is a small formatting constraint.
The placeholder rule cannot tell a forgotten [Your Name] from a deliberate template file, so it runs at warning; template repositories may want to exclude their template directory in files.
Comment extraction is line-based, not a parser. A marker counts as a comment when the code before it has balanced quotes, which handles "http://x" and '#' but not a string that spans lines or a comment marker inside a template literal. The usual failure direction is a missed comment rather than a false report against code, and the token lists are specific enough that a stray match inside a regex literal is unlikely.

Enforcement rules

adrs/GEN-001-no-llm-artifacts-in-prose.rules.ts

/// <reference path="../rules.d.ts" />

// ---------- Markdown prose extraction (duplicated per rules file; rule files
// cannot import each other) ----------

interface ProseLine {
  /** 1-based line number in the source file. */
  line: number;
  /** Line with HTML comments and inline code spans blanked to spaces. */
  text: string;
  /** `text` with link targets and bare URLs blanked as well. */
  words: string;
  /** Inside a fenced code block: content of its own, never prose. */
  code: boolean;
}

const FENCE = /^ {0,3}(`{3,}|~{3,})(.*)$/u;
const FRONTMATTER_DELIMITER = /^---\s*$/u;

/** Same-length run of spaces, so columns of later matches stay stable. */
function blank(s: string): string {
  return " ".repeat(s.length);
}

/**
 * Prose lines of a Markdown document: YAML frontmatter, fenced code blocks and
 * HTML comments are dropped; inline code spans, link targets and bare URLs are
 * blanked so their contents never match a prose pattern.
 */
function proseLines(content: string): ProseLine[] {
  const out: ProseLine[] = [];
  const lines = content.split(/\r?\n/u);
  let inFrontmatter = FRONTMATTER_DELIMITER.test(lines[0] ?? "");
  let fence: { marker: string; length: number } | null = null;
  let inComment = false;

  for (const [index, raw] of lines.entries()) {
    if (inFrontmatter) {
      if (index > 0 && FRONTMATTER_DELIMITER.test(raw)) inFrontmatter = false;
      continue;
    }

    const fenceMatch = FENCE.exec(raw);
    if (fence) {
      const closes =
        fenceMatch !== null &&
        fenceMatch[1][0] === fence.marker &&
        fenceMatch[1].length >= fence.length &&
        fenceMatch[2].trim() === "";
      if (closes) fence = null;
      out.push({ line: index + 1, text: "", words: "", code: true });
      continue;
    }
    if (fenceMatch) {
      fence = { marker: fenceMatch[1][0], length: fenceMatch[1].length };
      out.push({ line: index + 1, text: "", words: "", code: true });
      continue;
    }

    let text = raw;
    if (inComment) {
      const end = text.indexOf("-->");
      if (end === -1) continue;
      text = blank(text.slice(0, end + 3)) + text.slice(end + 3);
      inComment = false;
    }
    text = text.replace(/<!--[\s\S]*?-->/gu, blank);
    const open = text.indexOf("<!--");
    if (open !== -1) {
      text = text.slice(0, open) + blank(text.slice(open));
      inComment = true;
    }
    text = text.replace(/(`+)[\s\S]*?\1/gu, blank);

    const words = text
      .replace(/\]\(([^)]*)\)/gu, (_m, target: string) => `](${blank(target)})`)
      .replace(/<https?:\/\/[^>]*>/gu, blank)
      .replace(/https?:\/\/\S+/gu, blank);

    out.push({ line: index + 1, text, words, code: false });
  }
  return out;
}

// ---------- Source-comment extraction ----------

/** Double-slash line comments and slash-star block comments. */
const SLASH_COMMENT_EXTENSIONS = new Set([
  "ts",
  "tsx",
  "mts",
  "cts",
  "js",
  "jsx",
  "mjs",
  "cjs",
  "java",
  "kt",
  "kts",
  "go",
  "rs",
  "c",
  "h",
  "cpp",
  "hpp",
  "cc",
  "cs",
  "swift",
  "scala",
  "dart",
  "php",
  "groovy",
]);
/** `#` line comments. */
const HASH_COMMENT_EXTENSIONS = new Set([
  "py",
  "rb",
  "sh",
  "bash",
  "zsh",
  "pl",
  "r",
  "yml",
  "yaml",
  "toml",
  "tf",
  "ps1",
]);
/** `--` line comments. */
const DASH_COMMENT_EXTENSIONS = new Set(["sql", "lua"]);

function extensionOf(file: string): string {
  const dot = file.lastIndexOf(".");
  return dot === -1 ? "" : file.slice(dot + 1).toLowerCase();
}

/**
 * Heuristic string check: a comment marker counts only when the code before it
 * has balanced quotes, so `"http://x"` and `'#'` are not read as comments.
 */
function outsideString(prefix: string): boolean {
  let dq = 0;
  let sq = 0;
  let bt = 0;
  for (let i = 0; i < prefix.length; i++) {
    const c = prefix[i];
    if (c === "\\") {
      i++;
      continue;
    }
    if (c === '"') dq++;
    else if (c === "'") sq++;
    else if (c === "`") bt++;
  }
  return dq % 2 === 0 && sq % 2 === 0 && bt % 2 === 0;
}

/** Earliest comment marker on a line that sits outside a string literal. */
function findMarker(
  line: string,
  slash: boolean,
  hash: boolean,
  dash: boolean,
): { at: number; marker: "//" | "/*" | "#" | "--" } | null {
  let best: { at: number; marker: "//" | "/*" | "#" | "--" } | null = null;
  const consider = (re: RegExp, marker: "//" | "/*" | "#" | "--") => {
    for (const m of line.matchAll(re)) {
      const at = (m.index ?? 0) + (m[1]?.length ?? 0);
      if (best && at >= best.at) break;
      if (outsideString(line.slice(0, at))) {
        best = { at, marker };
        break;
      }
    }
  };
  if (slash) {
    consider(/(^|[^:])\/\//gu, "//");
    consider(/()\/\*/gu, "/*");
  }
  if (hash) consider(/(^|\s)#/gu, "#");
  if (dash) consider(/(^|\s)--/gu, "--");
  return best;
}

/**
 * Comment text of a source file as prose lines, everything else blanked so
 * columns hold. Covers slash-style line and block comments (C-family), `#` (script languages) and
 * `--` (SQL, Lua). Docstrings and heredocs are not comments and are skipped.
 */
function commentLines(content: string, ext: string): ProseLine[] {
  const slash = SLASH_COMMENT_EXTENSIONS.has(ext);
  const hash = HASH_COMMENT_EXTENSIONS.has(ext);
  const dash = DASH_COMMENT_EXTENSIONS.has(ext);
  if (!slash && !hash && !dash) return [];

  const out: ProseLine[] = [];
  let inBlock = false;
  for (const [index, raw] of content.split(/\r?\n/u).entries()) {
    if (index === 0 && raw.startsWith("#!")) continue;
    let text: string;
    if (inBlock) {
      const end = raw.indexOf("*/");
      text = end === -1 ? raw : raw.slice(0, end) + blank(raw.slice(end));
      if (end !== -1) inBlock = false;
      // Blank the decorative leading `*` of a block-comment continuation line.
      text = text.replace(/^(\s*)\*(?!\/)/u, (_m, ws: string) => `${ws} `);
    } else {
      const found = findMarker(raw, slash, hash, dash);
      if (!found) continue;
      const { at, marker } = found;
      const start = at + marker.length;
      if (marker === "/*") {
        const end = raw.indexOf("*/", start);
        const body = end === -1 ? raw.slice(start) : raw.slice(start, end);
        text = blank(raw.slice(0, start)) + body.replace(/^\*+/u, blank);
        if (end === -1) inBlock = true;
        else text += blank(raw.slice(end));
      } else {
        text = blank(raw.slice(0, start)) + raw.slice(start);
      }
    }
    if (text.trim() === "") continue;

    text = text.replace(/(`+)[\s\S]*?\1/gu, blank);
    const words = text.replace(/<https?:\/\/[^>]*>/gu, blank).replace(/https?:\/\/\S+/gu, blank);
    out.push({ line: index + 1, text, words, code: false });
  }
  return out;
}

/** Generated files carry an `@generated` marker in their first lines; nobody authored their prose. */
function isGenerated(content: string): boolean {
  return /@generated\b/u.test(content.slice(0, 300));
}

/**
 * Extracted prose per file, shared by every rule in this file for one run.
 * Keyed on the `scopedFiles` array, which the engine hands unchanged to each
 * rule of an ADR and rebuilds on the next run, so nothing goes stale.
 */
const PROSE_CACHE = new WeakMap<readonly string[], Map<string, Promise<ProseLine[] | null>>>();

/** Prose of one file, or null when it is unreadable, generated, or not a kind we read. */
function extractProse(ctx: RuleContext, file: string): Promise<ProseLine[] | null> {
  let perRun = PROSE_CACHE.get(ctx.scopedFiles);
  if (!perRun) {
    perRun = new Map();
    PROSE_CACHE.set(ctx.scopedFiles, perRun);
  }
  let pending = perRun.get(file);
  if (!pending) {
    pending = (async () => {
      const ext = extensionOf(file);
      const markdown = ext === "md" || ext === "mdx";
      if (
        !markdown &&
        !SLASH_COMMENT_EXTENSIONS.has(ext) &&
        !HASH_COMMENT_EXTENSIONS.has(ext) &&
        !DASH_COMMENT_EXTENSIONS.has(ext)
      )
        return null;
      let content: string;
      try {
        content = await ctx.readFile(file);
      } catch {
        return null;
      }
      if (isGenerated(content)) return null;
      return markdown ? proseLines(content) : commentLines(content, ext);
    })();
    perRun.set(file, pending);
  }
  return pending;
}

/**
 * Visit every scoped file's prose: Markdown documents through `proseLines`,
 * source files through `commentLines`. Generated files and files of any other
 * kind are skipped. Extraction happens once per file per run.
 */
async function scanProse(
  ctx: RuleContext,
  visit: (file: string, lines: ProseLine[]) => void,
): Promise<void> {
  await Promise.all(
    ctx.scopedFiles.map(async (file) => {
      const lines = await extractProse(ctx, file);
      if (lines) visit(file, lines);
    }),
  );
}

// ---------- Patterns ----------

/** Tokens emitted by a model's browsing/citation tooling, keyed by vendor. */
const CITATION_RESIDUE: { pattern: RegExp; vendor: string }[] = [
  {
    pattern:
      /contentReference\[oaicite|oaicite|oai_citation|attributableIndex|\bturn\d+(?:search|view|news|image|fetch|file)\d+\b|【[^】]*†[^】]*】/u,
    vendor: "ChatGPT",
  },
  {
    pattern: /\[cite:\s*\d+\]?|\[span_\d+\]\(start_span\)|\(end_span\)/u,
    vendor: "Gemini",
  },
  { pattern: /grok_card|grok_render_citation_card_json/u, vendor: "Grok" },
  { pattern: /ppl-ai-file-upload/u, vendor: "Perplexity" },
  { pattern: /:::writing/u, vendor: "an unattributed model" },
];

/** Chat-turn framing and capability disclaimers addressed to a chat user. */
const CHAT_RESIDUE: RegExp[] = [
  /\bAs an AI(?: language model| assistant| model)?\b/iu,
  /\bas of my (?:last |latest )?(?:knowledge (?:cutoff|update)|training (?:data|cutoff))\b/iu,
  /\bI (?:cannot|can't|am unable to|don't have the ability to) (?:browse|access) (?:the internet|the web|real-time|external|live)/iu,
  /^\s*(?:>\s*)?(?:Certainly|Great question|Absolutely)[!,]/u,
  /^\s*(?:>\s*)?Sure!/u,
  /\bI hope this helps\b/iu,
  /\bLet me know if you(?:'d| would) like\b/iu,
  /\bWould you like me to\b/iu,
  /\bHere(?:'s| is) (?:a|an|the|your) (?:revised|updated|improved|polished|rewritten|refined|complete|full) (?:version|draft|text|document|README)\b/iu,
  /\bI(?:'ve| have) (?:updated|revised|rewritten|drafted) the (?:above|following|document|text)\b/iu,
];

/** Bracketed template slots that were never filled in. */
const PLACEHOLDERS: RegExp[] = [
  // A bare `[` (not a link, aside or reference) opening a slot such as
  // `[Insert X here]`, `[Your Name]`, `[Add link]`, `[Company Name]`, `[TBD here]`.
  /(?<![\w:\]])\[(?:insert|your|add|enter|placeholder|company name|[^\]\n]{0,40} here)\b[^\]\n]{0,60}\](?!\s*[([:])/iu,
  /\blorem ipsum\b/iu,
];

const TRACKING_PARAM = /[?&](utm_(?:source|medium|campaign|term|content))=/u;

// ---------- Rules ----------

export default {
  rules: {
    "no-llm-citation-residue": {
      description:
        "Prose must not contain citation-tool tokens left behind by a language model (oaicite, turn0search0, [cite: N], grok_card, ppl-ai-file-upload, ...)",
      severity: "error",
      async check(ctx) {
        await scanProse(ctx, (file, lines) => {
          for (const { line, text } of lines) {
            for (const { pattern, vendor } of CITATION_RESIDUE) {
              const m = pattern.exec(text);
              if (!m) continue;
              ctx.report.violation({
                message: `"${m[0]}" is a citation artifact from ${vendor}'s tooling, not text a person wrote.`,
                file,
                line,
                fix: "Delete the token. If the sentence needs a source, add a real link or footnote in its place.",
              });
              break;
            }
          }
        });
      },
    },

    "no-chat-residue": {
      description:
        "Prose must not carry chat-turn framing or capability disclaimers (Certainly!, I hope this helps, As an AI language model, ...)",
      severity: "error",
      async check(ctx) {
        await scanProse(ctx, (file, lines) => {
          for (const { line, text } of lines) {
            for (const pattern of CHAT_RESIDUE) {
              const m = pattern.exec(text);
              if (!m) continue;
              ctx.report.violation({
                message: `"${m[0].trim()}" addresses a chat user, not the reader of this document.`,
                file,
                line,
                fix: "Delete the sentence, or rewrite it as a statement about the subject rather than about the conversation.",
              });
              break;
            }
          }
        });
      },
    },

    "no-unfilled-placeholders": {
      description:
        "Prose must not ship bracketed template slots ([Insert X here], [Your Name], lorem ipsum)",
      severity: "warning",
      async check(ctx) {
        await scanProse(ctx, (file, lines) => {
          for (const { line, words } of lines) {
            for (const pattern of PLACEHOLDERS) {
              const m = pattern.exec(words);
              if (!m) continue;
              ctx.report.warning({
                message: `"${m[0]}" is an unfilled placeholder.`,
                file,
                line,
                fix: "Fill the slot with real content, or remove the sentence that needed it. If this file is a deliberate template, exclude its directory in the ADR's `files` globs.",
              });
              break;
            }
          }
        });
      },
    },

    "no-tracking-params-in-links": {
      description:
        "Links must not carry utm_* tracking parameters copied from a model's link output",
      severity: "error",
      async check(ctx) {
        await scanProse(ctx, (file, lines) => {
          for (const { line, text } of lines) {
            const m = TRACKING_PARAM.exec(text);
            if (!m) continue;
            ctx.report.violation({
              message: `Link carries the tracking parameter "${m[1]}", which identifies where the URL was copied from rather than what it points to.`,
              file,
              line,
              fix: "Remove the utm_* parameters (and the trailing `?` or `&` left behind) so the link is the canonical address.",
            });
          }
        });
      },
    },
  },
} satisfies RuleSet;