{
  "title": "What AI resume writers invent",
  "url": "https://hirova.app/research/what-ai-resume-writers-invent",
  "publisher": "Hirova",
  "license": "https://creativecommons.org/licenses/by/4.0/",
  "date_published": "2026-10-15",
  "date_modified": "2026-10-15",
  "temporal_coverage": "2026-10-12/2026-10-13",
  "summary": "In our tests, the AI resume writer rarely made up numbers outright; more often it added claims the person never made. Across 30 synthetic profiles, 12 of 2,400 (0.5%) AI-written resume sections had a number that was not in the facts, while an AI judge flagged an unsupported claim in 64 of 2,355 (2.7%).",
  "definitions": {
    "profile": "A synthetic resume profile written for testing: a job, dates, tools, education, three to five facts and a job posting. No real person or user.",
    "section": "One output for one profile: facts read from the resume, interview questions, rewritten bullets, tailored bullets with a cover letter, or a suggested replacement bullet.",
    "number_flag": "An automated check found a number in the section that is not in the profile's supplied facts. It counts digits and the words one to twenty, ignores 'one of', and misses vaguer words such as 'dozens'.",
    "name_flag": "The automated check or the judge found an employer, tool or other name that is not in the profile's facts.",
    "unsupported_claim": "The AI judge listed at least one factual error: something the supplied facts do not establish, such as an upgraded credential, a cause nobody stated or a stronger verb.",
    "shown": "Hirova's server accepted the section, so the app would have shown it. Refused sections are never shown.",
    "refused": "Hirova's server checks rejected the section: a number or name not in the facts, a claim its support check could not tie to a fact, or a cover letter outside its length band."
  },
  "conditions": [
    {
      "id": "single-prompt",
      "name": "Single prompt, no checks",
      "hirova_checks": false,
      "writer_model": "openai/gpt-6-luna",
      "judge_model": "openai/gpt-6.1-sol",
      "profiles": 20,
      "runs": [
        {
          "run": "cf-gateway-2026-10-12",
          "started_utc": "2026-10-12T00:08:55.613455+00:00",
          "completed_utc": "2026-10-12T00:10:52.309828+00:00",
          "prompt_revision": null,
          "profiles": 20,
          "sections": 100,
          "number_flags": 1,
          "number_flags_shown": null,
          "name_flags": 12,
          "name_flags_shown": null,
          "judged_sections": 100,
          "unsupported_claim_sections": 15,
          "unsupported_claim_sections_shown": null,
          "refused_sections": null,
          "refusals": {}
        }
      ],
      "totals": {
        "sections": 100,
        "number_flags": 1,
        "number_flags_shown": null,
        "name_flags": 12,
        "name_flags_shown": null,
        "judged_sections": 100,
        "unsupported_claim_sections": 15,
        "unsupported_claim_sections_shown": null,
        "refused_sections": null,
        "refusals": {}
      },
      "by_section_type": {
        "extraction": {
          "name": "Facts read from the resume",
          "sections": 20,
          "number_flags": 0,
          "judged_sections": 20,
          "unsupported_claim_sections": 0
        },
        "questions": {
          "name": "Interview questions",
          "sections": 20,
          "number_flags": 0,
          "judged_sections": 20,
          "unsupported_claim_sections": 0
        },
        "rewrite": {
          "name": "Rewritten bullets",
          "sections": 20,
          "number_flags": 0,
          "judged_sections": 20,
          "unsupported_claim_sections": 3
        },
        "tailoring": {
          "name": "Tailored bullets and cover letter",
          "sections": 20,
          "number_flags": 1,
          "judged_sections": 20,
          "unsupported_claim_sections": 8
        },
        "suggestion": {
          "name": "Suggested replacement bullet",
          "sections": 20,
          "number_flags": 0,
          "judged_sections": 20,
          "unsupported_claim_sections": 4
        }
      }
    },
    {
      "id": "hirova-pipeline",
      "name": "Hirova pipeline, with checks",
      "hirova_checks": true,
      "writer_model": "openai/gpt-6-luna",
      "judge_model": "codex:gpt-6.1-sol",
      "profiles": 30,
      "runs": [
        {
          "run": "gate-2026-10-13-m5-sol-a/run-1",
          "started_utc": "2026-10-12T23:41:06.062170+00:00",
          "completed_utc": "2026-10-12T23:45:59.253494+00:00",
          "prompt_revision": "2026-10-13.3",
          "profiles": 30,
          "sections": 150,
          "number_flags": 0,
          "number_flags_shown": 0,
          "name_flags": 0,
          "name_flags_shown": 0,
          "judged_sections": 150,
          "unsupported_claim_sections": 7,
          "unsupported_claim_sections_shown": 4,
          "refused_sections": 7,
          "refusals": {
            "rejected:support": 7
          }
        },
        {
          "run": "gate-2026-10-13-m5-sol-a/run-2",
          "started_utc": "2026-10-12T23:49:33.849591+00:00",
          "completed_utc": "2026-10-12T23:52:54.151054+00:00",
          "prompt_revision": "2026-10-13.3",
          "profiles": 30,
          "sections": 150,
          "number_flags": 1,
          "number_flags_shown": 0,
          "name_flags": 1,
          "name_flags_shown": 0,
          "judged_sections": 150,
          "unsupported_claim_sections": 10,
          "unsupported_claim_sections_shown": 8,
          "refused_sections": 6,
          "refusals": {
            "rejected:support": 5,
            "rejected:numeric": 1
          }
        },
        {
          "run": "gate-2026-10-13-m5-final/run-1",
          "started_utc": "2026-10-13T00:06:58.413163+00:00",
          "completed_utc": "2026-10-13T00:11:00.498144+00:00",
          "prompt_revision": "2026-10-13.1",
          "profiles": 30,
          "sections": 150,
          "number_flags": 1,
          "number_flags_shown": 0,
          "name_flags": 0,
          "name_flags_shown": 0,
          "judged_sections": 150,
          "unsupported_claim_sections": 6,
          "unsupported_claim_sections_shown": 5,
          "refused_sections": 7,
          "refusals": {
            "rejected:support": 6,
            "rejected:numeric": 1
          }
        },
        {
          "run": "gate-2026-10-13-m5-final/run-2",
          "started_utc": "2026-10-13T00:14:52.431839+00:00",
          "completed_utc": "2026-10-13T00:18:59.715304+00:00",
          "prompt_revision": "2026-10-13.1",
          "profiles": 30,
          "sections": 150,
          "number_flags": 0,
          "number_flags_shown": 0,
          "name_flags": 1,
          "name_flags_shown": 1,
          "judged_sections": 150,
          "unsupported_claim_sections": 8,
          "unsupported_claim_sections_shown": 7,
          "refused_sections": 10,
          "refusals": {
            "rejected:support": 9,
            "rejected:numeric": 1
          }
        },
        {
          "run": "gate-2026-10-13-m5-final2/run-1",
          "started_utc": "2026-10-13T00:23:11.923537+00:00",
          "completed_utc": "2026-10-13T00:26:55.039609+00:00",
          "prompt_revision": "2026-10-13.5",
          "profiles": 30,
          "sections": 150,
          "number_flags": 1,
          "number_flags_shown": 0,
          "name_flags": 0,
          "name_flags_shown": 0,
          "judged_sections": 150,
          "unsupported_claim_sections": 4,
          "unsupported_claim_sections_shown": 4,
          "refused_sections": 4,
          "refusals": {
            "rejected:support": 3,
            "rejected:numeric": 1
          }
        },
        {
          "run": "gate-2026-10-13-m5-final2/run-2",
          "started_utc": "2026-10-13T00:30:20.585252+00:00",
          "completed_utc": "2026-10-13T00:36:00.761732+00:00",
          "prompt_revision": "2026-10-13.5",
          "profiles": 30,
          "sections": 150,
          "number_flags": 2,
          "number_flags_shown": 0,
          "name_flags": 0,
          "name_flags_shown": 0,
          "judged_sections": 150,
          "unsupported_claim_sections": 4,
          "unsupported_claim_sections_shown": 4,
          "refused_sections": 3,
          "refusals": {
            "rejected:support": 1,
            "rejected:numeric": 2
          }
        },
        {
          "run": "gate-2026-10-13-m5-final3/run-1",
          "started_utc": "2026-10-13T00:40:15.678647+00:00",
          "completed_utc": "2026-10-13T00:45:38.110323+00:00",
          "prompt_revision": "2026-10-13.5",
          "profiles": 30,
          "sections": 150,
          "number_flags": 1,
          "number_flags_shown": 0,
          "name_flags": 0,
          "name_flags_shown": 0,
          "judged_sections": 150,
          "unsupported_claim_sections": 6,
          "unsupported_claim_sections_shown": 5,
          "refused_sections": 4,
          "refusals": {
            "rejected:numeric": 2,
            "rejected:entity": 1,
            "rejected:support": 1
          }
        },
        {
          "run": "gate-2026-10-13-m5-final3/run-2",
          "started_utc": "2026-10-13T00:48:56.057262+00:00",
          "completed_utc": "2026-10-13T00:53:37.795892+00:00",
          "prompt_revision": "2026-10-13.5",
          "profiles": 30,
          "sections": 150,
          "number_flags": 0,
          "number_flags_shown": 0,
          "name_flags": 0,
          "name_flags_shown": 0,
          "judged_sections": 150,
          "unsupported_claim_sections": 4,
          "unsupported_claim_sections_shown": 4,
          "refused_sections": 2,
          "refusals": {
            "rejected:numeric": 1,
            "rejected:support": 1
          }
        },
        {
          "run": "gate-2026-10-13-55-r1/run-1",
          "started_utc": "2026-10-13T01:48:58.245662+00:00",
          "completed_utc": "2026-10-13T02:00:30.655573+00:00",
          "prompt_revision": "2026-10-13.6",
          "profiles": 30,
          "sections": 150,
          "number_flags": 1,
          "number_flags_shown": 0,
          "name_flags": 0,
          "name_flags_shown": 0,
          "judged_sections": 150,
          "unsupported_claim_sections": 3,
          "unsupported_claim_sections_shown": 2,
          "refused_sections": 3,
          "refusals": {
            "rejected:support": 1,
            "rejected:entity": 1,
            "rejected:numeric": 1
          }
        },
        {
          "run": "gate-2026-10-13-55-r1/run-2",
          "started_utc": "2026-10-13T02:10:24.662584+00:00",
          "completed_utc": "2026-10-13T02:25:12.523090+00:00",
          "prompt_revision": "2026-10-13.6",
          "profiles": 30,
          "sections": 150,
          "number_flags": 2,
          "number_flags_shown": 0,
          "name_flags": 0,
          "name_flags_shown": 0,
          "judged_sections": 150,
          "unsupported_claim_sections": 1,
          "unsupported_claim_sections_shown": 1,
          "refused_sections": 3,
          "refusals": {
            "rejected:numeric": 3
          }
        },
        {
          "run": "gate-2026-10-13-64-r1/run-1",
          "started_utc": "2026-10-13T07:07:38.931122+00:00",
          "completed_utc": "2026-10-13T07:12:37.754178+00:00",
          "prompt_revision": "2026-10-13.11",
          "profiles": 30,
          "sections": 150,
          "number_flags": 0,
          "number_flags_shown": 0,
          "name_flags": 0,
          "name_flags_shown": 0,
          "judged_sections": 125,
          "unsupported_claim_sections": 3,
          "unsupported_claim_sections_shown": 3,
          "refused_sections": 0,
          "refusals": {}
        },
        {
          "run": "gate-2026-10-13-64-r1/run-2",
          "started_utc": "2026-10-13T07:16:44.700151+00:00",
          "completed_utc": "2026-10-13T07:22:02.265540+00:00",
          "prompt_revision": "2026-10-13.11",
          "profiles": 30,
          "sections": 150,
          "number_flags": 2,
          "number_flags_shown": 0,
          "name_flags": 0,
          "name_flags_shown": 0,
          "judged_sections": 145,
          "unsupported_claim_sections": 0,
          "unsupported_claim_sections_shown": 0,
          "refused_sections": 4,
          "refusals": {
            "rejected:letter_length": 1,
            "rejected:numeric": 2,
            "rejected:support": 1
          }
        },
        {
          "run": "gate-2026-10-13-64-r2/run-1",
          "started_utc": "2026-10-13T07:27:16.793121+00:00",
          "completed_utc": "2026-10-13T07:32:41.905592+00:00",
          "prompt_revision": "2026-10-13.12",
          "profiles": 30,
          "sections": 150,
          "number_flags": 0,
          "number_flags_shown": 0,
          "name_flags": 0,
          "name_flags_shown": 0,
          "judged_sections": 140,
          "unsupported_claim_sections": 1,
          "unsupported_claim_sections_shown": 1,
          "refused_sections": 1,
          "refusals": {
            "rejected:letter_length": 1
          }
        },
        {
          "run": "gate-2026-10-13-64-r2/run-2",
          "started_utc": "2026-10-13T07:36:50.561364+00:00",
          "completed_utc": "2026-10-13T07:41:41.858911+00:00",
          "prompt_revision": "2026-10-13.12",
          "profiles": 30,
          "sections": 150,
          "number_flags": 0,
          "number_flags_shown": 0,
          "name_flags": 0,
          "name_flags_shown": 0,
          "judged_sections": 145,
          "unsupported_claim_sections": 3,
          "unsupported_claim_sections_shown": 3,
          "refused_sections": 3,
          "refusals": {
            "rejected:support": 2,
            "rejected:letter_length": 1
          }
        },
        {
          "run": "gate-2026-10-13-64-r3-recheck/run-1",
          "started_utc": "2026-10-13T08:20:25.983820+00:00",
          "completed_utc": "2026-10-13T08:20:48.570856+00:00",
          "prompt_revision": "2026-10-13.13",
          "profiles": 30,
          "sections": 150,
          "number_flags": 1,
          "number_flags_shown": 0,
          "name_flags": 0,
          "name_flags_shown": 0,
          "judged_sections": 150,
          "unsupported_claim_sections": 3,
          "unsupported_claim_sections_shown": 2,
          "refused_sections": 2,
          "refusals": {
            "rejected:support": 1,
            "rejected:numeric": 1
          }
        },
        {
          "run": "gate-2026-10-13-64-r3-recheck/run-2",
          "started_utc": "2026-10-13T08:20:49.170554+00:00",
          "completed_utc": "2026-10-13T08:21:05.217841+00:00",
          "prompt_revision": "2026-10-13.13",
          "profiles": 30,
          "sections": 150,
          "number_flags": 0,
          "number_flags_shown": 0,
          "name_flags": 0,
          "name_flags_shown": 0,
          "judged_sections": 150,
          "unsupported_claim_sections": 1,
          "unsupported_claim_sections_shown": 1,
          "refused_sections": 1,
          "refusals": {
            "rejected:support": 1
          }
        }
      ],
      "totals": {
        "sections": 2400,
        "number_flags": 12,
        "number_flags_shown": 0,
        "name_flags": 2,
        "name_flags_shown": 1,
        "judged_sections": 2355,
        "unsupported_claim_sections": 64,
        "unsupported_claim_sections_shown": 54,
        "refused_sections": 60,
        "refusals": {
          "rejected:support": 39,
          "rejected:numeric": 16,
          "rejected:entity": 2,
          "rejected:letter_length": 3
        }
      },
      "by_section_type": {
        "extraction": {
          "name": "Facts read from the resume",
          "sections": 480,
          "number_flags": 0,
          "judged_sections": 471,
          "unsupported_claim_sections": 2
        },
        "questions": {
          "name": "Interview questions",
          "sections": 480,
          "number_flags": 4,
          "judged_sections": 471,
          "unsupported_claim_sections": 0
        },
        "rewrite": {
          "name": "Rewritten bullets",
          "sections": 480,
          "number_flags": 0,
          "judged_sections": 471,
          "unsupported_claim_sections": 13
        },
        "tailoring": {
          "name": "Tailored bullets and cover letter",
          "sections": 480,
          "number_flags": 8,
          "judged_sections": 471,
          "unsupported_claim_sections": 45
        },
        "suggestion": {
          "name": "Suggested replacement bullet",
          "sections": 480,
          "number_flags": 0,
          "judged_sections": 471,
          "unsupported_claim_sections": 4
        }
      }
    }
  ],
  "excluded_runs": [
    "The pipeline gates of October 12, 2026, on 20 or 25 profiles, judged through the API with an earlier setup (gate-2026-10-12-*), and the one-change comparisons of that evening (compare-*).",
    "Earlier outputs judged a second time (sol-rejudge-*, sol-codex-baseline-*, luna-judge-baseline-*), so no output counts twice, and gate-2026-10-13-64-r3, replaced by its recheck.",
    "Gates judged by GPT-6 Luna, the writing model itself (gate-2026-10-13-m5-r8, -r9 and -r10), and probes of single prompt changes (probe-*).",
    "The single-prompt rerun of 14 profiles after a prompt fix (cf-gateway-2026-10-12/retry), which would mix two prompts in one condition."
  ],
  "limits": [
    "The profiles are synthetic and short. Real resumes are longer and messier, so real rates can differ.",
    "One writing model, GPT-6 Luna. A planned comparison of 8 models on October 11, 2026 returned no outputs (every request failed), so nothing here describes other models.",
    "The judge is an AI model too. It can miss errors and flag sentences a person would accept. We read the notes quoted here, not all of them.",
    "The pipeline runs reuse the same 30 profiles, so the 2,400 sections are not independent cases. The same section can be flagged in several runs.",
    "Prompts changed between runs (revisions 2026-10-13.1, 2026-10-13.3, 2026-10-13.5, 2026-10-13.6, 2026-10-13.11, 2026-10-13.12, 2026-10-13.13), and Hirova's prompts have changed since. The results describe the versions tested.",
    "The two conditions differ in profiles, prompt and judge setup. Compare them as a description, not as a controlled experiment.",
    "The number check can flag harmless words such as 'one', and it cannot see a vague claim that has no number in it. That is what the judge is for."
  ]
}