Check an AI video summary against the video before you trust it
Tsugunori Katayama, a retiree in Japan born in 1953 who writes about using AI in his 70s, asked two chatbots to summarize the same long interview video. Then he watched the video himself. Gemini's summary matched it, but ChatGPT's included points the video never made, and they sounded so reasonable he might have believed them.
What they did
He wanted the gist of a long YouTube interview about how top performers spend their days off without sitting through the whole thing. He pasted the video's link into Gemini, which he usually uses for this, and asked for a summary. Then he tried the same thing in ChatGPT. Afterward he watched the video himself and compared both summaries with what was actually said.
How it went
Gemini's summary lined up with the video. ChatGPT also wrote a tidy, well-organized summary, but it included things he couldn't find anywhere in the video, such as advice about using Saturday and Sunday differently. When he asked ChatGPT about it, it turned out it hadn't been able to read the video's content properly and had likely pieced together a plausible summary from the title and related information. What worried him most was that the made-up points sounded like sensible advice from a self-help book, so he'd have believed them if he hadn't checked. His takeaway: for summarizing a YouTube link, Gemini worked better for him, but even then, check anything important against the original.
Worth knowing
An AI can write a confident summary of something it never actually read or watched. Before you rely on one, ask it directly whether it could open the video or page, and for an important video, spot-check a few points by watching those parts yourself. Asking for timestamps makes checking easier. Summaries of public videos are fine to request; don't paste private recordings or anything confidential.
Try it yourself
Copy this into ChatGPT (or any AI assistant). Replace the parts in [brackets] with your own details.
Please summarize this video: [paste the video link]. Before you summarize, tell me plainly whether you were able to access the video's actual content (for example, its transcript). If you couldn't, say so and don't guess. If you could, give me the main points in plain language, with the approximate time in the video where each point comes up, so I can check them myself.
Source: 片山嗣規 (Tsugunori Katayama), note (Tsugunori Katayama's blog), Sep 4, 2026. Read the original →
First-hand account by the person who did it. Summarized in our own words.
Report a problem with this story. Spotted a mistake, a broken link, or want this removed? Tell us and we'll fix it within 24 hours.