On September 13, 2026 I checked every water filter reviewed on h2oinsider.com against the NSF, IAPMO and WQA certification directories, and 22 of the 23 certification lists were wrong or incomplete. Claude had written all of them in a single commit on April 24, 2026, and nobody touched them for 142 days.
H2O Insider is my own water filter review site. This post is the audit, the part of the fix that missed the live site for a month, and what the structured data was saying in the meantime.
How did wrong certifications get into the product data?
Through one data file. The commit that created products.json landed at 22:38 on April 24 with a Claude Sonnet 4.6 co-author line. It held 25 products and 49 certification claims. The data model had a field for the standard codes and no field for where each claim came from.
The rule that should have stopped it was already in the repo. The project’s PRD, committed 4 minutes and 31 seconds earlier, says certification claims “must be sourced from official NSF or WQA databases” and are “never reproduced without verification.” By April 25 the About page told readers the same thing: “We verify NSF certification claims against the NSF Certified Products Database before publishing.”
I have no record of a lookup in that April session. No listing URL appears anywhere in the April data. A rule written in a planning document is an intention. It checked nothing.
My earlier audit of 95 AI-written blog posts was about prose. This one is about a data file, and data files spread further. One JSON list fed product cards, scores, buying-guide rankings, the filter finder and the markup.
What did the certifier check actually find?
The September 13 session searched all three ANSI-accredited directories for each reviewed product and saved the results. The commit message says “22 of 25.” Two of those 25 are Tap Score water test kits, which carry no certifications at all. So the honest count is 22 of 23 filters. Only the ZeroWater 10-cup pitcher’s list matched.
| What was wrong | Filters |
|---|---|
| Claimed standards, and no matching listing found at any of the three directories | 10 |
| Listed, but the list claimed standards the listing does not show | 4 |
| Listed for more than the list claimed (under-claimed) | 5 |
| A listing exists for a similar model name; claims removed until the model number is confirmed | 3 |
| Correct | 1 |
That is 14 filters showing a standard no certifier lists for them. At the level of individual claims, 15 of the original 49 survived. The other 34 came out: NSF/ANSI 42 twelve times, 53 ten times, P473 five times, 58 four times, 244 twice and 177 once. Ten listed claims went in that the April file had missed.
One example, phrased the way the audit file records it. The NSF and WQA listings, checked on September 13, 2026, show the Clearly Filtered pitcher under NSF/ANSI 42 for chlorine and taste and odor only. The audit found no 53, 244, 401 or PFOA/PFOS listing for it at NSF, WQA or IAPMO. The April file had claimed 42, 53, 244 and P473.
The fix also caught a scoring bug. The filter finder gave five points for every certification, including NSF/ANSI 372. That standard covers lead-free materials, not contaminant reduction, so it now scores nothing.
Is searching the NSF database enough to verify a filter?
No. On my own catalog it would have failed most of the listed products. WQA’s Find Certified Products page says it lists every product that has earned its Gold Seal. NSF’s DWTU listings search by company, brand or model, and standard. IAPMO runs its own product listing directory. All three certify to the same NSF/ANSI standards.
I split the 10 filters that the audit verified by which directory listed them. NSF found 4 of them. WQA found 6. IAPMO found 4. Seven of the ten appear in only one directory. No single directory found more than six.
Until October 13 the About page promised an NSF check. Done exactly as promised, it would have marked 6 of 10 genuinely listed filters as uncertified and docked their scores. The saved NSF results carry their own caveat: “Absence from NSF does not mean uncertified.”
So search all three, by company and by model number, and record the listing URL and the date next to every claim. A one-directory check moves the error from over-claiming to under-claiming. You also publish a methodology that is wrong about its own scope.
Did the corrections reach the live site?
Mostly. On October 13, 2026, thirty days after the fix, I fetched the Clearly Filtered review on www.h2oinsider.com. The body is correct. Its heading reads “Strong PFAS Test Results, No PFAS Certification” and the text says the pitcher is listed under NSF/ANSI 42 and 372.
The same page’s meta description, Open Graph description and Twitter description all still said “with NSF P473 certification for PFAS removal.” So did the summary line above the score, and the brand card on /brands. The claim contradicted the heading a few lines below it.
# October 13, 2026, before the fix
$ curl -s https://www.h2oinsider.com/brands/clearly-filtered-review \
| grep -o '<meta name="description"[^>]*>'
<meta name="description" content="Clearly Filtered leads the pitcher category
on contaminant reduction, with NSF P473 certification for PFAS removal ..."/>The cause is mundane. The correction commits rewrote about 370 certification statements across 62 content files and the product data. The brand descriptions live in a different file, brands.json, whose description lines had not changed since April 24. The page template uses that description as the meta description.
Google’s guidance on generative AI content, updated October 1, 2026, says it is critical to manually fact-check AI-generated content. It applies that review to metadata: title elements, meta descriptions, structured data and alt text. My fix checked the body and left the snippet. The snippet is the line most likely to appear in search results.
A second brand description read “several NSF-certified pitcher models that remove PFAS” for Epic. The audit covered only the one Epic pitcher I review, and found no matching listing for it. That sentence made a claim about models I never checked.
I fixed both the same day. I checked all 15 brand descriptions against the audit file and rewrote four: Clearly Filtered, Epic, Aquasana and PUR. Aquasana’s said “strong NSF certifications” when one of its four reviewed filters has a listing. PUR’s said “NSF 53 certification for lead reduction” when NSF lists that filter under 53 without a lead claim; the lead listing is at IAPMO and WQA.
Does the structured data carry the certification claims?
Not by name. The live pages carry no certification property. But they carry the scores, and the scores were built on the claims. Each buying guide outputs an ItemList of Products with an aggregateRating, and each brand page outputs a Review with a reviewRating.
The re-scoring commit states its formula. Contaminant reduction is 35% of the score, set by certification tier: listed for lead and PFAS 10, lead 8, TDS only 5, NSF/ANSI 42 only 4, no listing 2. Each score moved by 0.35 times the tier change. Nineteen of the 23 filter scores changed.
Before the audit, the five highest-scored filters were 9.2, 9.1, 9.0, 9.0 and 8.9. All five carried a standard no certifier lists for them. Four had no matching listing at any directory. The fifth was the Clearly Filtered pitcher, listed under 42 only. After re-scoring, the top two are the Waterdrop G3P800 at 9.5 and the Aquasana AQ-5300+ at 9.3, both with listings on file.
Google’s structured data policies, updated July 10, 2026, say markup “must be a true representation of the page content.” They say not to mark up misleading content. A rating computed from an unverified claim is that claim in another form. Until September 13, my markup ranked filters on certifications no certifier lists.
Then there is the obvious next idea: mark the certifications up directly. Google’s merchant listing documentation, updated September 8, 2026, supports hasCertification on a Product. It requires issuedBy and name. The supported names are EPREL, Vehicle_CO2_Class and Vehicle_CO2_Class_Discharged_Battery. None is a water standard.
So Google has no supported markup for “NSF/ANSI 53.” A water filter certification lives in your prose, your snippet and your scores. Nobody validates it but you. Put the listing link and the checked date in the visible text, where a reader can click it.
Weren’t the AI’s mistakes the obvious, made-up kind?
No, and this is where I expected to be wrong in a different way. I assumed an audit would find exotic standards invented from nothing. The most-removed standard was NSF/ANSI 42, the most common filter certification there is. Twelve of the 34 removed claims were 42.
The errors were the plausible defaults. A whole-house filter “probably” has 42 and 53. A pitcher sold on PFAS “probably” has P473. The model also under-claimed: Berkey’s April list was empty. NSF’s listing, checked on September 13, shows the Big Berkey under NSF/ANSI 42.
Plausible errors pass a read-through, because they look like what a knowledgeable writer would say. Only a lookup catches them. If you review AI product data by reading it, you will approve exactly the errors that matter.
What did the site promise readers about corrections?
The How We Test page says a wrong certification claim gets corrected with “the date of correction at the top of the article.” The editorial standards page says the same. The About page adds: “We do not silently revise published content.”
On September 13 the Clearly Filtered pitcher lost three claimed certifications and 2.1 points of product score. On October 13 it showed no correction note. The published policy is itself a claim, and it failed the same check the data failed. That afternoon I added a dated note to the top of all 15 brand pages. Each lists every reviewed product’s removed and added standards and its score change, generated from the audit file rather than typed.
Checking the policy pages turned up a bigger promise. The homepage said every recommended filter was “verified by independent Tap Score lab analysis.” The About and How We Test pages described baseline and mid-life lab samples. The site has never had a lab testing budget, and its own voice guide, written on September 12, flags those lines as unconfirmed. They came from the same April build as the certification lists.
On October 13 I removed them: the homepage, About, How We Test, the author bios and the review pages that quoted lab results of ours. Unconfirmed credentials in the bios came out too. The methodology page now says what the site does, which is buy the filters, use them and check the listings, and it says plainly that we do not run lab tests. The September audit checked certification claims and never looked at the claims about how the checking was done.
Did the wrong certifications cost traffic or sales?
I cannot tell you. The repository holds no Search Console export, no analytics data and no affiliate reports for h2oinsider.com. I have no measure of how many people saw the wrong claims, clicked through or bought. The cost in this post is the one I can show: what the pages and markup said, and for how long.
How do you audit AI-written product claims on your own site?
- List every file that holds a product claim. Include brand descriptions, schema builders and anything a template reads into the head.
- Add a source field next to each claim: listing URL, certifier and date checked. A claim without one does not render.
- Search every certifier that issues the standard, by company and by model number. For water filters that means NSF, IAPMO and WQA.
- Record three outcomes: listed, not found, and similar model unconfirmed. Show nothing for the last two until you confirm the model on the unit.
- Recompute every score, ranking and badge that used the old claims, using a formula you state on the page.
- Fetch the live HTML and grep the meta description, Open Graph tags and JSON-LD for the old claim text.
- Add the correction note your own policy promises, with the date.
- Re-run the lookups on a schedule, because listings change and the check has a date on it.
To stop thin or wrong pages before they go live, see building a programmatic SEO quality gate.
Sources
- Google Search Central, Google Search’s guidance on using generative AI content on your website (page updated 2026-10-01).
- Google Search Central, General structured data guidelines (page updated 2026-07-10).
- Google Search Central, Merchant listing (Product, Offer) structured data, Certification properties (page updated 2026-09-08).
- NSF, Certified Drinking Water Treatment Units and Water Filters listings (no updated date shown; checked 2026-10-13).
- Water Quality Association, Find Certified Products (no updated date shown; checked 2026-10-13).
- IAPMO R&T, Product Listing Directory (checked 2026-10-13).
- The H2O Insider repository:
src/data/certification-audit.jsonand the saved directory lookups of 2026-09-13, the git history ofsrc/data/products.json, and live pages on www.h2oinsider.com fetched 2026-10-13.
