The Compliance Checkbox That Does Not Protect You
Compliance teams in financial services have a web archiving requirement. SEC Rule 17a-4, FINRA Rule 2210, FCA COBS 4, MiFID II, ESMA guidelines – the regulatory landscape is clear: if you publish it on your website, you must be able to prove exactly what was published and when.
So organisations buy a web archiving solution, configure the crawl schedule, and check the box. Website archiving: done.
Until a regulator asks to see the archive.
The problem is not that the archive does not exist. It is that the archive does not contain what the website actually showed to visitors. And in a regulatory examination or enforcement action, an incomplete archive is often worse than no archive at all – because it creates a false impression that you were diligent when, in fact, your tooling was not up to the task.
What Crawl Fidelity Actually Means
Crawl fidelity is the degree to which an archived page matches the page as it was experienced by a real visitor at the time of capture. A high-fidelity archive captures the page exactly – every element, every interaction, every piece of content that a visitor would have seen. A low-fidelity archive captures something, but not the complete picture.
The difference matters enormously in regulated industries, because the content regulators care about is often the content that low-fidelity archiving tools miss.
The JavaScript Problem
The single biggest source of archiving failure in 2026 is JavaScript. The majority of enterprise websites are built with JavaScript frameworks – React, Angular, Vue, Next.js, Nuxt – that assemble the page in the browser rather than on the server. When a visitor loads the page, the browser downloads a JavaScript application, executes it, fetches data from APIs, and renders the final content on screen.
A web archiving tool that downloads the raw HTML from the server – without executing the JavaScript – captures an empty shell. The server-delivered HTML for a React application might contain nothing more than a single empty container element with no visible content whatsoever. The actual content – product descriptions, disclaimers, pricing, risk warnings – exists only after the JavaScript runs.
Many archiving solutions still operate this way. They download the HTML, store it in a WARC file, and report the capture as successful. The compliance team sees a green checkbox. But the archive contains none of the content that matters.
Dynamic and Personalised Content
Modern websites do not serve the same content to every visitor. Pricing pages adjust based on the visitor’s location. Product pages display different offers based on browsing history or customer segment. Disclaimers and risk warnings may vary by jurisdiction. Cookie consent banners alter the visible content depending on whether the visitor accepts or declines.
A high-fidelity archiving platform captures these variations because it renders the page through a real browser engine, just as a visitor’s browser would. A low-fidelity tool captures whatever the server returns to an anonymous HTTP request – which may bear little resemblance to what any actual visitor saw.
Login-Gated Content
Financial services organisations increasingly host critical content behind authentication: client portals, investor dashboards, regulatory filing systems, advisor tools. This content is subject to the same archiving requirements as the public website, but it is invisible to any archiving tool that cannot authenticate.
Capturing login-gated content requires the archiving platform to manage browser sessions, handle multi-factor authentication, maintain cookies and tokens, and navigate complex authentication flows. Tools that simply send HTTP requests cannot access this content at all.
Single-Page Applications
Single-page applications (SPAs) load once and then dynamically update the content without full page reloads. From the browser’s perspective, the visitor navigates through multiple “pages” of content. From the server’s perspective, only one HTML document was ever served. A traditional web crawler sees one page where a visitor sees dozens.
This architectural pattern is now standard in financial services web applications. If your archiving tool cannot navigate SPAs – triggering route changes, waiting for content to load, capturing each state – it will miss the majority of the application’s content.
What Regulators Actually Ask For
When the SEC, FINRA, or FCA examines your web archives, they are not testing whether you have an archiving tool. They are testing whether you can produce evidence of what was published on your website at a specific point in time.
The examination typically follows a pattern:
“Show me what was on this URL on this date.” You need to produce a faithful reproduction of the page as it appeared on the requested date. If your archive captured an empty HTML shell because JavaScript did not execute, you cannot comply.
“Show me the disclaimer that accompanied this product offering.” If the disclaimer was rendered by JavaScript, injected by a third-party compliance script, or displayed conditionally based on the visitor’s jurisdiction, your archive must have captured it in that context.
“Show me that this content was not modified after publication.” This requires cryptographic verification of the archived content. Hashing algorithms like SHA-512 create a mathematical proof that the archive has not been altered. Without this, your archive is an assertion, not evidence.
“Show me the complete history of changes to this page.” Regulators increasingly want to see how content evolved over time. This requires multiple captures of the same page at different dates, each faithfully reproducing the page as it appeared.
If your archiving platform cannot satisfy these requests because it lacks crawl fidelity, the consequences are material. FINRA fines for recordkeeping failures have exceeded $100 million in aggregate over the past three years. SEC enforcement actions increasingly cite inadequate website and digital communications preservation.
How Competitors Fall Short
The web archiving market has fragmented into vendors that prioritise different things. Understanding where each category falls short on crawl fidelity is important for making an informed decision.
Communications Archiving Platforms
Some vendors built their businesses on email and messaging archiving. They added web archiving as a secondary feature to offer a broader compliance platform. The engineering investment in their web capture capabilities reflects this – it is a checkbox feature, not a core competency. These platforms typically use lightweight HTTP-based crawlers that do not execute JavaScript, cannot handle SPAs, and have no mechanism for authenticated captures.
Social Media Archiving Vendors
Several vendors expanded from web archiving into social media, collaboration tools, and messaging platforms. This diversification has split their engineering resources across multiple product lines. Their web archiving capabilities, once competitive, have not kept pace with the complexity of modern websites. When your archiving vendor is simultaneously building integrations for Slack, Teams, WhatsApp, and LinkedIn, the investment in solving hard web capture problems inevitably decreases.
Compliance-First Platforms
Some vendors focus primarily on regulatory compliance in specific jurisdictions (particularly UK financial services). Their strength is understanding the regulatory landscape and providing compliance-oriented reporting. However, compliance reporting is only as good as the underlying archive. If the capture engine cannot handle JavaScript-rendered content or dynamic page elements, the compliance reports describe an incomplete record.
What High-Fidelity Archiving Looks Like
A web archiving platform built for crawl fidelity operates fundamentally differently from the tools described above.
Full browser rendering. Every page is loaded in a complete browser engine that executes JavaScript, renders CSS, loads fonts, plays media, and handles cookies – exactly as a visitor’s browser would. The archived page is the page as it was experienced, not the page as it was served.
Session-aware crawling. The archiving platform can authenticate using stored browser profiles, maintaining session cookies, handling MFA flows, and navigating gated content areas. Content behind login walls is captured with the same fidelity as the public site.
SPA navigation. The crawler understands client-side routing and can trigger navigation events, wait for content to load, and capture each state of a single-page application as a discrete archived page.
Scroll and lazy-load handling. Modern websites load content as the visitor scrolls. The archiving platform simulates scroll behaviour, triggering lazy-loaded images, infinite scroll content, and dynamically loaded sections.
Cryptographic verification. Every captured resource is hashed with SHA-512 and RIPEMD-160 at the moment of capture. The archive is stored on WORM (Write Once Read Many) storage. Together, these mechanisms create a tamper-evident chain of custody that transforms the archive from a copy into evidence.
The Cost of Low-Fidelity Archiving
The most dangerous outcome is not a missing archive – it is a false sense of compliance. Your archiving tool reports successful captures. Your compliance dashboard shows green. And when a regulator asks for evidence, you discover that your “archives” are empty shells, static snapshots, or incomplete representations of what was actually on your website.
At that point, the fact that you had an archiving tool works against you. A regulator sees an organisation that invested in compliance tooling but failed to ensure it actually worked. The inference is not flattering.
The alternative is straightforward: test your archives before a regulator does. Pick ten pages from your website – including JavaScript-heavy pages, personalised content, login-gated sections, and SPAs. Ask your archiving vendor to show you the captures. Browse them interactively. Click the links. Check that the disclaimers, pricing, and risk warnings are present.
If the archive matches what your visitors see, your compliance foundation is solid. If it does not, you know what you need to fix – and you know before a regulator tells you.
Next Steps
If you would like to test crawl fidelity against your own website, get in touch. We will capture your site and walk you through the results side by side, so you can see exactly what a high-fidelity archive looks like compared to what you have today.
