Building the Compliance Layer: Product Design for Regulated Data Intermediaries

There is a product category that nobody has bothered to name. And because nobody named it, most builders in this space keep reinventing the same architecture from scratch.

The category is the regulated data intermediary: a platform that sits between a government data authority and a class of licensed professionals, governing access to records that the authority cannot release openly but which professionals need to do their work. The intermediary does not own the data. It governs the terms on which the data is touched.

This sounds narrow. It is not. It describes Plaid between banks and fintech apps. Palantir Gotham between intelligence agencies and analysts. HM Land Registry's portal between the government property database and conveyancers. Singapore's SingPass MyInfo between ICA and any business that needs verified identity. Veeva Vault between pharmaceutical regulators and clinical teams. Epic MyChart between hospital systems and clinical professionals.

Each of these was built as if it were a novel problem. Each converged on the same dozen design patterns.

I spent the last three years building Valuer Copilot, Malaysia's regulated data intermediary for the property valuation market. We sit between JPPH, the government's authoritative property transaction database, and BOVAEP-registered valuers who need vendor and purchaser identity data to complete professional valuations. We are not the first to solve this problem. We are the first to solve it for this market.

Here is what we learned by studying the platforms that got there first.

The Category Nobody Named

Most platforms in regulated markets are built around one of three wrong analogies.

The data warehouse analogy: collect everything, store it, let users query what they need. Wrong for regulated data. The authority never cedes custody. Data is accessed, not stored.

The API gateway analogy: provide a clean interface to an upstream source, add authentication, manage rate limits. Wrong because it ignores the accountability layer. An API gateway logs technical events. A regulated data intermediary logs access events with declared purpose, session identity, professional credential, and firm affiliation.

The compliance-tool analogy: add audit logging, add RBAC, add MFA. Treat these as features bolted onto a product. Wrong because compliance as a layer fails at the edge cases that matter most. The egress controls fail. The audit logs are missing fields that matter to auditors. The session model does not survive the threat model.

The defining characteristic of a regulated data intermediary is that it is accountable for the access event, not just the data. The authority retains sovereignty. The intermediary holds the access record and answers for every entry in it.

This is a different design problem. And it has a different solution shape.

Twelve Patterns. Three Groups.

Seven products. Plaid, Palantir Foundry and Gotham, HM Land Registry, SingPass MyInfo, Veeva Vault, Epic's clinical EHR, LexisNexis Risk Solutions. Different domains, different scales, different regulatory environments. Twelve patterns in common.

I have grouped them into three functional layers.

Access Architecture: Who Gets In, and Why

Pattern 1: Purpose declaration as a first-class UI element. Before a user can access restricted data, they declare why. Not as a text field they can skip. As an architectural gate. Palantir calls these Checkpoints. The declared purpose is stored with the access event, making the audit record readable by a compliance officer, not just a system log parser. The professional sees a brief purpose drop-down, not a legal wall.

Pattern 2: Tiered access with architectural enforcement. Access tiers are enforced at the data layer before results reach the user interface. Plaid strips responses to the minimum authorised fields before transmitting them to any consuming app, even if the upstream bank returns more. A Commercial-tier session in VCP cannot navigate to a regulated endpoint regardless of how the HTTP request is constructed. The restriction is architectural, not advisory.

Pattern 3: Identity-bound sessions with short time limits. HIPAA requires session timeouts for clinical systems. Palantir bounds access windows to specific time periods. VCP caps restricted-field sessions at four hours with MFA re-challenge on extension. The principle is the same: time-unlimited sessions are the mechanism through which authorised access becomes unintended access after the legitimate use case ends.

Pattern 4: Professional credential verification at onboarding. HM Land Registry verifies firm solicitor registration before portal access. Veeva verifies GxP roles. VCP cross-checks BOVAEP registration numbers against the Board's current register. The credential is not self-declared. It is verified against an authoritative source, with annual renewal and automatic access lapse if renewal is not confirmed.

Data Sovereignty: What Users Can See and Take

Pattern 5: View-only rendering with export denial. No platform in this category allows bulk download of restricted fields. The professional sees what they need to do their work. They cannot extract a batch. This is universal. In practice, it means the data is rendered in-browser, never included in PDF exports, never returned in API responses to integrated systems, and never eligible for clipboard copy without triggering an access event.

Pattern 6: Screen watermarking. Every page displaying restricted data carries a watermark embedding the user's identity, session ID, and timestamp. DataPatrol's compliance documentation puts it well: the watermark serves dual purposes. It deters users who know it exists. And it provides forensic evidence when unauthorised sharing is discovered. The watermark should survive screenshots. Dynamic embedding at render time is the implementation approach that achieves this.

Pattern 7: Egress stripping at all output channels. Restricted fields are stripped at every point where data leaves the platform: email gateways, API responses to integrated systems, PDF report generation, and clipboard intercepts. The intermediary is responsible for ensuring restricted data does not travel through any channel. This is a closed set, not a best-effort filter.

Pattern 8: Immutable audit logs with tamper evidence. Veeva's automatic version history, Palantir's tamper-evident audit logs, Epic's HIPAA access records: all share the same architectural property. The log cannot be modified after the fact without detection. The implementation choice that achieves this is an append-only database account with no UPDATE or DELETE grants, plus a daily cryptographic hash chain. Any retrospective modification breaks the chain. Auditors can verify the log has not been altered without asking the platform operator to assert it.

Oversight Infrastructure: How Regulators Observe

Pattern 9: Anomaly detection with behavioural context. Volume thresholds are the baseline: 100 queries per 24-hour period per user is a typical cap. Sophisticated platforms layer in behavioural context. Is this pattern consistent with the user's normal workflow? Does it correspond to an active engagement? Is the geographic origin consistent with prior sessions? Flagged sessions trigger immediate re-verification, not a deferred queue.

Pattern 10: Role and task-based UI. Veeva's core design principle applies broadly: configure the interface by role, showing each user the actions and information their credential authorises rather than a full feature set they must self-restrict. A Commercial-tier user should not see a disabled "view identity data" button. They should not see the route exists. The interface reflects the access model.

Pattern 11: Policy-as-code for purpose enforcement. The emerging architectural pattern: data access policies encoded directly into infrastructure tooling, validated at runtime. Each access request carries a declared purpose tag. The enforcement engine validates requests against declared purposes and blocks mismatches in under one second. This shifts purpose enforcement from human review to automated systems that scale without adding staff.

Pattern 12: The liaison observer portal. Government-adjacent platforms build a distinct access portal for oversight roles. The oversight user, whether a regulator's designated representative, an internal compliance officer, or an external auditor, sees who accessed what and when, without seeing the underlying sensitive data. They receive alert notifications from the anomaly engine. They have structured log exports. They do not share the same interface as operational users. Their access is always read-only.

The Hard Problem: Compliance Without Friction

Here is what most compliance-layer implementations get wrong.

They treat the compliance controls as speed bumps. The purpose declaration is a mandatory field the user resents. The MFA re-challenge interrupts workflow at a random moment. The session timeout logs the professional out mid-valuation. These are not compliance failures. They are UX failures that happen to involve compliance.

The platforms that get this right, Plaid is the clearest example, treat the compliance layer as a UX design problem, not a legal requirement. Plaid's 2025 product data shows a 5% conversion improvement from redesigning their consent flow. Not from removing the consent requirement. From designing the consent flow well enough that users complete it without friction.

The design principle: accountability should be embedded in the access event without adding meaningful delay to the work. The professional should experience the compliance controls as part of their workflow, not as an obstacle to it.

This means the purpose declaration should be a brief drop-down with an optional detail field, not a free-text box that feels like a legal form. The watermark should be present and persistent but visually calibrated to be noticeable rather than obstructive. The session timer should surface 15 minutes before expiry, not at expiry. The anomaly re-challenge should explain what triggered it, not just present a generic MFA prompt.

The Intent Gap framework I wrote about earlier this year applies directly here. The licensed professional's intent is not just "access the transaction data." It is "access the transaction data in a way that I can account for under professional scrutiny." A compliance layer that ignores the second clause creates friction precisely where it should not.

Malaysia's Moment

Malaysia is entering the regulated data intermediary wave at an unusual moment.

MyDigital ID, the national digital credential framework, is targeting 17 million enrolled users by end of 2026, with 95% of federal government services integrated by 2030. The 2026 integration window covers financial services, immigration, and healthcare. Property and professional licensing follow the same wave.

VCP's planned MyDigital ID integration in Q3 2026 would make it among the first regulated professional platforms in Malaysia to anchor session identity to the national digital credential. The session identity problem, verifying that the person querying JPPH data is the licensed valuer on record, becomes substantially simpler when the national identity infrastructure provides the anchor.

The broader pattern here echoes what I described in Regulation as an Operating System: the regulatory stack is not a constraint on what you build. It is a substrate that, read correctly, does your distribution for you. MyDigital ID's rollout is the infrastructure investment that makes regulated professional platforms viable at scale. The government is laying the track. The builders who understand where the track goes first have a structural advantage.

Singapore built SingPass MyInfo in 2016. It now connects to close to 1,000 digital services. Malaysian property data intermediaries will follow the same curve, but on a compressed timeline and with the benefit of SingPass as a design reference. The architecture is already documented. The failure modes are already known.

Three Things Builders Get Wrong

Wrong 1: Building the compliance layer after the product. The most common failure mode. The core product ships. Compliance controls are added as a post-launch hardening cycle. The result is a product where the compliance layer is architecturally separate from the data access layer, meaning every new feature requires a compliance review cycle because the two layers never learned to speak the same language.

The correct sequence: the access model defines the product. Every feature is scoped against it. The compliance layer and the product layer are the same layer.

Wrong 2: Treating the oversight user as a secondary persona. Most platforms treat the regulator's representative, the liaison observer, the internal audit function, as a viewer of logs. An afterthought. The best platforms treat the oversight role as a first-class user with distinct needs: real-time alert access, structured export formats, aggregate metrics, and a UI that reflects the oversight function rather than the operational one.

Building for the oversight user well is what makes a regulated data intermediary credible to regulators, which is the only distribution channel that matters in a market gated by government approval.

Wrong 3: Designing the purpose declaration as a friction gate. The purpose declaration is the most important UX element in the compliance layer. It is also the element most often designed as a text field that users resent, skip-with-a-boilerplate response, or eventually ignore.

The purpose declaration that works is a structured choice with a short controlled vocabulary: the professional selects from four or five categories that cover the actual use cases. An optional detail field is available but not required. The declared purpose is stored with the access event and available to auditors. The whole interaction takes under three seconds.

This is not hard UX design. It is UX design that requires the builder to understand the professional's workflow well enough to define the right vocabulary of purposes.

The regulated data intermediary category has been waiting to be named. The design patterns are documented, distributed across platform documentation, academic papers, and practitioner blogs in different industries that rarely read each other.

The builders in ASEAN GovTech, PropTech, and HealthTech who find these patterns early have a significant advantage. Not because the patterns are secret. Because most builders are still treating the compliance layer as a cost centre rather than the architecture that makes the product defensible.

Twelve patterns. Three groups. One category.

Build accordingly.

KG is ACMA, CGMA and General Manager of PEPS Ventures Berhad, where he leads the product and regulatory strategy for Valuer Copilot, ValuationXchange, and PaymentXchange. He works at the intersection of Malaysian property regulation, government data access, and product architecture.