One gateway, one warehouse: a reference architecture for AI-ready data in a UAE mid-market group

Digital transformation

The bottleneck for AI in a UAE mid-market group is not the model. It is that the same customer and the same invoice are re-keyed across three to eight systems and no layer owns the definition. Here is a reference architecture that fixes identity first, builds UAE data obligations into the design, and starts with one process instead of a platform programme.

Your AI programme is not stuck on models. It is stuck on the fact that the same customer, the same item and the same invoice exist in three to eight systems, each with its own numbers, and no layer in your estate owns the definition. A UAE mid-market group running an ERP, a CRM, an HRMS, a POS, a ticketing tool and a couple of SaaS subscriptions will re-key "Gulf Trading LLC" five times with five spellings, then ask a language model to reason about it. The model will oblige, confidently, on dirty data.

This reference architecture fixes the identity problem first: change-data-capture into a Bronze, Silver and Gold lakehouse, a governed semantic layer with one customer, one product and one invoice, an OpenAI-compatible gateway in front of your own GPUs and hosted providers, and agents that write back to the ERP only through an audited, approved path. The UAE obligations you have to meet (PDPL, sector residency, keeping prompts and model weights in-country) are designed into the layers, not bolted on afterwards. At the end is a decision checklist and a 12-week sequence that starts with one process, not a platform programme.

Why identity, not models, is the bottleneck

The conversations we have with CTOs and heads of data in the UAE all follow the same shape. The board has asked for AI. Someone has read that retrieval-augmented generation or an autonomous agent will transform operations. Then we open the ERP and the CRM and find that there is no shared key: the customer is identified by an account number in the ERP, an email address in the CRM and a loyalty card in the POS. The item is "Widget 22mm" in one system and "WDG-22" in another. The invoice has a credit note in the ERP that the billing spreadsheet has never seen.

Three consequences follow, and they cost more than any model licence:

1. Every answer is a negotiation. Finance and sales reconcile numbers by phone call. An agent that pulls from both systems inherits the argument and makes it faster and wrong.
2. No one owns the definition. "Revenue recognised" means something different in the ERP, the CRM dashboard and the board pack. When there is no owner, there is no fix.
3. Write-backs become dangerous. Once an agent can act (issue a credit note, update a price list, close a ticket), acting on a guessed identity is a real financial event, not a chat message.

The promise of this architecture is straightforward: spend the first weeks on identity and definitions, and the model work becomes an integration exercise instead of a research project. The bottleneck is data, and data is a solved engineering problem.

The reference architecture at a glance

  ERP (SQL Server / Oracle)   CRM (SaaS)   POS   HRMS   Ticketing   SaaS APIs
        |                       |           |      |        |           |
        +---- Debezium CDC -----+           +------|--------|-----------+  (binlog / transaction log
        |                       |                  |        |                    + dual-write / API extract)
        v                       v                  v        v
  ------------------------------------------------------------------
  | BRONZE  raw, immutable, exactly as received, one table per source |
  | SILVER  cleansed, conformed, deduplicated: ONE customer, ONE item  |
  | GOLD    marts per domain: invoice ageing, stock, service levels    |
  ------------------------------------------------------------------
        |
        v
  Semantic layer: certified definitions (one customer, one product, one invoice),
  metrics with owners, row-level access rules
        |                                        |
        v                                        v
  BI dashboards / SQL              OpenAI-compatible model gateway
  (human consumers)                (routing, budgets, access control)
                                               |
                                               v
                                  Agents with scoped tool access
                                  (read: gold + knowledge base; write: via approval)
                                               |
                                               v
                                  Append-only audit ledger -> back to ERP/CRM

Each layer has a single owner and a single job:

Ingestion: change data capture, not nightly exports

Nightly full extracts fail you in three ways: they are slow at month end when you need them most, they cannot tell you what changed (so Silver logic must diff whole tables), and they turn the ERP into a batch bottleneck. Change-data-capture (CDC) reads the transaction log instead of the tables. For SQL Server and Oracle, the standard route is Debezium running on Kafka Connect; the SQL Server connector documentation covers capture instances, snapshot modes and schema history in detail.

A typical connector for an ERP sitting on SQL Server looks like this:

{
  "name": "erp-sqlserver-connector",
  "config": {
    "connector.class": "io.debezium.connector.sqlserver.SqlServerConnector",
    "database.hostname": "erp-sql.internal",
    "database.port": "1433",
    "database.user": "cdc_reader",
    "database.password": "${file:/opt/kafka/secrets:erp_cdc_password}",
    "database.dbname": "ERP_PROD",
    "topic.prefix": "erp",
    "table.include.list": "dbo.Customer,dbo.SalesInvoice,dbo.SalesInvoiceLine,dbo.Item",
    "snapshot.mode": "initial",
    "schema.history.internal.kafka.bootstrap.servers": "kafka:9092",
    "schema.history.internal.kafka.topic": "erp.schemahistory",
    "transforms": "unwrap",
    "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState"
  }
}

Three practical points from the field:

• Large object columns need explicit handling. Debezium's default mode handles numeric, string and temporal columns and ignores others, including LOB fields, unless you configure LOB support. Decide per column whether you need document blobs in Bronze at all. Usually you need the metadata and a pointer to the document store, not the PDF itself.
• Schema history is part of your data. Store the schema history topic durably and back it up with the warehouse. Without it you cannot replay a table after a DDL change.
• Credentials and TLS belong in secrets, not in connector JSON. The pattern above loads the password from a mounted secret file. Rotate it like any production credential.

If your group runs Microsoft Dynamics 365 or already holds data in ADLS Gen2 or S3, there is an alternative path: OneLake shortcuts let Microsoft Fabric read external storage in place rather than copying it. The OneLake shortcuts documentation lists the supported sources including ADLS Gen2, Amazon S3, Google Cloud Storage and Dataverse. Shortcuts move bytes zero times; CDC still gives you change semantics for on-premises transactional systems.

In practice you will use two of these: CDC for the on-premises transactional core, and API extracts or dual-write for SaaS tools. Both land in Bronze in the same shape.

Bronze, Silver, Gold: where the definitions actually live

The medallion naming comes from Databricks lakehouse guidance: Bronze keeps raw records as is, Silver holds cleansed, validated and conformed data, and Gold serves business-level aggregates and metrics, as detailed in the medallion architecture docs. The layering language is useful even if your warehouse is Snowflake, BigQuery, Postgres or Fabric. The discipline that matters is ownership: identity lives in Silver, business meaning lives in Gold, and no dashboard or agent is allowed to invent either.

Silver is where you resolve duplicate identity. For a UAE group, the natural key is usually the trade licence number for legal entities and the Emirates ID or mobile number for individuals, subject to PDPL minimisation (store a hash or the last four digits if you do not need the full identifier operationally). A Silver customer model with conformed keys and tests looks like this:

-- models/silver/dim_customer.sql (Databricks SQL / Spark SQL)
with unioned as (
    select trade_licence_no, customer_name, phone, updated_at, 'ERP' as source_system
    from stg_erp_customer
    union all
    select trade_licence_no, customer_name, phone, updated_at, 'CRM'
    from stg_crm_account
    union all
    select trade_licence_no, customer_name, phone, updated_at, 'POS'
    from stg_pos_member
),
ranked as (
    select
        md5(trade_licence_no) as customer_key,
        trade_licence_no,
        customer_name,
        phone,
        source_system,
        updated_at,
        row_number() over (
            partition by trade_licence_no
            order by updated_at desc
        ) as rn
    from unioned
    where trade_licence_no is not null
)
select * except (rn) from ranked where rn = 1

# models/silver/_silver.yml
version: 2
models:
  - name: dim_customer
    description: "One customer per trade licence number across ERP, CRM and POS."
    columns:
      - name: customer_key
        tests: [unique, not_null]
      - name: trade_licence_no
        tests: [unique, not_null]
      - name: source_system
        tests:
          - accepted_values:
              values: ['ERP', 'CRM', 'POS']
      - name: updated_at
        tests: [not_null]

Run those tests on every load and publish the failure rate next to the pipeline. A Silver layer without automated assertions is just an assumption.

For the semantic layer you have four realistic options, and the choice is usually settled by what you already run:

Whatever you pick, every certified metric needs a name, a formula, an owner, a refresh expectation and the source systems it draws on. "One invoice" is a good first test: does it include credit notes? Pro-forma documents? If your answer is "ask finance", the definition does not exist yet.

UAE constraints, designed in from the first sprint

Governance in the UAE is consolidating. In June 2026 the Cabinet established the UAE Artificial Intelligence and Data Authority (AIDA) under Cabinet Resolution No. (68) of 2026, folding the former Digital Authority remit into one regulator that issues the AI and Digital Codes of Practice and coordinates with sector regulators on data classification and risk (analysis of AIDA and what it means for governance). For a CTO, consolidated oversight means one direction of travel: documented data classification, demonstrable control over personal data, and clear accountability for automated decisions. Design for that now rather than re-platforming later.

Four constraints change the architecture diagram itself:

1. PDPL obligations sit in the semantic layer, not in a policy PDF. Federal Decree-Law No. 45 of 2021 and its executive regulations require purpose limitation, minimisation and a lawful basis per processing activity, and they constrain cross-border transfers. In this architecture this shows up concretely: personal data columns classified at the schema level, row-level access rules enforced in the semantic layer, and retention rules executed in Silver rather than left to individual application settings. PromptQuorum's analysis of PDPL and local AI deployment provides a useful plain-English map of the executive regulations and what on-premises processing actually secures.
2. Sector residency expectations are real, especially for financial and health data. Expectations differ by sector and sensitivity: financial data residency (with sovereign financial cloud options among them), health data kept in-country, and additional classification requirements for government supply chains. Hyperscaler UAE regions and sovereign offerings exist precisely for this (sector residency expectations in the UAE). Put Bronze, Silver and Gold for regulated data in a UAE region or on your own infrastructure. Analytics replicas abroad are the classic accidental violation.
3. Prompts, embeddings and model weights are data too. If your legal or risk team is uncomfortable with personal data leaving the country, they should be equally uncomfortable with the same data leaving inside a prompt to a foreign hosted endpoint. The practical answer is tiering: keep an in-country or on-premises path for regulated workloads (open-weights models on your own GPUs, vector stores and logs inside the UAE), and use hosted providers for workloads whose data classification allows it. PromptQuorum frames this as criteria covering where weights, inference, vector databases, monitoring logs and support personnel actually sit. Use that as a procurement checklist whenever a vendor claims sovereignty.
4. Every automated action needs a reconstructable trail. When an agent changes a price or issues a credit note, regulatory codes of practice and your own auditors will expect you to show the input, the decision rationale, the approval and the resulting transactional record.

The gateway, the agents and the audit trail on write-back

Do not let each internal application hold its own provider API key. Put an OpenAI-compatible gateway in front of everything: your own LLM servers, plus hosted providers, with routing, budgets and access control in one place. Applications and agents call one endpoint and one model alias; the gateway resolves the alias to an active provider, applies the team's monthly budget and logs the transaction. That is exactly what FastLLM Proxy provides in front of your own LLM servers and hosted providers, and the client code stays completely standard:

import os
from openai import OpenAI

# The base URL points at your internal gateway, not at an external provider.
client = OpenAI(
    base_url="https://llm-gateway.internal/v1",
    api_key=os.environ["AGENT_API_KEY"],
)

resp = client.chat.completions.create(
    model="dept-finance-default",          # alias, resolved by the gateway policy
    messages=[
        {"role": "user",
         "content": "Summarise overdue invoices for customer 10428."}
    ],
    # Pass identity through so the gateway can attribute cost and access
    extra_body={"x-agent-id": "credit-control-assistant", "x-cost-centre": "finance"},
)

Two deployment patterns earn their keep here. First, run agents and models on your own GPU estate when the data classification requires it: Kuvryn AI is built for this model, letting teams run agents and models on your own GPU servers, each in its own isolated space. An open-weights model of the 7B class quantised to 4-bit fits comfortably on a single 24 GB card and is sufficient for classification, structured extraction and routing tasks. Second, give the team and the agents one shared knowledge base to read and write: Kryton provides a self-hosted knowledge base with wiki links and a knowledge graph where humans and assistants work from the same source of truth. Your agents are only as accurate as the documents they can inspect.

Now comes the element most reference architectures omit: the operational write-back. If an agent can act on the ERP, every action must follow this sequence:

1. The agent produces a proposed change as a structured tool call (customer ID, action, payload), never as unstructured free text.
2. A policy check validates the action against the semantic layer (does this customer key exist? is this price within the agent's authorised band?).
3. Actions above a risk threshold enter an approval queue where a named team member approves or rejects. Start with everything requiring human approval, relaxing only with documented operational evidence.
4. The approved action is executed against the core system and the ERP transaction ID is captured.
5. An append-only ledger records the entire chain:

{
  "ts": "2026-11-04T09:31:22+04:00",
  "agent_id": "credit-control-assistant",
  "actor": "user:m.alharbi@company.ae",
  "model_alias": "dept-finance-default",
  "prompt_hash": "sha256:9f2c7104…",
  "input_ref": "gold.fct_invoice_ageing:inv-10428",
  "action": "issue_credit_note",
  "payload_hash": "sha256:1b77a902…",
  "approval": {"by": "user:finance.manager", "at": "2026-11-04T09:33:01+04:00"},
  "result": {"erp_txn_id": "CN-2026-00841", "status": "committed"}
}

Keep this ledger outside both the model and the ERP, immutable, and retained according to your PDPL retention schedule. This is the artefact that turns an opaque agent execution into a record you can confidently present to an auditor.

Anti-patterns and the 12-week sequence that replaces them

Stop these common shortcuts before committing budget to new tooling:

A worked example. Consider a typical scenario we encounter: a Dubai distribution group with three legal entities, four core systems (ERP on SQL Server, cloud CRM, POS on Oracle, SaaS ticketing), approximately 27,400 customer records across systems, roughly 6,100 invoices a month, and a collections team that spends its day asking who a customer is and what they currently owe. Choose one process to start: credit control on overdue invoices. It spans three systems, it carries a measurable financial cost (days sales outstanding), and its core output is a figure finance and sales already dispute.

After week 12, add write-back behind an approval step for a single low-risk action (such as sending a formal reminder or logging an agreed payment date), then extend the pattern to your next operational process. Integrating three systems in twelve weeks is standard engineering. Spending twelve months on eight systems without delivering a single live process is also common, and it is the outcome this architecture avoids.

Decision checklist and what to do next

Use this checklist in your next architectural review. If your team cannot answer two or three of these questions, that identifies your immediate work package:

1. Identity: What is the natural key for customer, item, invoice and employee, and who officially owns each definition?
2. Change capture: For each source system, is it log-based CDC, a scheduled extract or an API pull, and where is schema drift detected?
3. Layering: Do you maintain Bronze (raw, replayable), Silver (conformed, tested) and Gold (business marts) layers with named owners?
4. Semantics: Where do certified metrics live, and can finance and sales both query a single definition of revenue and overdue amounts?
5. Placement: For each data class (regulated financial, health, personal, internal), which UAE region or on-premises environment holds it, and where are prompts, embeddings, logs and model weights processed?
6. Gateway: Is there one OpenAI-compatible endpoint with model aliases, budgets and access controls, or are raw API keys scattered across applications?
7. Action: When an agent writes back, is there a structured proposal, a schema policy check, an approval threshold and an append-only ledger capturing the resulting ERP transaction ID?
8. Regulatory mapping: Who at the executive level owns AI risk under the consolidated AIDA oversight, and can they produce your data classification and transfer documentation on request?

What to do next, concretely: pick one operational process this month whose pain is already on your executive agenda, run the two-week definitions workshop on it, and build only the ingestion and transformation pipelines that process requires. That is how we work with UAE clients at Azrty: we map how the business runs, connect the systems and bring the data together first (digital transformation), and we tell you plainly where AI will pay off and what it takes (AI strategy). Azrty is a UAE company based in Dubai. Our own team in the UAE does the consulting, the deployment and the support for our clients. The model is the straightforward part. Own your definitions, and everything after that is engineering.

https://www.azrty.com/blog/one-gateway-one-warehouse-a-reference-architecture-for-ai-ready-data-in-a-uae-mi

<a href="https://www.azrty.com/blog/one-gateway-one-warehouse-a-reference-architecture-for-ai-ready-data-in-a-uae-mi">One gateway, one warehouse: a reference architecture for AI-ready data in a UAE mid-market group</a> (Azrty)