A Zoho-to-Supabase sync agent gives your team something Zoho on its own cannot: a fast, queryable, AI-ready mirror of your CRM that you can join against orders, ad spend, support tickets, or anything else in your Postgres warehouse. This tutorial is the production-grade version — not the demo.
The reason matters. Zoho is excellent as a system of record and as a front-end UI for operations teams. It is awkward as an analytics backend, expensive as a search engine, and impossible as a vector store. A one-way mirror into Postgres flips all three constraints without changing how your sales team works.
What follows is a working pattern for the Leads and Deals modules, covering server-to-server OAuth, COQL deltas keyed by Modified_Time, snake_case schema mapping, idempotent upserts with conflict targets, a Vercel cron trigger, and the observability layer that lets you sleep through a long weekend without paging yourself.
- 01Modified_Time as a watermark works — until it doesn't.Concurrent edits during a read can be missed by a strict greater-than cursor. Use a small overlap window (e.g. last_run minus 60 seconds) or COQL's id-cursor pattern to recover the gap on the next pass.
- 02Snake_case in Postgres is non-negotiable.Zoho's CamelCase field names are fine inside Zoho. In Postgres they make every query awkward and every join brittle. Map at ingestion time, not at read time, and store the mapping in source.
- 03Upsert with on_conflict is the right primitive — but watch the index choice.A natural-key unique index on the Zoho id covers the steady state. Add a partial unique index where you need to support soft deletes or domain-level dedup. Get the index wrong and the upsert silently inserts duplicates.
- 04Lock before sync. Always.Two cron invocations overlapping is a guaranteed primary-key conflict storm and a Zoho rate-limit blowout. A single row in a sync_locks table, claimed with INSERT … ON CONFLICT DO NOTHING, is enough to serialize runs.
- 05Observability is what makes this production.A runs table that records start/end/status/counts, a Slack alert on consecutive failures, and a small dashboard reduces operator load from constant to occasional. Without it, this pipeline is a future incident waiting for a calendar.
01 — Why MirrorCRM is the system of record — Postgres is the system of query.
A mirror is not a migration. The CRM remains authoritative for inserts, updates, ownership, workflow, and the front-end UI your consultants use every day. What the mirror unlocks is everything that lives downstream of the CRM record — and there is more of it than most teams realise.
The four workloads that justify a mirror, in our experience: analytics that join CRM against revenue, ad spend, or product telemetry; reporting that needs to span multiple modules and calculate derived metrics; AI features that read CRM context (RAG over past deals, scoring, summary generation); and search that needs to be fast across thousands of records with arbitrary filters.
The cases where a mirror does not pay back: low record counts (under a few thousand) where the Zoho API is fast enough; workloads that require writing back to the CRM in the same operation; and teams without a Postgres-fluent engineer to maintain the pipeline. If you are not sure which bucket you are in, start with the questions below.
When a one-way mirror pays back
Analytics · AI · SearchMulti-module reporting, RAG over CRM history, sub-second search across the full record set, joins against revenue or product data. Anywhere the Zoho UI is the wrong tool but the Zoho data is the right data.
Most agency workloadsWhen the CRM API is enough
Small org · Low query rateFewer than a few thousand records, queries measured in tens per day, no derived analytics. The mirror adds operational complexity without unlocking anything you cannot do with a direct API call.
Small CRMsWhen you need bidirectional sync
Webhooks · CDC · Conflict resolutionWrite-back to the CRM in the same workflow, multi-master operation, or external systems that need to push records into Zoho. A separate design — different rules, different failure modes, more moving parts.
Out of scope hereOne more upstream choice worth naming. Webhook-driven CDC (push) and polling-based delta extraction (pull) are both legitimate architectures, and many production pipelines use both — webhooks for near-real-time updates on hot records, polling as the backstop. This tutorial focuses on the polling path because it is the one that fails the worst when implemented naively, and the one that benefits most from getting the contract right. Add webhooks on top once the pull side is reliable.
02 — Zoho AuthServer-to-server OAuth — the refresh token lifecycle.
Zoho OAuth for server-to-server integrations is a two-token model. You register a self-client (or a server-based client) in the Zoho API Console, generate a one-time grant code, exchange it for a refresh token that does not expire under normal operation, and then mint short-lived access tokens (60-minute TTL) on demand. The refresh token is the long-lived credential; treat it like any other production secret.
Three scopes cover everything in this tutorial: ZohoCRM.modules.leads.READ, ZohoCRM.modules.deals.READ, and ZohoCRM.coql.READ. Add ZohoCRM.bulk.READ if you plan to use the bulk read endpoint for the initial backfill (Section 6 in the FAQ block at the end). Do not over-grant; a read-only mirror should hold read-only credentials.
Below is the minimal access-token broker. It refreshes lazily, caches in module scope to avoid a refresh per request, and falls back to a forced refresh if Zoho returns a 401 mid-run.
// lib/zoho/auth.ts
const ZOHO_ACCOUNTS = process.env.ZOHO_ACCOUNTS_URL!; // e.g. accounts.zoho.eu
const REFRESH_TOKEN = process.env.ZOHO_REFRESH_TOKEN!;
const CLIENT_ID = process.env.ZOHO_CLIENT_ID!;
const CLIENT_SECRET = process.env.ZOHO_CLIENT_SECRET!;
type Cached = { token: string; expiresAt: number };
let cached: Cached | null = null;
export async function getAccessToken(force = false): Promise<string> {
const now = Date.now();
if (!force && cached && cached.expiresAt > now + 60_000) {
return cached.token;
}
const params = new URLSearchParams({
refresh_token: REFRESH_TOKEN,
client_id: CLIENT_ID,
client_secret: CLIENT_SECRET,
grant_type: 'refresh_token',
});
const res = await fetch(`https://${ZOHO_ACCOUNTS}/oauth/v2/token`, {
method: 'POST',
headers: { 'content-type': 'application/x-www-form-urlencoded' },
body: params,
});
if (!res.ok) throw new Error(`Zoho refresh failed: ${res.status}`);
const json = await res.json() as { access_token: string; expires_in: number };
cached = {
token: json.access_token,
// Zoho returns ~3600s; cache for slightly less to avoid edge expiry.
expiresAt: now + (json.expires_in - 120) * 1000,
};
return cached.token;
}Three production touches worth lifting out. First, the cache window is two minutes shorter than the token TTL — Zoho will occasionally mark a token expired a few seconds before its nominal expiry, and the buffer eliminates that class of flake. Second, every fetch against the Zoho API should be wrapped in a helper that catches a 401 once, calls getAccessToken(true) to force a refresh, and retries the original request before giving up. Third, the refresh token itself should be rotated annually — not because Zoho will revoke it, but because the discipline of rotation forces you to verify the credentials path still works.
Region matters. Zoho operates separate data centers (com, eu, in, au, jp, ca) and the accounts host plus the API host both vary by region. Use the host pair tied to the org your refresh token was issued for, not the global default — a mismatch produces a confusing "invalid client" error rather than a clear region error.
03 — COQL DeltasModified_Time as the watermark.
COQL is Zoho's SQL-like query language, exposed at /crm/v8/coql. For a mirror, the right query shape is a modified-time delta — pull every record whose Modified_Time is greater than the last successful run, paginated until exhausted. The maximum page size is 200, the maximum offset is 2000, and pagination beyond that requires a cursor strategy (covered below).
// lib/zoho/coql.ts
const ZOHO_API = process.env.ZOHO_API_URL!; // e.g. www.zohoapis.eu
export type CoqlResponse<T> = {
data: T[];
info?: { more_records: boolean; count: number };
};
export async function coql<T>(query: string): Promise<CoqlResponse<T>> {
const token = await getAccessToken();
const res = await fetch(`https://${ZOHO_API}/crm/v8/coql`, {
method: 'POST',
headers: {
'Authorization': `Zoho-oauthtoken ${token}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({ select_query: query }),
});
if (res.status === 204) return { data: [] };
if (!res.ok) throw new Error(`COQL ${res.status}: ${await res.text()}`);
return res.json() as Promise<CoqlResponse<T>>;
}
export async function fetchLeadsDelta(sinceIso: string) {
const out: ZohoLead[] = [];
let offset = 0;
// 60-second overlap protects against records modified during read.
const watermark = new Date(new Date(sinceIso).getTime() - 60_000)
.toISOString();
while (true) {
const page = await coql<ZohoLead>(`
SELECT id, First_Name, Last_Name, Email, Phone, Lead_Status,
Lead_Source, Owner.id, Owner.full_name,
Created_Time, Modified_Time
FROM Leads
WHERE Modified_Time > '${watermark}'
ORDER BY Modified_Time ASC
LIMIT ${offset}, 200
`);
out.push(...page.data);
if (!page.info?.more_records) break;
offset += 200;
if (offset >= 2000) {
// Switch to id-cursor mode for runs that exceed the offset cap.
return continueWithIdCursor(out, watermark);
}
}
return out;
}Three things in that snippet are doing real work. The 60-second overlap window catches records modified during the read — if a consultant edits a lead while pagination is in flight, that record's Modified_Time may move past the cursor and would be missed by a strict greater-than. The ORDER BY Modified_Time ASC makes the result set deterministic, which matters once you start switching to the id-cursor pattern. And the early return at the 2000-row offset cap is where the naive tutorial pattern silently breaks at scale — production code has to handle it.
The id-cursor pattern is straightforward. Once you hit the offset cap, re-issue the query with a new WHERE Modified_Time = <last_row_time> AND id > <last_id> clause and paginate forward from there. It is more code than the offset path, but it is the only reliable way to read past 2000 modified records in a single window.
WHERE Modified_Time > last_run with no overlap window will eventuallymiss a record. Zoho's modified-time precision and the latency between the API and your sync are non-zero; without the overlap, a record modified at exactly the cursor boundary slips through. The cost of the overlap is a few duplicate upserts per run, which the idempotent upsert in Section 05 absorbs at no observable cost.04 — Schema MappingZoho fields → snake_case Postgres columns.
The single most consequential design decision in a CRM mirror is where the schema mapping lives. There are three options: map at ingestion (rewrite payloads before insert), map at read (use views or a query layer to rename), or do not map at all (store Zoho-style field names in Postgres). The right answer is almost always ingestion. The other two trade ingestion-time effort for query-time pain that compounds with every analyst, every dashboard, and every AI feature that touches the data.
The table below covers the type coercions you will encounter most often. Build the mapping table once, keep it in source, and treat it like any other piece of schema — reviewed in PRs, versioned with migrations, tested against fixtures.
Zoho Single Line / Email → text
Straight rename. CamelCase → snake_case. NULL-safe. Trim trailing whitespace; Zoho returns it inconsistently from imports and from form inputs.
TEXT, NOT NULL where requiredZoho ISO-8601 → timestamptz
Zoho returns ISO-8601 with timezone offset. Coerce to TIMESTAMPTZ at ingestion and store in UTC. Index Modified_Time — it is the most-queried column in any mirror.
TIMESTAMPTZ + indexZoho Picklist → text + CHECK
Resist the urge to use a Postgres ENUM. Picklist values change in Zoho more often than you expect, and ENUM migrations are awkward. TEXT with a CHECK constraint, refreshed by a migration when Zoho adds values, is the safer pattern.
TEXT + CHECK constraintZoho Owner → owner_id + owner_name
Zoho returns lookups as nested objects with id + display name. Flatten to two columns at ingestion: id for joins, display name as a denormalized convenience. Do not store the raw object unless you intend to mirror users separately.
Two columns: id + labelUnknown / sparse → jsonb
Org-specific custom fields belong in a custom_fields jsonb column at first. Promote individual fields to first-class columns only after they prove themselves load-bearing in queries.
JSONB columnZoho Currency → numeric(14,2)
Never store currency as float. NUMERIC with explicit precision and scale. Store the currency code separately if your org is multi-currency. Zoho's exchange-rate field is per-record — mirror it if your reporting needs base-currency totals.
NUMERIC + currency_codeOne pattern worth lifting out: the custom_fields jsonb column. Zoho orgs accumulate custom fields, often dozens of them, many sparse. Promoting all of them to first-class columns up front is a sunk cost most orgs do not recover. The pragmatic approach is a single custom_fields jsonb column populated at ingestion with everything the mapping table does not recognise, plus a GIN index on the keys most often queried. Promote a field to its own column the day a query starts caring about it.
-- migrations/0001_create_leads.sql
CREATE TABLE leads (
id TEXT PRIMARY KEY, -- Zoho id
first_name TEXT,
last_name TEXT,
email TEXT,
phone TEXT,
lead_status TEXT,
lead_source TEXT,
owner_id TEXT,
owner_name TEXT,
custom_fields JSONB NOT NULL DEFAULT '{}'::jsonb,
created_time TIMESTAMPTZ NOT NULL,
modified_time TIMESTAMPTZ NOT NULL,
synced_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX leads_modified_time_idx ON leads (modified_time);
CREATE INDEX leads_lead_status_idx ON leads (lead_status);
CREATE INDEX leads_owner_id_idx ON leads (owner_id);
CREATE INDEX leads_custom_fields_gin ON leads USING GIN (custom_fields);The synced_atcolumn at the bottom is the small detail that pays back later. When a query produces an unexpected result, the first question is always "how stale is this row?" —synced_at answers it in one column without needing to cross-reference the runs table.
05 — UpsertsINSERT … ON CONFLICT DO UPDATE.
The upsert pattern is the heart of the mirror. Every record from COQL goes through the same path — INSERT if new, UPDATE on conflict — keyed on the Zoho id. The conflict target is the primary key (the Zoho id), the update set is every mirrored column except the id itself, and a WHERE excluded.modified_time > leads.modified_time guard prevents an older payload from overwriting a newer row in the rare case a delta arrives out of order.
// lib/sync/upsert-leads.ts
import { createClient } from '@supabase/supabase-js';
const supabase = createClient(
process.env.SUPABASE_URL!,
process.env.SUPABASE_SERVICE_ROLE_KEY!,
{ auth: { persistSession: false } },
);
export async function upsertLeads(rows: MappedLead[]) {
// Supabase upsert sets ON CONFLICT DO UPDATE for all non-pk columns.
// The 'ignoreDuplicates: false' is explicit because the default differs by version.
const { error, count } = await supabase
.from('leads')
.upsert(rows, {
onConflict: 'id',
ignoreDuplicates: false,
count: 'exact',
});
if (error) throw error;
return count ?? 0;
}
// Or via raw SQL when you need the modified_time guard:
const sql = `
INSERT INTO leads (id, first_name, last_name, email, phone, lead_status,
lead_source, owner_id, owner_name, custom_fields,
created_time, modified_time, synced_at)
SELECT * FROM jsonb_to_recordset($1::jsonb)
AS x(id text, first_name text, last_name text, email text, phone text,
lead_status text, lead_source text, owner_id text, owner_name text,
custom_fields jsonb, created_time timestamptz, modified_time timestamptz,
synced_at timestamptz)
ON CONFLICT (id) DO UPDATE SET
first_name = EXCLUDED.first_name,
last_name = EXCLUDED.last_name,
email = EXCLUDED.email,
phone = EXCLUDED.phone,
lead_status = EXCLUDED.lead_status,
lead_source = EXCLUDED.lead_source,
owner_id = EXCLUDED.owner_id,
owner_name = EXCLUDED.owner_name,
custom_fields = EXCLUDED.custom_fields,
modified_time = EXCLUDED.modified_time,
synced_at = EXCLUDED.synced_at
WHERE EXCLUDED.modified_time > leads.modified_time;
`;Two practical notes. First, batch size matters. The Supabase client handles batching internally up to a generous limit, but Postgres performance on jumbo upserts is better at 500 to 1000 rows per statement than at 5000 — split larger result sets at the application layer. Second, the modified-time guard is the backstop, not the strategy. If your pipeline is producing out-of-order deltas regularly, the right fix is in the read path (the id-cursor pattern from Section 03), not in the upsert path.
For the Deals module, the same pattern applies with one extension: soft deletes. Zoho can mark a record deleted without removing it from the system; the COQL filter WHERE $deleted = false excludes them, but you also want a separate pass that picks up deletions and marks them in the mirror — either a deleted_at timestamptz column with a partial unique index, or a separate leads_deletedtable populated from Zoho's deleted-records endpoint.
06 — IdempotencyTokens, backoff, and resumable runs.
A production pipeline fails. The question is whether it fails cleanly. Three failure modes account for almost everything: Zoho rate limiting, network errors during a long run, and schema drift (a new Zoho field, a removed picklist value, a column type mismatch). Each has a different right handler.
HTTP 429 / quota
Exponential backoffRetry with jitter at 1s, 2s, 4s, 8s, 16s. Capped at five retries. After five, mark the run failed and resume from the last successful watermark on the next cron tick. Do not loop forever — Zoho quotas reset on the hour.
Recoverable on next runTimeout / 5xx
Resume from last idPersist the last processed Zoho id and watermark after every successful batch. A mid-run network failure resumes from the last checkpoint rather than restarting from the watermark — important when a single run pulls thousands of records.
Checkpoint per batchUnknown field / type
Quarantine + alertIf the COQL response contains a field the mapping does not recognise, route the affected rows into a quarantine table and continue. Alert the team. Do not block the rest of the run; the mirror is more useful slightly stale than fully halted.
Hard alert · soft continueThe idempotency token is the design that ties this together. Each cron invocation generates a UUID, persists it to a sync_runs row at start, and stamps every database mutation with that run_id. The token serves three purposes: traceability when a row looks wrong, replay safety if a run is rerun manually, and the natural key for the runs table that Section 08 builds on.
// lib/sync/run.ts
import { randomUUID } from 'node:crypto';
export async function runSync(module: 'leads' | 'deals') {
const runId = randomUUID();
const startedAt = new Date().toISOString();
// 1. Claim the sync lock (Section 07).
const claimed = await claimLock(module, runId);
if (!claimed) return { runId, skipped: 'lock-held' };
// 2. Insert the run row up-front so a crash leaves a record.
await supabase.from('sync_runs').insert({
id: runId, module, started_at: startedAt, status: 'running',
});
try {
const since = await getWatermark(module);
const rows = await fetchDelta(module, since);
const count = await upsert(module, rows, runId);
const newWatermark = rows.length
? rows[rows.length - 1].modified_time
: since;
await supabase.from('sync_runs').update({
ended_at: new Date().toISOString(),
status: 'ok',
records_processed: count,
watermark: newWatermark,
}).eq('id', runId);
await setWatermark(module, newWatermark);
return { runId, count };
} catch (err) {
await supabase.from('sync_runs').update({
ended_at: new Date().toISOString(),
status: 'failed',
error: (err as Error).message,
}).eq('id', runId);
throw err;
} finally {
await releaseLock(module, runId);
}
}"A sync agent is only as good as its worst Monday morning. The patterns in this section are the ones that decide which Monday you get."— Digital Applied CRM Automation team
07 — CronVercel cron, 15-minute cadence.
Vercel cron is the deployment layer that turns the sync agent into an unattended service. The configuration is two pieces: a vercel.json entry that defines the schedule, and an API route under /api/sync that the cron invokes. The route handler runs runSync() from the previous section, returns a small JSON response with the run id and the counts, and is protected by a shared secret so it cannot be hit externally.
// vercel.json
{
"crons": [
{ "path": "/api/sync?module=leads", "schedule": "*/15 * * * *" },
{ "path": "/api/sync?module=deals", "schedule": "*/15 * * * *" }
]
}
// app/api/sync/route.ts
import { NextResponse, type NextRequest } from 'next/server';
import { runSync } from '@/lib/sync/run';
export const maxDuration = 60;
export async function GET(req: NextRequest) {
const header = req.headers.get('authorization');
if (header !== `Bearer ${process.env.CRON_SECRET}`) {
return new NextResponse('forbidden', { status: 403 });
}
const module = req.nextUrl.searchParams.get('module');
if (module !== 'leads' && module !== 'deals') {
return new NextResponse('bad module', { status: 400 });
}
const result = await runSync(module);
return NextResponse.json(result);
}Vercel cron passes the project's CRON_SECRET automatically in the Authorization header when invoked from the platform side, which is how the route distinguishes a legitimate cron tick from a random request. maxDuration: 60 is a generous cap for a 15-minute cadence; if a run is consistently approaching that limit, the right move is to drop the cadence to 5 minutes and let each run handle a smaller delta, rather than push the function timeout higher.
sync_locks table is a single row per module with a claimed_by UUID and a claimed_at timestamp. Acquire with INSERT … ON CONFLICT (module) DO UPDATE SET claimed_by = EXCLUDED.claimed_by WHERE sync_locks.claimed_at < now() - interval '30 minutes' and release in the run's finally block. The 30-minute stale-lock window is your seatbelt against a crashed run that never released.Two operational notes on Vercel cron worth knowing. First, cron invocations run on the project's production deployment by default — they will not fire against preview deployments unless you explicitly enable that. Second, Vercel does not retry failed cron invocations; if a 15-minute tick fails, the next 15-minute tick picks up the work because the watermark did not advance. That is usually the right behaviour, but it means your alerting (Section 08) has to surface consecutive failures explicitly.
08 — ObservabilityStructured logs, failure alerts, and a run-history table.
Observability is what makes the difference between a pipeline that quietly drifts wrong for three weeks and one that pages you the morning it breaks. Three layers cover almost every operator need: structured logs with a stable run-id correlation, a sync_runs table you can query like any other table, and a Slack alert on consecutive failures.
JSON output, run-id correlation
Every log line tagged with run_id and module. Pino at info level for happy paths, warn for retries, error for failures with the stack. Vercel ingests JSON logs natively; pipe to Logtail or Datadog when the volume justifies it.
Default loggerOne row per invocation
Module, started_at, ended_at, status, records_processed, watermark, error. Index on (module, started_at DESC) so the dashboard query is instant. A 90-day partition policy keeps the table small.
Postgres tableTwo consecutive failures → page
A small worker queries sync_runs after every run and posts to Slack when the last two runs for any module have status='failed'. Single failures are normal — Zoho hiccups, network blips. Two in a row means the next run probably will too.
Incoming webhook-- migrations/0002_sync_runs.sql
CREATE TABLE sync_runs (
id UUID PRIMARY KEY,
module TEXT NOT NULL,
started_at TIMESTAMPTZ NOT NULL,
ended_at TIMESTAMPTZ,
status TEXT NOT NULL CHECK (status IN ('running', 'ok', 'failed')),
records_processed INT,
watermark TIMESTAMPTZ,
error TEXT
);
CREATE INDEX sync_runs_module_started_idx
ON sync_runs (module, started_at DESC);
CREATE TABLE sync_locks (
module TEXT PRIMARY KEY,
claimed_by UUID NOT NULL,
claimed_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE sync_watermarks (
module TEXT PRIMARY KEY,
watermark TIMESTAMPTZ NOT NULL
);Sentry is the right addition once you have more than one consumer of the mirror. Wrap runSync() with Sentry.captureException in the catch block, tag with the module and run id, and you get grouping and trends across runs without writing a custom dashboard. The Slack alert remains the pager; Sentry is for diagnosis.
A small operator dashboard pays back faster than most teams expect. The minimum useful view is two tiles per module: "runs in the last 24 hours" with a green/red status count, and "average duration" with a sparkline. Beyond that, a table of the last 20 runs with click-through to the error message handles 90% of operator interactions. Build it as a Supabase view or a small Next.js dashboard route protected by your existing auth — do not invest in a generic dashboarding tool until the agency reasons demand it.
A CRM mirror in Postgres is the lowest-cost path to AI-augmented analytics and reporting.
The pipeline this tutorial walks through is intentionally small — two modules, one direction, a fifteen-minute cadence — but every decision in it is the one we make on real client engagements. The COQL watermark with an overlap window, the snake_case mapping at ingestion, the upsert with a modified-time guard, the run-id token threaded through every mutation, the lock table that serialises cron invocations: each pattern is the one that fails the worst when replaced with something simpler.
What it unlocks is the part that matters. Once the mirror is live, analytics over CRM data costs near-zero — every Postgres feature is yours, every BI tool joins against it, every dashboard query measures in milliseconds instead of seconds. RAG over CRM history becomes a normal pgvector query. Custom dashboards stop being quarterly engineering projects and start being weekend features. AI scoring, lead enrichment, summary generation all become routine because the data is right there in the same database your application already reads.
Next steps once the Leads and Deals pipeline is steady: add a second pair of modules (Contacts and Accounts is the usual second wave), layer a webhook trigger on top for near-real-time updates on hot records, and start building the AI features the mirror was always there to enable. The pattern compounds — every additional module gets cheaper to add because the auth, locking, runs table, and alerting are already paid for.