Every Data Cloud project I’ve touched hits the same wall about six weeks in. The demo worked because someone fed it clean, pre-shaped data by hand. Then someone asks “can we pull in the real feed from the call centre platform?” and the confident answers get vague fast.
Ingestion is where Data Cloud projects quietly succeed or fail. Not the Calculated Insights, not the segmentation, not the flashy Agentforce demo sitting on top. The pipe. Get the pipe wrong and everything downstream inherits the mess, except now it’s harder to see because it’s been through a few layers of transformation first.
Three ways data actually gets in
Data Cloud gives you three real paths, and most orgs end up using at least two of them whether they planned to or not.
Batch, via connectors or the Bulk Ingestion API
This covers the bulk of what lands in Data Cloud: CRM objects through the native Salesforce connector, Marketing Cloud data extensions, files sitting in S3 or Google Cloud Storage, data from a warehouse like Snowflake or Redshift. It’s scheduled, it’s batched, and it’s forgiving if a run is a few minutes late. Good fit for anything that doesn’t need to react to an event the moment it happens: historical case data, product catalogues, loyalty tiers, order history.
Streaming, via the Ingestion API
This is the one teams reach for too early. Streaming ingestion exists for data that needs to show up in near real time, things like a web click, an app event, a call just logged in the contact centre. It’s genuinely useful when a Calculated Insight or a segment needs to react within seconds. It’s overkill, and more expensive to run and maintain, when the actual business need is “update it sometime today.” I’ve seen teams build a streaming pipeline for data that gets looked at once a week.
Native connectors for Salesforce-native sources
If the data already lives in Sales Cloud, Service Cloud, or Marketing Cloud Engagement, the native connectors do the heavy lifting without custom code. They’re the easiest path and usually the first one worth setting up, because they prove the rest of the model works before you go anywhere near a third-party source.
The mistake that shows up in almost every org: mapping without modelling
Data Cloud makes it very easy to wire up a source, map some fields, and watch rows land in a Data Lake Object. That’s the trap. Mapping fields is not the same as deciding what those fields mean in your data model.
I worked with a contact-centre team that ingested case data from two systems: Salesforce Service Cloud and a legacy ticketing tool that hadn’t been decommissioned yet. Both had a field called “status.” Both got mapped straight into Data Cloud. Nobody stopped to ask whether “Resolved” in one system meant the same thing as “Closed” in the other, or whether they should feed the same Calculated Insight at all. The segment built on top of that data looked fine in testing and was wrong in production, because it was quietly blending two different definitions of “resolved.”
Before you map a single field, write down what the Data Model Object is actually for, who’s allowed to write to it, and what “done” looks like for the data flowing into it. That document takes an afternoon. Skipping it costs weeks later, usually right after go-live when someone notices the numbers don’t add up.
Schema drift will happen. Plan for it before go-live, not after
Source systems change. A field gets renamed, a picklist value gets added, an API version gets deprecated. On a standard integration, that breaks loudly and you fix it in an afternoon. In Data Cloud, a schema change upstream can quietly degrade a mapping, a Calculated Insight, or an identity resolution rule without throwing an obvious error. The pipeline keeps running. It’s just running on slightly wrong assumptions.
Build monitoring around ingestion volume and null rates, not just success and failure flags. A batch job that “succeeds” but ingests zero rows, or ingests rows with a key field suddenly empty, needs the same attention as a failed job. Most teams only find out about a quiet schema break when someone downstream notices a number that looks off, and by then the bad data has already fed a few cycles of Calculated Insights.
Identity resolution depends on what you ingested, not what you intended to ingest
This one trips people up because it feels backwards. You don’t design identity resolution first and then feed it data. You design ingestion carefully, because identity resolution can only work with the identifiers you actually brought in. If the source system’s email field is inconsistently formatted, or a phone number comes through with inconsistent country codes, no amount of matching rule tuning in Data Cloud fixes that at the resolution stage. The fix belongs upstream, in how the data is cleaned on the way in.
A practical habit: before connecting any new source, list the identifiers it carries (email, phone, loyalty ID, device ID, whatever applies) and check their format against what’s already in Data Cloud. A five-minute check here saves a painful unification exercise three months in, when unified profiles turn out to be quietly duplicating because two sources wrote the same phone number two different ways.
A sequence that actually works
Start with one source, ideally a native connector, and get it fully modelled: DMO defined, field meanings documented, identifiers validated. Add monitoring on that single pipeline before adding a second source. Only then bring in the next source, and check its identifiers against the first before mapping a single field.
It’s slower than connecting everything in week one. It’s also the difference between a Data Cloud instance leadership trusts and one where every number needs a caveat attached. The second kind doesn’t get used for long, however good the dashboards look on day one.
Leave a Reply