CRM Export Pipeline

A B2B lead-data platform let its users export the companies and contacts they found straight into the tools their sales teams already use. This is the infrastructure behind that export: a serverless pipeline handling around 7,000 export jobs a day, each carrying up to 25,000 records, into Salesforce, HubSpot, Microsoft Dynamics, Pipedrive, Salesloft and Outreach.

Exporting a few hundred contacts into a CRM is simple. Doing it thousands of times a day, for lists of up to 25,000 records, into six destinations that each model data differently, is not. Every CRM has its own objects, fields and API limits, one bad record must not sink the rest of a job, and users need to know exactly what happened to each one.

Export requests go onto an Amazon SQS queue, and AWS Step Functions orchestrate each job through Lambda functions. A 25,000-record export and a small one take the same path, with no servers to size for the peak, and the processing framework is built for low latency.

Each CRM and sales engagement platform has its own objects, fields and rules, so every destination has its own field mapping. Rule-based logic decides, record by record, whether to create or update, and deduplicates so the same company or contact is not written twice.

Exports draw on a credit system that charges users only for net-new data, so a record they already have costs nothing to send again. Every export produces a structured CSV report that lists each record as succeeded, failed or skipped. A history page keeps every export with its downloadable report, role-based access control governs who can export what, and the data is fully audited.