The Challenge: Keeping Account Data in Sync
At Realtor.com, our enterprise systems team manages the critical integration between Salesforce and downstream internal systems. Over the years, a consistent challenge has been ensuring that account updates made in Salesforce are reliably reflected in our internal systems and in real time.
Traditional approaches such as scheduled Extract, Transform, Load (ETL) jobs, periodic polling of Salesforce APIs, or middleware-based sync mechanisms were either too slow, brittle, or operationally heavy. They introduced unnecessary complexity, often failed silently, and couldn’t scale well with growth in data volume.
We wanted a solution that was:
- Real-time, so updates are processed instantly
- Decoupled, to reduce dependency between systems
- Observable, with robust monitoring and error handling
- Scalable, with low operational overhead
The Old Approach
Our earlier strategies included:
- Polling Salesforce APIs: Custom-built scripts hit the Salesforce REST API on a schedule to check for changes. This risked hitting rate limits and introduced unnecessary latency.
- Scheduled ETLs: Batch jobs ran at fixed intervals to extract and sync updates. These introduced lags and required complex deduplication logic.
- Third-Party Sync Tools: We evaluated Amazon AppFlow and MuleSoft, which supported Salesforce CDC and Platform Events. However, these tools had limitations around throughput, customization, and observability.
What We Needed
The right solution would allow Salesforce to push account updates as events directly to AWS services, so our internal consumers could react to them asynchronously. We wanted to:
- Move away from polling
- Get updates in near-real-time
- Minimize custom integration code
- Scale seamlessly with traffic
The Solution: Salesforce Platform Events + AWS EventBridge
We built a fully event-driven architecture using Salesforce Platform Events, Amazon EventBridge, and serverless AWS compute.
Step 1: Emit Platform Events from Salesforce
We configured Salesforce to publish custom platform events whenever an account was created or updated. These events contain the relevant account metadata and an event ID for traceability.
Step 2: Use Salesforce Event Bus Relay
We enabled Salesforce Event Bus Relay, which pushes platform events directly into a partner event bus in Amazon EventBridge. This eliminated the need for intermediate polling mechanisms.
Step 3: Route Events Using EventBridge
Once in AWS, we used EventBridge Pipes to route platform events to downstream consumers. Pipes allowed us to filter and enrich events before sending them to targets such as Amazon SQS, AWS Lambda, or Kinesis Data Streams.
Step 4: Process Events with Lambda
Lambda functions subscribed to these queues handle the processing of each account update. These functions:
- Transform and validate the data
- Update internal databases and systems
- Log processing results to CloudWatch for observability
This pipeline is fully asynchronous, scalable, and fault-tolerant.
Architecture Diagram
Operational Benefits
The new event-driven architecture delivers significant operational benefits, improving performance, reliability, and maintainability across Realtor.com’s systems.
- Near-real-time sync: Changes in Salesforce are processed within seconds.
- Decoupling: No more tight coupling or custom polling logic.
- Observability: CloudWatch logs, DLQs, and metrics provide insight into failures or delays.
- Resilience: If a consumer fails, the event is retried automatically or routed to a dead-letter queue.
- Low maintenance: No cron jobs, retry loops, or middleware to manage.
These improvements create a more scalable, efficient, and resilient integration framework with far less operational overhead.
Lessons Learned
Implementing the new architecture surfaced key lessons that helped strengthen system design and reliability:
- Schema versioning is key: We added version fields in our Platform Events to support safe evolution.
- Idempotency matters: We ensured all downstream consumers could safely handle repeated events.
- AWS limits need monitoring: Both EventBridge and Lambda have quota limits that should be tracked.
These takeaways are essential for building scalable, fault-tolerant event-driven systems and avoiding common pitfalls.
Final Thoughts
By combining Salesforce Platform Events and AWS EventBridge, we transformed how account updates flow across our systems. We now operate a real-time, event-driven integration that is scalable, resilient, and easy to maintain. This shift has significantly improved data consistency across our platforms and set the stage for future expansion into more Salesforce-driven event pipelines.
This event-based pattern is now a foundational building block for many systems at Realtor.com, and we’re excited to build on it further in the future.
Feel free to contact me on LinkedIn with any comments or questions. I’m always eager to talk with others in the industry!
Are you ready to reach new heights at the forefront of digital real estate? Join us as we build a way home for everyone.
