Overview
Walmart’s last-mile delivery lifecycle is highly complex, involving 20 to 30+ microservices. Because each microservice maintains its own state of a given entity (such as a customer’s order), tracking the complete life cycle of an order as it moves between different systems was difficult. Without a centralized view, monitoring the overall state of an order, identifying gaps between systems, and pinpointing exact failure points required significant effort.
To address this, Walmart implemented Apache Pinot as a “one-stop” real-time platform to track the life cycle of an order across all involved microservices.
Solution & Architecture
- Data Ingestion: Walmart primarily ingests data from Kafka and Cosmos. For Cosmos, they track the change feed, pass the data through a transformation application, and then feed it directly into Pinot.
- Deployment & Scale: Pinot is deployed on Google Cloud Platform (GCP). The system currently manages over 50 tables, a number that is continuously increasing as more services onboard to Pinot to gain a better life cycle view of their entities.
- Analytics & Tooling: The data in Pinot powers multiple dashboards, alerts, and monitoring systems across the last-mile ecosystem. Walmart utilizes Apache Superset for dashboarding and Apache Airflow for alerting.
Business Impact
Implementing Apache Pinot has driven several major operational improvements for Walmart:
- 50% Reduction in Resolution Time: Pinot serves as a single source of truth for anomaly detection and monitoring. Engineering teams now know exactly which system is experiencing an issue, cutting resolution time in half.
- Automated Issue Resolution: By combining Pinot with Apache Airflow, Walmart has moved beyond mere alerting to implementing automated issue resolution. This drastically reduces the developer time spent on troubleshooting.
- Real-Time Business Metrics: The platform allows Walmart to create real-time business metrics, enabling fast analysis of system performance and improving the investigation process for next steps.
Future Roadmap
Walmart is actively expanding the capabilities and reach of its Apache Pinot deployment:
- AI Agents and Bots: Walmart is tuning and tracking AI agents that can utilize this data to answer queries like “where is my order?” and further automate the resolution of issues.
- Advanced Integrations: They are planning to integrate with ThirdEye to further enhance their anomaly detection capabilities.
- International Expansion: Originally limited to US markets, Walmart is in the process of introducing this Pinot-backed architecture to international markets, with Canada and Mexico already in place.
