ThirdEye 101: An Introductory Course on StarTree’s Real-Time Anomaly Detection Platform
Companies like financial services plus rideshare and delivery services depend on up-to-the-minute data that is accurate and consistent. Fluctuations and anomalies in your key metrics can critically impact customers and business performance, such as when a driver faces unexpected delays that affect pickup or delivery times.
StarTree ThirdEye is an anomaly detection application that runs on top of StarTree Cloud, and watches for deviations in your mission-critical business operations. It allows you to spot real-time fluctuations in your key metrics, perform root-cause analysis, and respond immediately.
We recently launched a series of videos to walk curious minds through the basics of ThirdEye. ThirdEye 101 is your quick guide to understanding the features of ThirdEye and the value of real-time anomaly detection with Apache Pinot.
Why do data teams use StarTree ThirdEye?
StarTree ThirdEye watches to ensure your real-world data conforms with your expectations for performance, providing enterprise visibility through customized dashboards. Whenever there is a deviation from expected behavior, stakeholders are immediately alerted so they can take appropriate action.
Data teams benefit from ThirdEye’s comprehensive feature set, which includes:
- Easy Data Onboarding – Onboard metrics/KPIs directly from Apache Pinot
- Simple Interface – Point-and-click interactive GUI
- Programmatic Access – Read and write APIs using Swagger
- Multiple Algorithms – Various statistical methods for sophisticated anomaly detection
- Dimension Exploration – For real-time monitoring across multiple dimensions
- Dimension Recommender – Use past data to make intelligent projections and project trends
- Improved Accuracy – Use anomaly filters during alert creation to reduce false alarms
- Root Cause Analysis – Discover via heatmaps, events, and top contributors
Check out our documentation to learn more about the features and how to get started with ThirdEye.
What does the ThirdEye 101 course cover?
Join Barkha Herman, Developer Advocate at StarTree, as she covers everything from the basics of real-time anomaly detection to advanced topics like root cause analysis and anomaly detection algorithms. These introductory courses are all under 5 minutes, and you’ll walk away with the skills to create alerts and effectively respond to anomalies.
In the ThirdEye 101 course, you’ll learn about:
- Architecture
- Entities
- Worker management
- Creating alerts
- Built-in detection algorithms
- Notifications
- Root cause analysis
We recommend starting your journey by checking out this quick introduction to ThirdEye:
Architecture
Barkha will walk you through the core architecture of ThirdEye, as well as cover entities (data entities, alert entities, and notification entities) and worker management.
Architecture
Explore the sophisticated architecture of StarTree ThirdEye, which is structured around four essential service categories: a user-friendly UI for setting up alerts and conducting root cause analysis, the Coordinator for overseeing API interactions and CRUD operations, the Scheduler for orchestrating task timing, and the Worker for task execution. Discover how ThirdEye’s intuitive no-code/low-code interface, along with its pre-configured templates and algorithms, streamlines the anomaly detection and notification process, making complex data monitoring straightforward and efficient.
Entities
Delve into the core components of ThirdEye entities within the StarTree ThirdEye platform, encompassing data entities for monitoring anomalies, alert entities for defining anomaly parameters, and notification entities for alerting relevant parties. Learn how these entities, from data sources and datasets to alert templates and detection pipelines, integrate to effectively identify and communicate anomalies, utilizing tools like Apache Pinot and customizable alert systems for comprehensive anomaly detection and management.
Worker Management
Discover how ThirdEye achieves real-time anomaly detection through efficient worker management, scaling the number of workers to distribute workload evenly. This overview highlights the worker lifecycle, from task assignment to completion, and the role of the worker manager in ensuring optimal performance and scalability. Learn how ThirdEye maintains high efficiency and reliability in real-time data monitoring by dynamically adjusting worker numbers based on workload, ensuring each task is uniquely processed and efficiently managed.
Functionality
Next, you’ll learn about how key features of ThirdEye work. Barkha will show you how to create alerts and notifications, how the built-in detection algorithms work, and how to perform root cause analysis.
Creating Alerts
Explore the alert creation capabilities within StarTree ThirdEye, where you can craft basic or multi-dimensional alerts tailored to your specific needs. Whether monitoring sales fluctuations at a single location or delving into intricate multi-dimensional analyses involving time, product types, and external factors like weather, ThirdEye equips you with the tools for precise monitoring. Learn how the Dimensions Recommender utilizes statistical analysis to highlight impactful dimensions, while customizable filters and sensitivity settings help fine-tune alerts, reducing false positives and ensuring relevance to your specific scenarios.
Built-In Detection Algorithms
Dive into the variety of built-in detection algorithms offered by StarTree ThirdEye, enabling immediate alert creation for a wide range of scenarios. From the simple threshold algorithm for stable signals to the sophisticated matrix profile for time series analysis, each algorithm is designed to cater to specific data patterns, including seasonality, trends, and noise. Understand the strengths and limitations of each, such as the threshold’s simplicity versus its inability to handle seasonality, and the exponential smoothing’s adeptness at trend and seasonality management at the cost of sensitivity to data drift. This overview provides a foundation for selecting the most appropriate algorithm for your data monitoring needs.
Notifications
Explore the versatile notification system within StarTree ThirdEye, offering multiple methods to alert you to anomalies, from email and Slack notifications to webhook calls for custom app integrations. Learn how to use subscription groups to tailor who receives notifications, which alerts trigger these notifications, and the frequency of these alerts, all configurable through an intuitive UI or cron syntax. Whether for broad alerts or specific dimensional triggers, ThirdEye ensures you’re promptly informed about critical data events, enhancing your ability to respond swiftly.
Root Cause Analysis
Delve into the advanced root cause analysis capabilities of StarTree ThirdEye, designed to pinpoint and understand the underlying factors of data anomalies. Through an intuitive UI, ThirdEye offers tools like heatmaps for visual deviation representation, Dimension Exploration for detailed metric correlations, Top Contributors for identifying key anomaly factors, and Events for contextualizing anomalies with external factors. These features facilitate a comprehensive analysis, allowing you to distinguish between genuine anomalies and false positives, share insights with your team, and enhance collective understanding and response to data irregularities.
View the complete course
Catch up on the full course by visiting our ThirdEye 101 playlist on YouTube. We created the videos with your busy schedule in mind, so it should only take you about 30 minutes to watch all 10 videos!
Learn more about ThirdEye and real-time anomaly detection
Hear how DoorDash is using StarTree ThirdEye for anomaly detection at Real-Time Analytics Summit 2024, which will be hosted in San Jose, California, May 7-9 2024. DoorDash and StarTree will co-present a session on Revolutionizing API Partnerships: Real-Time Anomaly Detection at DoorDash. Visit the Summit website to learn more about the talk and register for the event.
REGISTER NOW FOR RTA SUMMIT 2024
You can also check out these great resources for more information on ThirdEye and the latest news in the community:
Interested in fully managed Apache Pinot? You can start querying in minutes with our free trial of StarTree Cloud.