Summary
- Amberdata reduced latencies to sub-second, even millisecond-scale ranges with StarTree Cloud
- Amberdata now averages over 350,000 insert events per second, peaking at nearly 1 million inserts per second
- Amberdata now sustains far higher levels of query concurrencies, meeting SLAs and successfully serving more customers
- Amberdata saved 66% on infrastructure costs moving to StarTree Cloud
Amberdata is a financial services company that bills itself as “your lens into the entire crypto-economy.” It provides a platform for digital assets research, market intelligence, portfolio management, and application development with the rich features of its API and data dictionary. It provides support for all aspects of the business, from front office (research and trading) to mid-office (risk and treasury) to back office (tax, accounting, and compliance) processes.
The digital economy matures, and Amberdata is at the forefront of that maturation. Amberdata caters to traditional financial institutions, helping them manage risk and navigate the regulatory and compliance space as they seek to diversify and invest in digital assets. For example, GSR, a leader in digital asset management, integrated Amberdata to empower its systemic signal research and enhance its competitive edge. As GSR’s Simran Singh observed, “From the ease of use… to the quality of the data that existed, to the history and the broader richness of the datasets. No other market data provider in crypto truly has all three.”
Advantages of StarTree Cloud
In adopting StarTree Cloud, powered by Apache Pinot, Amberdata was able to find advantages in four areas:
- Improved query performance — StarTree reduced query latencies to sub-second times — even to as little as millisecond-scale ranges.
- Meeting SLAs — Amberdata served more customers faster, supporting the increased query concurrencies of their growing user base.
- Cost savings — By moving to StarTree and utilizing tiered storage, Amberdata saw its infrastructure costs shrink by 66%.
Handling real-time upserts
With 2 billion rows of data, each representing an asset whose price can change at any time, Amberdata needed to support scalable real-time upserts. An upsert is two operations in one: if a doesn’t already exist (if this is a new asset), the data is inserted into StarTree Cloud. However, if the record already exists, it simply updates the existing record.
Amberdata processes nearly 1 million insert events at peak workloads per second; average workloads exceed 350,000 events per second. While upserts are a small proportion of Amberdata’s overall workload, it is a vital capability to handle them properly for scalable non-stop operations. Amberdata uses StarTree Cloud’s infrastructural advantages which go beyond open source Apache Pinot to be able to handle upserts efficiently and performantly.
Productivity as the ultimate measure of success
StarTree Cloud is a fully managed service. Amberdata’s developers can focus on writing their applications rather than administering a real-time analytical data platform. Amberdata CTO and co-founder Joanes Espanol says, “The best measure of success is where do our engineers spend their time? Rather than spending time managing infrastructure, chasing SLAs, they can now focus on building and delivering insights for our customers.”
StarTree Cloud is not just a platform; it’s also the team of experts behind it. The professional collaboration between StarTree and Amberdata is deep and strong. As Joanes explains, “It almost feels at times like the StarTree team is just an extension of the Amberdata team, which has been a great experience.”
“The benefits of migrating to StarTree and Pinot center around three areas: query performance, SLAs, and cost. With Pinot, we are now in the subseconds to milliseconds range, and the higher query concurrency means we can serve more customers faster. With the switch to Pinot, we’ve been able to reduce our infrastructure costs and reduce our dependencies on older technologies.”
CTO and Co-Founder
Build with StarTree on AWS
Amberdata deployed on AWS as its cloud provider. StarTree is an AWS ISV Accelerate partner. Discover how to build your business on AWS, provisioning StarTree Cloud directly from the AWS Marketplace.