About AdGear
Founded in 2008, AdGear has been at the forefront of digital advertising technology for quite some time. The Canadian company’s online ad serving platform compiles vast amounts of user data, which it uses to bid on the ad placements that online advertising exchanges auction off in real time.
In the ad serving business, time is of the essence. As David Haguenauer, Software Engineer at AdGear, explains, “We have to be able to decide quickly and communicate with the exchange if we want to participate in a given bid. Auctions typically complete within 100 milliseconds.”
Challenges with Latency
AdGear had been using Apache Cassandra as the back-end for its bidding gateway service, which performs about 1,000,000 bids per second. They had 31 Cassandra nodes across 3 data centers serving regional traffic for the western United States, eastern United States and western European Community.
From an advertising database perspective, the most critical factor is read latency. The cluster must sustain traffic while keeping a read latency consistently in the low milliseconds. Haguenauer explains that “we have only 10 – 15 milliseconds to wait for the database to retrieve user profiles. If we have to wait longer, it’s too late — we’ve already lost the opportunity to bid.”
AdGear liked Cassandra but they were experiencing too many read latency spikes, either due to higher loads or for reasons unknown. “We hit a wall with Cassandra. We weren’t able to tune it to behave in a more reliable way,” said Haguenauer. “If it was too slow that meant lost business to us.”
AdGear Transitions to ScyllaDB for Higher Performance
“ScyllaDB was the natural choice for us since it’s fully compatible with Cassandra,” said Haguenauer. “We realized we could just put it where Apache Cassandra was and everything would keep working. We started our transition to ScyllaDB without rewriting any of our stack.’
“We realized we could just put it where Cassandra was and everything would keep working.”
– David Haguenauer, Software Engineer, AdGear
Since this smooth transition, AdGear has seen significant performance improvements.
With ScyllaDB, AdGear is able to serve the same amount of requests with about half the hardware. Each ScyllaDB node serves a bit more than twice as many queries as each Cassandra node did. At peak traffic the effective read latency fell from 21 milliseconds on 31 Cassandra nodes to 13 ms on 13 ScyllaDB nodes. More importantly, ScyllaDB latencies are much more stable over the course of a day.
However, the measurement that hits closest to home for AdGear is time-outs, when a database read is so slow that it must be discarded. At AdGear, a time-out means they failed to participate in a bid. “With Cassandra we were unable to respond to 1 in every 7 bid requests. With ScyllaDB in place we’re able to bid 95% of the time — and when we’re not able to bid it’s not because of the database.”
As an added bonus, moving to ScyllaDB also improved the bulk insert process for AdGear, who are now able to load user data directly into the database without a time-consuming intermediate step and without causing spikes in read latency. This has also enabled them to simplify their client-side code.
“ScyllaDB was perfect for our use case,” said Haguenauer. “We save money on hardware and we get much lower and more predictable read latencies.”
