Lessons from Success: How Smart Businesses Build their Foundations for Innovation

The pursuit of innovation is paramount for any business that wants to stay ahead. But it’s a quest that has its fair share of hurdles. At a panel session I hosted at Aiven Live London, experts from Revenir, Dojo and Hookdeck shared their insights into some of those hurdles and how their organizations are getting over them.

The need to optimize costs while maximizing business value quickly emerged as a universal challenge for all the panelists — which is perhaps not surprising in a tight economic climate! All three of our experts pointed to the value of using managed services, when done properly, to help with cost control.

Brian Wagner, CTO at Revenir, a London-based fintech, said: “As a start-up, we are always focused on the cost for us to process a single transaction. When I consider managing Apache Kafka® in-house versus having Aiven running it as a service, it’s just a fraction of the cost. It’s a no brainer.”

A related issue is resource optimization, and all our panelists highlighted that the use of Aiven services helps them to focus internal resources on their core value proposition. When the focus is innovation and delivering on what customers need, then directing internal talent to infrastructure management is often a wasted opportunity.

Revenir’s Wagner put it very succinctly, when he said: “If it’s not part of our IP, we ideally want to buy it in.” And Phil Leggeter, Head of DevRel and PLG at Hookdeck, a fast-growing webhook infrastructure company, said: “We’re a small team. We have to be innovative and think about what new products and features we can deploy to deliver more value to our customers.”

He continued: “We need the ability to try different data solutions, to swap and change. If we were managing the data infrastructure ourselves, we wouldn’t have the time to do that, and it would take us longer to deliver value to our customers.”

Another fintech, Dojo provides card-payment solutions for small and medium-sized businesses, which also relies on Aiven to manage its Kafka clusters. Sandeep Mehta, its Engineering Lead, Data Platforms reiterated the point. He said: “Our engineers should not have to worry about the infrastructure.”

Mehta also raised another important benefit associated with using Aiven services, this time related to the explosive growth that Dojo has experienced. Mehta has to build high levels of scalability into his data platform to respond to changing demands on the system. “Scaling up is important but so is scaling down,” he said. “We’re now looking at FinOps and putting the foundations in place.”

Incidentally, FinOps is an “Evolving cloud financial management discipline and cultural practice that enables organizations to get maximum business value by helping engineering, finance, technology and business teams to collaborate on data-driven spending decisions.” That description comes from the FinOps Foundation Technical Advisory Council – and is an example of exactly the kind of approach that Aiven’s services are intended to enable.

Back to London Live, where conversation at our panel session turned to the importance of enabling engineers to work independently and quickly, while making sure that the correct governance and control remain in place — and its role in fostering innovation.

For Dojo, the use of self-service tools is key. As Mehta said: “My team, the data platform team, is centralized. We provide generic infrastructure, generic tooling, generic governance and we have plans to build a discovery platform where our engineers can come, click a button and get whatever they need.”

For Revenir, the focus is not so much on self-service for engineers, but on freeing up their time in different ways. It does this by using Aiven to take care of the important but “undifferentiated heavy lifting” associated with data infrastructure management.

Wagner said: “Our product, our ability to grow the team and innovation are all underpinned by technology. The conversation always starts by asking ourselves: ‘How are we going to elevate the product? What features are we going to release? And how quickly can we get this to market?’ All of those things are enabled by undifferentiated heavy lifting.”

Managing data infrastructures across multiple cloud providers is another challenge to innovation that is getting attention. Hookdeck and Revenir currently use one public cloud provider but both Leggeter and Wagner acknowledge this could change in the future for a number of reasons that range from satisfying compliance and customer requirements to cost and performance optimization.

Doja, on the other hand, already uses GCP, AWS and Oracle Cloud, which Mehta acknowledges can add more complexity to an already complex environment. “But if we had an outage and people couldn’t make payments, we’d lose business straight away,” he said.

“We need an architecture that is scalable, compliant and secure — and which cannot fail. A challenge for us is having lots of clouds, with services running everywhere, and bringing in a centralized streaming architecture that consumes the data from everywhere, and making sure it’s all secure, won’t fail, will scale and can be monitored.”

All our guests reminded us that establishing the foundations for innovation in a data-driven world is not necessarily straightforward. But they also reminded us how necessary it is — and the rewards that are on offer. They represent businesses who all engage with open source solutions, real-time data platforms and cloud-enabled scalability to support innovation, and their success provides an inspiring example of what can be achieved.