Conrad Electronic Expands E‑Commerce Platform with Aiven

German-based Conrad Electronic provides an advanced sourcing platform serving 21 million customers worldwide. Its mission is to become the leading European B2B sourcing platform for technical supplies and its e-commerce database is at the heart of its business. With its online platform fast expanding and the associated data exploding, the retailer needed its data infrastructure to be faster, more robust and scalable. Conrad selected Aiven to manage open source data technologies PostgreSQL®, Apache Kafka®, OpenSearch® and M3 on Google Cloud. The e-commerce platform now benefits from real-time data streaming and a highly stable database handling 11.2 million requests an hour, ensuring its customers enjoy a fast, superior experience. With Aiven managing the cloud data infrastructure, Conrad’s developers can focus on what they do best — delivering even more customer value.

Advanced B2B sourcing platform selling 9 million products from 6,000 brands on conrad.de

Since 1923, Conrad Electronic have been a reliable partner when it comes to technology and Electronics. Today, as a sourcing platform, the company provides all parts for a successful sourcing of technical supplies. At Conrad, business customers get exactly what makes their project or business a success: a wide and deep product range comprising nine million products to choose from, customer-centric solutions and services, and face-to-face expert advice. The company has successfully evolved from a traditional B2C retailer to an international B2B sourcing platform. It sells more than 9 million products from 6,000 brands on conrad.de ranging from computing equipment and tools to batteries and automation products. The Conrad group achieves more than €1 billion annual turnover and has its sights set on becoming the leading European B2B sourcing platform for technical supplies.

PostgreSQL upgrade risks e-commerce marketplace interruption

Conrad’s e-commerce platform provides its 21 million customers worldwide with a quick and convenient online shopping experience. They can compare products from all the brands and buy everything they need in one place.

Conrad’s database is central to its e-commerce operations, storing critical data like users, wish lists, orders, SKUs and product prices. “Our database is the heart of our e-commerce platform. Without it, nothing else works,” says Janek Wonner, Head of SRE and Cloud Technology, Conrad Electronic.

Until 2020, Conrad used self-managed PostgreSQL running on Google Cloud for its e-commerce database. With its platform expanding, the retailer wanted to upgrade to a faster, more robust version. However, due to the limited resources, there were concerns about the practicality of a successful database upgrade, especially considering the specialized expertise required for upgrading the PostgreSQL system.

“We can’t risk any disruption to the database — no downtime is allowed. If the e-commerce platform is offline then we can’t process any orders. Every minute costs money,” says Wonner.

Wonner considered, but quickly discounted, the option of continuing with self-managed PostgreSQL. The upgrade and subsequent management would require a huge amount of resources and skills Conrad simply didn’t have at the time.

Seamless transition to Aiven platform on Google Cloud

Wonner evaluated PostgreSQL managed services from five different vendors against a wide range of criteria and Aiven came out top. “There were many reasons for choosing Aiven but if I had to pick the top three, I’d say great people, ease-of-use and fair, transparent pricing,” says Wonner. “We found the whole Aiven set-up and the team really easy to work with. We contacted lots of vendors for Proof of Concepts and Aiven was by far the easiest and fastest to get started with.”

The transition to Aiven and subsequent PostgreSQL upgrade went seamlessly. This was helped by the integration of Aiven services with Google Cloud. “It’s great we can get open source data services like PostgreSQL from Aiven already set-up on Google Cloud. It saves us time and effort,” says Wonner. His team ran several trials and load tests and then Aiven ran compatibility checks to determine necessary adjustments before the upgrade. Conrad then imported its whole database into an Aiven instance, and performed several tests which showed the new database to be much faster. It was put into production and quickly embraced by Wonner’s team.

“Aiven has an elegant, simple dashboard which our developers find easy to understand and use. We can spin up new clusters with a couple of clicks. Aiven makes PostgreSQL really easy for us,” says Wonner.

At that time, the Conrad business was still expanding and the company started facing other data infrastructure challenges. As the number of marketplace sellers and products grew, the amount of product data increased exponentially. “We needed an architecture to better support the exploding volume of data that was moving between all our applications. We liked the look of Apache Kafka for real-time data streaming but knew it was tricky to manage,” says Alexander Kaiser, Head of Platform Architecture.

Given the success of Aiven for PostgreSQL, the team decided to try out Aiven for Apache Kafka on Google Cloud. Again, the transition went smoothly. “We were worried we’d have to do something special to get it working with Aiven, but that wasn’t the case,” says Diego Irismar da Costa, Platform Architect, Conrad Electronic. “We like that it’s the open-source version of Apache Kafka, not a customized version with vendor lock-in. If we research online how to use Kafka, it applies to Aiven for Apache Kafka. That makes it really simple for us.”

E-commerce database handles 11.2 million requests an hour while delivering 99.99% uptime and latency below 1 millisecond

Since upgrading to fully managed Aiven for PostgreSQL, Conrad is benefiting from a high-performance, fast and reliable e-commerce database. It handles a huge volume of traffic — 11.2 million requests an hour — and can easily scale to accommodate peaks in demand. Wonner’s team no longer worries about maintenance and upgrades as this is managed by Aiven with an uptime SLA of 99.99%.

Aiven’s PostgreSQL integrates with key e-commerce applications. This includes Conrad’s e-procurement solutions designed to build customer loyalty by simplifying the sourcing process for B2B customers. For example, the database communicates with Conrad’s Open Catalog Interface (OCI). This allows customers to integrate the Conrad marketplace with their SAP procurement system, enabling them to order products and take advantage of tailored offers and discounts more conveniently.

Another benefit is that Conrad can use the Google Cloud VPC network peering with Aiven which achieves latency below one millisecond. “We have a huge amount of database calls going on so the incredibly low level of latency enabled by the Google Cloud VPC network peering is important to us. It keeps the e-commerce platform operating at speed resulting in a better experience for the customer,” says Wonner.

Continuous, real-time data streaming between applications

By using Aiven for Apache Kafka, Conrad can scale to accommodate the increase in data moving at speed around its e-commerce platform, and key integrations have been straight-forward. For example, Conrad now uses Apache Kafka to stream data from its Product Information Management (PIM) system to its product service system in real-time which delivers product information to customers via an API.

“Aiven for Apache Kafka is now our number one technology for managing information flow between the many different systems underpinning our e-commerce marketplace,” says Kaiser. “It’s fast, reliable and scalable.”

Conrad has also started using Aiven for OpenSearch and Aiven for M3. These services are primarily used to gather logs and metrics about Apache Kafka and PostgreSQL, which it then visualizes with Grafana.

“Having this combination of managed services from Aiven brings great value. We’ve been able to easily add new services without the hassle of onboarding a new vendor. We can operate everything from one place. The costs are transparent and it's easy for us to see the price of using other services as we plan ahead,” says Wonner.

Achieving great things together — today and tomorrow

Conrad now has a highly stable and scalable data infrastructure underpinning its e-commerce platform which ensures that, during the 200 million visits made each year on its platforms, customers always enjoy a fast, reliable experience. With the day-to-day management handled by Aiven, Conrad’s engineers and developers can focus on making the customer experience even better.

“We have a small team looking after PostgreSQL, Kafka and MongoDB. If it were all self-managed, we’d need to double our team. Aiven enables our team to focus on what brings value to our customers. With a limited number of people, we’re achieving great things. As the company keeps growing, Aiven will remain a solid partner,” says Kaiser.