Navigating Kafka: Challenges, solutions, and the future of real-time data streaming
In this Data (R)evolution episode, Matan Mizrahi and Filip Yonov join us to discuss how Kafka effectively handles and optimizes data streams, the pursuit of standardized APIs, and the vital role of AI in optimizing complex systems. They also share how AI and open-source tools are transforming the landscape. Tune in to hear about the evolution of data streaming and the transformative impact of AI and cloud-native solutions.
Key takeaways:
- Emphasize the need for customer-defined scaling logic based on metrics and auto-scaling events for efficient Kafka management.
- The importance of integrating AI, like the Ivan AI database optimizer, to enhance workload optimization and complex system management in Kafka.
- The significance of strategic partner relationships in providing advanced software solutions and necessary knowledge and experience to mitigate risk in critical data infrastructure projects.
Podcast series: Data (R)Evolution
Topics around data, innovation and economics in a rapidly-changing cloud world.
Data drives innovation and revolutionizes the tech world. Data (R)evolution, powered by Aiven, explores the intersections of data, AI, innovation, and economics through the ever-evolving landscape of cloud technology. Led by hosts Francesco Tisiot and Janki Patel Westenberg, navigate this digital revolution in conversation with experts at the forefront of change.
Watch the series
Hosts
Francesco Tisiot
Field CTO & Head of DevEx, Aiven
Francesco comes from Verona, Italy and works as a Field CTO & AI Lead at Aiven. With his many years of experience as a data engineer, he has stories to tell and advice for data-wranglers everywhere. Francesco loves sharing knowledge with others as a speaker and writer, and is on a mission to defend the world from bad Italian food!
Janki Patel Westenberg
AI, Platform, Open Source Product Marketing Leader, Aiven
Over the last 10 years, Janki has successfully led technical partnerships collaborating with Google Cloud and AWS to build database and caching solutions for gaming and media. Additionally, she has spearheaded GTM strategy partnered with AWS' product leadership for 19 AWS Analytics and Database services for multiple industries. Janki also is a proud alumna from University of Wisconsin-Madison (Go Badgers!)...fun fact, she studied Genetics and did research before moving to the cloud industry!
Guests
Filip Yonov
Director of Product, Data Streaming Services
Filip has many years of experience in consulting, IT project management, and product development. His previous positions include being responsible for the data streaming platform at ING with Apache Kafka and its ecosystems of 40 ING markets. He has experience as a senior product owner at REWE Digital and as a product owner at IBM for Docker Containers.
Matan Mizrahi
Senior DevOps Engineer, Claroty
Matan is a talented Senior DevOps Engineer with extensive experience in cloud architecture and infrastructure. He enjoys experimenting with new technologies and introducing innovative solutions, always keeping dynamic demands and product needs in mind. Matan is passionate about enhancing the collaboration between R&D and DevOps, effectively bridging the gap between infrastructure and application development.