Why Kubeflow in your Infrastructure?

CEOs and CTOs are being challenged by customers, analysts and investors to define how Artificial Intelligence and Machine Learning (ML) will impact their revenues and costs. The leading research and development organizations are quickly migrating to open source machine learning frameworks, especially those that take advantage of the operational and infrastructure efficiencies provided by containers, micro-services and Kubernetes. This trend is demonstrated in a recent 451 Research survey which found that over 70% of enterprise organizations surveyed are using Kubernetes. GitHub has over 95M projects, and Kubernetes and TensorFlow are frequently in the top 10 projects, in terms of contributors, discussions, forks, and reviews.

With an ever-increasing availability of data and compute power, machine learning is turning out to be a powerful tool to solve various problems and helping achieve state of the art results. In such interesting times, Kubeflow has grown very quickly to be one of the most promising ML toolkits in the cloud native open source world.

We at Cliqz are also solving some of the most complex problems around user privacy and web search using self-managed Kubernetes (kops) on AWS. Since January 2017, we started our cloud native journey and have been building Web Search solutions using Kubernetes. Since December 2017, the Search-Recency system has been in production, helping us towards near-real time index updates leading to most recent and up-to-date search results.

To solve this problem at that scale, we heavily use Machine Learning, Natural Language Processing, Deep Learning and core Information Retrieval techniques which led us to explore Kubeflow. We are currently evaluating Kubeflow as a general alternative to our custom ML workflow.

Learn more about our initial assessments and how Kubeflow might work well for one’s Kubernetes infrastructure by reading a guest post by Ankit Bahuguna & Faheem Nadeem on Medium.