Lessons learned from successful MLOps implementation — DevOps meets Machine Learning MLOps has been a hot topic in 2021, with many people talking about and companies aiming at implementing it. The reason is apparent: MLOps enables the application of agile principles to machine learning projects, which means shorter release cycles and higher quality standards. From the technology standpoint, I would say…

How to set up containers for MLOps pipelines — This blog post is about a challenge that my team recently faced when working on gcloud beta ai and docker dependent tasks in the scope of an MLOps project. Such tasks consist of developing lightweight integration tests for custom training applications. We use the Google Cloud Vertex AI unified ML…

And leverage Durable Functions to reduce the burden when managing long-living serverless workloads — Microservices architectures have been widely used by enterprise organizations to split their applications into smaller parts. Experience shows that, along with benefits, microservices also bring challenges for software development teams. Integrating related parts is among the most common issues and there’s no silver bullet to tackle the problem. Microservices Choreography…

A sequel to “WebSockets and AsyncIO: Beyond 5-line Samples” — In this sequel to a previous article, I discuss design, libraries, and quality assurance in the scope of WebSockets, asynchronous programming, and concurrency with Python. In the first article, we took a look at the WebSockets technology and how it fits Python. I’ve also proposed a fictitious use case to…

Thoughts on async and concurrent programming with Python — My team was recently assigned a task that comprises writing a Python client to interact with a specific WebSockets API. It’s led us to learn new things and the goal of this article is to share our progress. We believe there’s always room for improvement, so please feel free to…

Leverage Data Catalog to discover & annotate Qlik Sense assets — In this blog post, I’m going to share thoughts and knowledge that drove the development of a new sample connector for Google Data Catalog: the Qlik Sense connector. This is the third open-source connector that allows customers to ingest metadata from BI Systems into Data Catalog: the ones for Looker…

A custom model — More and more companies migrate their data workloads to the cloud every day. At the same time, new data protection regulation programs become effective around the globe. Such simultaneous events lead companies to enforce their Data Governance standards intending to avoid legal charges due to inappropriate data management. Data Governance…

Hints and tips to create your own driver from scratch — GitLab is rich and scalable when it comes to software architecture, and Runner is one of the key components of such an ecosystem. It is responsible for running CI/CD jobs and deployed decoupled from the GitLab instance. Runners come in multiple flavors: Docker, Kubernetes, Shell, SSH, VirtualBox, and others. Technically…

How to leverage CSV, Cloud Storage, and Data Studio to support fast data-driven decision making — What comes to mind when you hear “data-driven”? Expensive BI system licenses? Complex ETL processes that prepare data to populate dashboards? Corporate or technical constraints that prevent you from accessing relevant data to make decisions? Or affordable tools that are available whenever you need them, even to guide simple decisions…