A sequel to “WebSockets and AsyncIO: Beyond 5-line Samples”

sports court with lines and numbers marked
sports court with lines and numbers marked
Photo by Pocky Lee on Unsplash

In this sequel to a previous article, I discuss design, libraries, and quality assurance in the scope of WebSockets, asynchronous programming, and concurrency with Python.

In the first article, we took a look at the WebSockets technology and how it fits Python. I’ve also proposed a fictitious use case to add more context to the discussion.

Now it’s time to shift attention to AsyncIO, a built-in library to write concurrent code using the async/await syntax. (I recommend taking a look at this tutorial if you’re not familiar with it.) …


Thoughts on async and concurrent programming with Python

A man and a woman at a computer
A man and a woman at a computer
Photo by heylagostechie on Unsplash

My team was recently assigned a task that comprises writing a Python client to interact with a specific WebSockets API. It’s led us to learn new things and the goal of this article is to share our progress. We believe there’s always room for improvement, so please feel free to comment and bring your expertise to the table.

When we say “beyond 5-line samples,” we mean that we found guidance when talking to our coworkers and searching the internet, but we missed references that bring related technology, design tips, programming libraries, and quality assurance matters together. …


Leverage Data Catalog to discover & annotate Qlik Sense assets

Background photo by Lauren Mancke on Unsplash

In this blog post, I’m going to share thoughts and knowledge that drove the development of a new sample connector for Google Data Catalog: the Qlik Sense connector.

This is the third open-source connector that allows customers to ingest metadata from BI Systems into Data Catalog: the ones for Looker and Tableau have been available for a while.

Disclaimer: Google and/or Google Cloud do not officially support any tool to connect Data Catalog to non-GCP systems at the time this article has been written down (January 2021). …


A custom model

More and more companies migrate their data workloads to the cloud every day. At the same time, new data protection regulation programs become effective around the globe. Such simultaneous events lead companies to enforce their Data Governance standards intending to avoid legal charges due to inappropriate data management.

Data Governance frameworks are hence becoming more common, but providers not necessarily speak the same language — what any of us could certainly expect as each player has their own strategies to tackle problems… On the other hand, customers have distinct requirements that are not fulfilled by a single platform. …


Hints and tips to create your own driver from scratch

Photo by Aron Visuals on Unsplash

GitLab is rich and scalable when it comes to software architecture, and Runner is one of the key components of such an ecosystem. It is responsible for running CI/CD jobs and deployed decoupled from the GitLab instance.

Runners come in multiple flavors: Docker, Kubernetes, Shell, SSH, VirtualBox, and others. Technically speaking, each “flavor” is implemented as an Executor. But what if you want to run the CI/CD jobs in an infrastructure that is not supported by the native executors or to scale your Runners fleet in a custom strategy that better fits your needs? …


How to leverage CSV, Cloud Storage, and Data Studio to support fast data-driven decision making

Photo by Luke Chesser on Unsplash

What comes to mind when you hear “data-driven”? Expensive BI system licenses? Complex ETL processes that prepare data to populate dashboards? Corporate or technical constraints that prevent you from accessing relevant data to make decisions? Or affordable tools that are available whenever you need them, even to guide simple decisions? I hope you answered “The last option!”, but I bring good news for you in case you didn’t.

Well, “data-driven” has been a trending topic for a long time and it is not difficult to understand why. As all of us know, people, gadgets, and devices — aka “things” —…


Leverage Data Catalog to discover & annotate Tableau assets

Background photo by Lauren Mancke on Unsplash

The Google Cloud Data Catalog Team has recently announced its product is now GA and ready to accept custom (aka user-defined) entries! This brand new feature opens up scope for integrations and now users can leverage Data Catalog’s well-known potential to manage metadata from almost any kind of data asset.

To demonstrate how it works, I’ll share design thoughts and sample code to connect Data Catalog to market-leader Business Intelligence/Data Visualization tools, covering Tableau metadata integration in this blog post. They come from the experience of participating in the development of fully operational sample connectors, publicly available on GitHub.

Disclaimer…


Leverage Data Catalog to discover & annotate Looker assets

Background photo by Lauren Mancke on Unsplash

The Google Cloud Data Catalog Team has recently announced its product is now GA and ready to accept custom (aka user-defined) entries! This brand new feature opens up scope for integrations and now users can leverage Data Catalog’s well-known potential to manage metadata from almost any kind of data asset.

To demonstrate how it works, I’ll share design thoughts and sample code to connect Data Catalog to market-leader Business Intelligence/Data Visualization tools, covering Looker metadata integration in this blog post. They come from the experience of participating in the development of fully operational sample connectors, publicly available on GitHub.

Disclaimer…


How the most important soft skill may help you to succeed

Photo by Power Digital Marketing on Unsplash

Some of my co-workers have been encouraging me to write something about soft skills for a while, so let me try!

Soft skills are a set of competencies a person should develop in order to empathize and do a better job, especially when working with teams. They’re no more or less important than technical (aka hard) skills. They are peers — they make relationships more human.

I’ll start with some insights from Gordon Haff (Red Hat). …


Leverage Public-key cryptography to establish secure connections

SSH into an AWS Fargate managed container
SSH into an AWS Fargate managed container
Background photo by chuttersnap on Unsplash

I’ve recently joined a conversation on how to establish secure connections to AWS Fargate-managed containers. SSH is one of the first options that come into mind when we talk about this… but how do these technologies fit together? After discussing with my teammates and some research, we came with a solution that seems to be compliant with AWS security standards, making use of a few products:

  • Amazon Elastic Container Registry
  • AWS Systems Manager Parameter Store (optional)
  • Amazon ECS Task Definitions
  • AWS Fargate

Ricardo Mendes

staff engineer @ ciandt.com • google cloud certified architect • hobbyist tech writer • father of a young princess • birder

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store