Photo by fabio on Unsplash

Magda is an open-sourced data catalog system aims to solve the data discovery problem. In the recent v0.0.60 release, we introduced more features that help people to create their own tailored build and integration with Discourse. Here are a few highlights.

External UI Plugin Support

Prior to v0.0.60, Magda allows us to adjust frontend looking & feels by suppling SCSS variables via Helm Chart configuration. However, when we want to alter the frontend logic, we will have to fork the codebase and completely replace the Magda frontend module.

External UI Plugin offers a more flexible solution. Instead, you can supply a prebuilt React UI…

Photo by Denny Müller on Unsplash

Why Docker Registry Mirror?

Since November 20, 2020, Anonymous and Free Docker Hub users are limited to 100 and 200 container image pull requests per six hours. If you build docker images or deploy test application with your local minikube dev cluster frequent enough, eventually you will hit the “Too Many Requests” or “Pull Rate Limit Reached” errors.

To overcome this issue, we can install a docker registry mirror works as a pull-through cache in your minikube cluster. It pulls the image from the Docker Hub registry and stores it locally before handing it back to you when the first time you request an…

Photo by Robson Hatsukami Morgan on Unsplash

As an open-sourced data catalog system, many people build their own data catalog system on top of Magda. To help developers to build customised authentication process around Magda, we introduced the external authentication plugin concept in release v0.0.58. Together with the history API improvement and distribution version view, we will talk about the key enhancements that v0.0.58 release brought to you.

Authentication Plugin

Since v0.0.58, Magda allows you to build external authentication plugins to support different authorisation servers/identity providers. It also allows you to customise the user onboarding process (i.e. creating & mapping the user’s profile to a local user (e.g. …

Photo by Isis França on Unsplash

Why HTTPS on a Local Dev Cluster

Since Google’s SameSite Cookie Changes Roll-out, you will find it is getting harder to test a local Web UI client with an API backend running in a local K8s dev cluster. Firstly, you need to set your cookie’s SameSite setting to “None” in order to enable cross-origin requests. However, a cookie withSameSite=None setting only will also be rejected by Google Chrome unless you set Secure=true to make the cookie only available through HTTPS connections. This makes HTTPS access a “must-have” when test your local cluster for the use case above.

Install Cert Manager

To issue a self-signed certificate for your cluster, you need…

Photo by Jo Szczepanska on Unsplash

Why do we ever want to modify the deployed Helm release manifest?

Helm always tries to match Kubernetes objects in the manifest by “kind” (e.g. Deployment) and “API version” (e.g. apps/v1). However, the “API version” supported by your Kubernetes cluster could change, e.g. after upgrading the cluster to a higher version or installing a different version CRD (Custom Resource Definitions) for a third-party application (e.g. Cert Manager).

When the “API version” that was used to deploy a Kubernetes object of a “kind” is no longer supported by the cluster, Helm will fail to match the object in the release manifest and report an error:

unable to recognize "": no matches for kind…

Photo by Joe Leahy on Unsplash

Magda is an open-sourced data catalog system that provides a single place where all of your organisation’s data can be catalogued, enriched, searched, tracked and prioritized.

We’ve just published Magda v0.0.57, which is the first production release in the last 8 months. However, we also packed a lot of good features which I believe is worth the wait :-)

Kubernetes API 1.16 Support

Since version 1.16, Kubernetes API stopped serving many deprecated API versions in favour of newer and more stable API versions. Consequently, earlier version Magda won’t support Kubernetes API 1.16 …

Photo by Rick Mason on Unsplash

When managing a production cluster, you might need to import existing k8s cluster resources to a Helm deployment from time to time. It could be because you try to migrate existing cluster app that is not deployed by Helm or manually fix some errors during the migration/upgrade. Being able to import existing resources means you don’t have to delete all resources (which is usually not an acceptable option for production) and redeploy the whole application, which is crucial for managing production deployment.

Since Helm 3.2, it’s possible to import/adopt existing resources into the helm release¹. …

Photo by Brusk Dede on Unsplash

Helm upgrade --install is an idempotent way of deploying a Helm Chart depends on if the release exists, which is very useful in an automated CI environment.

However, as of Helm 2.7.1, if you run Helm upgrade --install on a previous failed release, the following error will be printed out:

Error: UPGRADE FAILED: "xxx" has no deployed releases

This new behaviour was introduced by this pull request since Helm 2.7.1 and impact Helm 3 as well.

Why this behaviour has been introduced?

In fact, Helm 2’s tiller doesn’t support 3-ways merge of the current manifest, the new manifest, and the actual state of the resources. Instead…

Photo by Mika Baumeister on Unsplash

Why Policy-based Access Control?

Policy-based access control (or known as attribute-based access control) solution often comes with a central Policy Engine that allows you to centralize all authorisation decision logic across your application stack into a central service.

Photo by Steve Johnson on Unsplash

What is React Hooks?

React Hooks have been officially released with React v16.8.0 on 6th Feb 2019. With React Hooks, you can easily introduce stateful logic into your React Function Components. Here is an example:

At line 6, we call the useState hook to create a local state count and add it to the function component Example . To update the local state count , just simply call the update function setCount .

Here, you don’t need to worry about the useState(0) call during the rendering will reset your local state to initial value 0 as the useState will only initialize the local…

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store