Per Flink's doc, we can deploy a standalone Flink cluster on top of Kubernetes, using Flink’s standalone deployment， or deploy Flink on Kubernetes using native Kubernetes deployments.
The document says
We generally recommend new users to deploy Flink on Kubernetes using native Kubernetes deployments.
Is it because native Kubernetes is easier to get start with, or is it because standalone mode is kind of legacy ?
In native Kubernetes mode, Flink is able to dynamically allocate and de-allocate TaskManagers depending on the required resources. While in standalone mode, task managers have to be provisioned manually.
It sounds to me that native Kubernetes mode is a better choice.
CodePudding user response：
Kubernetes is a PaaS platform, where you just worry about your application deployment and Kubernetes takes care of the infrastructure, networking, scaling and LCM.
Also, Flink can be easily installed on Kubernetes using a helm chart. This makes the installation trivial.
$ helm repo add riskfocus https://riskfocus.github.io/helm-charts-public
$ helm repo update
$ helm install --name my-flink riskfocus/flink
ps - requires helm - super easy to install as well
CodePudding user response：
Posted community wiki based on other answers - David Anderson answer and austin_ce answer. Feel free to expand it.
Good explanation from the David Anderson answer:
In a Kubernetes session or per-job deployment, Flink has no idea it's running on Kubernetes. In this mode, Flink behaves as it does in any standalone deployment (where there is no cluster framework available to do resource management). Kubernetes just happens to be how the infrastructure was created, but as far as Flink is concerned, it could have been bare metal. You will have to arrange for kubernetes to create the infrastructure that you will have configured Flink to expect.
- Session deployment
In a Native Kubernetes session deployment, Flink uses its
KubernetesResourceManager, which submits a description of the cluster it wants to the Kubernetes ApiServer, which creates it. As jobs come and go, and the requirements for task managers (and slots) go up and down, Flink is able to obtain and release resources from kubernetes as appropriate. - Application mode
In Application Mode (blog post) (details) you end up with Flink running as a kubernetes application, which will automatically create and destroy cluster components as needed for the job(s) in one Flink application.
The native mode is recommended because it is just simpler, I would not say it is legacy:
The
Nativemode is the current recommendation for starting out on Kubernetes as it is the simplest option, like you noted. In Flink 1.13 (to be released in the coming weeks), there is added support for specifying Pod templates. One of the drawbacks to this approach is its limited ability to integrate with CI/CD.