Home > Net >  Jenkins K8s plugin WebSocket Timeout in container step
Jenkins K8s plugin WebSocket Timeout in container step

Time:01-11

I have created a K3d cluster. I've Deployed a jenkins 2.319.1 controller inside, along with kubernetes plugin 1.31.1 (and git, pipeline and the like) The idea is to run both the controller and the agents in the same cluster. To do so I've configured a cloud like in this picture: [Cloud Configuration ][1] [1]: https://i.stack.imgur.com/u91Fr.png

(I've done several attempts with different combinations for the agents to connect to the controller. Finally I've stayed with JNLP - NO WEBSOCKET - although, anyway, the timeout I'm about to describe is common to both of them)

With the cloud configured and being able to spawn the agents, after many attempts I finally discovered that the job was hanging (and dying of timeout) in the execution inside a container step. Actions within the default "jnlp" container are ok but the moment you do something as trivial as: sh 'ls -l' inside another container the job dies after 30 seconds with the following log:

    >    .
.
.
.
   readOnly: false
      nodeSelector:
        kubernetes.io/os: "linux"
      restartPolicy: "Never"
      volumes:
      - emptyDir:
          medium: ""
        name: "workspace-volume"
    
    Running on prueba-6-tj9w5-r0qt9-kcst4 in /home/jenkins/agent/workspace/Prueba
    [Pipeline] {
    [Pipeline] stage
    [Pipeline] { (Get a Maven project)
    [Pipeline] sh
      git config --global http.proxy http://10.11x.xx.xx:8080
    [Pipeline] sh
      git config --global https.proxy http://10.11x.xx.xx:8080
    [Pipeline] git
    The recommended git tool is: NONE
    No credentials specified
    Cloning the remote Git repository
    Cloning repository https://github.com/jenkinsci/kubernetes-plugin.git
     > git init /home/jenkins/agent/workspace/Prueba # timeout=10
    Fetching upstream changes from https://github.com/jenkinsci/kubernetes-plugin.git
     > git --version # timeout=10
     > git --version # 'git version 2.30.2'
     > git fetch --tags --force --progress -- https://github.com/jenkinsci/kubernetes-plugin.git  refs/heads/*:refs/remotes/origin/* # timeout=10
    Avoid second fetch
    Checking out Revision a61114b275425489761f095e8a89b19cf2ab5c8e (refs/remotes/origin/master)
     > git config remote.origin.url https://github.com/jenkinsci/kubernetes-plugin.git # timeout=10
     > git config --add remote.origin.fetch  refs/heads/*:refs/remotes/origin/* # timeout=10
     > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
     > git config core.sparsecheckout # timeout=10
     > git checkout -f a61114b275425489761f095e8a89b19cf2ab5c8e # timeout=10
     > git branch -a -v --no-abbrev # timeout=10
     > git checkout -b master a61114b275425489761f095e8a89b19cf2ab5c8e # timeout=10
    Commit message: "[maven-release-plugin] prepare for next development iteration"
    First time build. Skipping changelog.
    [Pipeline] container
    [Pipeline] {
    [Pipeline] stage
    [Pipeline] { (Build a Maven project)
    [Pipeline] sh
    [Pipeline] }
    [Pipeline] // stage
    [Pipeline] }
    [Pipeline] // container
    [Pipeline] }
    [Pipeline] // stage
    [Pipeline] }
    [Pipeline] // node
    [Pipeline] }
    [Pipeline] // podTemplate
    [Pipeline] End of Pipeline
    java.io.IOException: Timed out waiting for websocket connection. You should increase the value of system property org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator.websocketConnectionTimeout currently set at 60 seconds
        at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:457)
        at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:344)
        at hudson.Launcher$ProcStarter.start(Launcher.java:507)
.
.
.

I don't know the websocket message as in the cloud configuration this is clearly unchecked. As I mention if I check websocket (along with clearing tunnel) I get the same result.

I don't know if this has anything to do with jenkins running inside a dockerized cluster. I have installed the same cluster (k3d v4.4.7) both in wsl2 in windows and also in rhel 7.9.

Sample pipeline used (this last one borrowed from ):

> podTemplate(containers: [
    containerTemplate(name: 'maven', image: 'maven:3.3.9-jdk-8-alpine', ttyEnabled: true, command: 'cat'),
    containerTemplate(name: 'golang', image: 'golang:1.8.0', ttyEnabled: true, command: 'cat')
  ]) {

    node(POD_LABEL) {
        env.http_proxy='http_proxy=http://10.11x.xx.xx:8080'
        env.https_proxy='http_proxy=http://10.11x.xx.xx:8080'
        stage('Get a Maven project') {
            sh 'git config --global http.proxy http://10.11x.xx.xx:8080'
            sh 'git config --global https.proxy http://10.11x.xx.xx:8080'
            git 'https://github.com/jenkinsci/kubernetes-plugin.git'
            container('maven') {
                stage('Build a Maven project') {
                    sh 'mvn -B clean install'
                }
            }
        }

        stage('Get a Golang project') {
            git url: 'https://github.com/hashicorp/terraform.git'
            container('golang') {
                stage('Build a Go project') {
                    sh """
                    mkdir -p /go/src/github.com/hashicorp
                    ln -s `pwd` /go/src/github.com/hashicorp/terraform
                    cd /go/src/github.com/hashicorp/terraform && make core-dev
                    """
                }
            }
        }
    }
}

Regards

CodePudding user response:

As mentioned in previous comment referring to post (Jenkins with Kubernetes Client Plugin - NoSuchMethodError) upgrading kubernetes plugin to v1.31.2 solved the problem. Already patched and tested.

KR

  •  Tags:  
  • Related