Connecting to Azure CosmosDB (MongoDB APIs) from Node Server hosted inside AKS fails-CodePudding

I have Azure CosmosDB account with MongoDB APIs enabled, i'm trying to connect from Node Server deployed with 3 replicas (3 Pods) inside Azure Kubernetes.

Only one POD managed to connect to CosmosDB and the other Pods fails, the error is persistent and nothing in CosmosDB is configured to cause this issue.

Note that same issue happened with another Java application so it's not the Node Server i believe.

The Exception When NodeJS Try to Connect:

MongoServerSelectionError: getaddrinfo EAI_AGAIN abc.mongo.cosmos.azure.com
    at Timeout._onTimeout (/app/node_modules/mongodb/lib/sdam/topology.js:312:38)
    at listOnTimeout (internal/timers.js:554:17)
    at processTimers (internal/timers.js:497:7) {
  reason: TopologyDescription {
    type: 'ReplicaSetNoPrimary',
    servers: Map {
      'abc.mongo.cosmos.azure.com:10255' => [ServerDescription]
    },
    stale: false,
    compatible: true,
    heartbeatFrequencyMS: 10000,
    localThresholdMS: 15,
    setName: 'globaldb',
    logicalSessionTimeoutMinutes: undefined
  }
}

The code:

export const connectToDB = async () => {
  const connectionString: string = process.env.CONNECTION_STRING;

  try {
    const mongoClient = new MongoClient(connectionString);
    mongoClient
      .connect()
      .then(() => {
        console.log("Connected successfully to server");
        return Promise.resolve();
      })
      .catch((error: any) => {
        console.log("failed to connect to mongodb!");
        console.log(error);
      });
  } catch (error: any) {
    console.log(error);
  }
};

CodePudding user response：

The error EAI_AGAIN indicates it's a DNS related timeout.

You could bash into one of the pods and dig abc.mongo.cosmos.azure.com to check whether it has a valid DNS configuration.

DNS in Kubernetes can be a little bit error prone, this guide has an excellent walkthrough how they investigated it and they ended up using NodeLocal DNSCache

Also restarting the Code DNS Pods sometimes helps and having your overall cluster in a good state.