Firebase Auth emulator fails intermittently with "auth/emulator-config-failed"-CodePudding

I'm trying to use the emulator only when it is available. connectAuthEmulator does not fail if the emulator is not available (i.e.: if I haven't ran firebase emulators:start). It fails later when I try to make a request.

To do this, I'm fetching the emulator URL (http://localhost:9099) using fetch. If the request succeeds, then I call connectAuthEmulator. Otherwise, I do nothing (uses the configured cloud service).

I have a problem where the fetch works, but connectAuthEmulator throws an error: auth/emulator-config-failed. For some weird reason, this seem to happen only when I'm logged in and I make no requests for about 15 seconds. If I spam requests however, the error never occurs.

import { initializeApp } from "firebase/app";
import { getAuth, connectAuthEmulator } from "firebase/auth";

const app = initializeApp({ /** ... */ });
const auth = getAuth(app);

if (NODE_ENV === "development") {
    (async () => {
        try {
            const authEmulatorUrl = "http://localhost:9099";
            await fetch(authEmulatorUrl);
            connectAuthEmulator(auth, authEmulatorUrl, {
                disableWarnings: true,
            });
            console.info("