Speed up Elasticsearch Test Containers-CodePudding

I have moved the java integration tests to use elasticsearch test containers instead of using embedded elasticsearch. The tests have become slower by 1 hour which is a huge productivity hit. I am looking for ways to speed that up.

I tried using params like reuse on Elasticsearch container but that didn't make a dent. My latest configuration is

    private static final String ELASTICSEARCH_VERSION = "7.11.2";
    private static ElasticsearchContainer elasticsearchContainer;
    private static final DockerImageName ELASTICSEARCH_IMAGE =
          DockerImageName
                .parse("docker.elastic.co/elasticsearch/elasticsearch")
                .withTag(ELASTICSEARCH_VERSION);

            elasticsearchContainer = new ElasticsearchContainer(ELASTICSEARCH_IMAGE)
                  .withEnv("foo", "bar").withSharedMemorySize(1000000000L);
            elasticsearchContainer.addExposedPorts(9200, 9300);
            elasticsearchContainer.withStartupTimeout(Duration.of(5, ChronoUnit.MINUTES));
            elasticsearchContainer.start();

    private static RestHighLevelClient getRestHighLevelClient(ElasticsearchContainer container) {
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY,
                                           new UsernamePasswordCredentials(ELASTICSEARCH_USERNAME,
                                                                           ELASTICSEARCH_PASSWORD));
        RestClientBuilder restClientBuilder = RestClient.builder(HttpHost.create(container.getHttpHostAddress()))
              .setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder
                    .setDefaultCredentialsProvider(credentialsProvider)
                    .setKeepAliveStrategy((response, context) -> 3 * 60 * 1000));
        // Try to prevent SocketTimeoutException when fetching larger batch size
        restClientBuilder.setRequestConfigCallback(
              requestConfigBuilder -> requestConfigBuilder.setSocketTimeout(2 * 60 * 1000));

        return new RestHighLevelClient(restClientBuilder);
    }

Gradle config (gradle daemon crashed once so increased to 2g)

org.gradle.jvmargs=-Xms2g -XX: HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8

Any suggestions on making the tests faster?

CodePudding user response：

General advice for any performance work or optimizations is to measure before introducing changes. I'd recommend profiling your test runs before you draw conclusions.

There are a few metrics there that you can estimate without extensive use of profilers.

How long does the Elastic container start?

For me for example it takes 8.2 seconds (I used elasticsearch-oss:7.10.2):

14:35:55.803 [main] INFO