I have two collections, that are store on Atlas.

I want to be able to search for a text, using the regex operator of Mongo Atlas Search on these two collections, but with only one aggregation.

These two collections have nothing in common, so I can't do a $lookup on it.

I need to do this search in one aggregation because the result of this aggregation must be paginated with $limit and $skip in the aggregation.

How to perform that ?

CodePudding user response：

You can use Atlas Data Lake on top of these two collections to federate queries across both. To do this you would create a Data Lake, create a virtual collection in your Data Lake, and then have both of these collections listed as "datasources" for that virtual collection.

The Data Lake infrastructure will push down queries to the underlying cluster and union the results before returning them to the client application.

In this example there are multiple types of data sources, but you can just list multiple cluster collections. https://docs.mongodb.com/datalake/reference/format/data-lake-configuration/#example-configuration-for-running-federated-queries

CodePudding user response：

One option is to use a combination of change streams & materialized view to create a collection which contains documents from other collections. The workflow occurs as follows:

1. Listen in on collections you'd like to merge from via Change Streams (or any PubSub tool like Kafka, RabbitMQ, etc.)
2. Whenever there's a change to these collections (update, delete, insert) use $merge to pipe it into a new collection
3. Build your index on this new "converged" collection, where you can then perform $search queries