When I want to remove all objects from my mongoDB collection comments I do this with this command:

mongo $MONGODB_URI --eval 'db.comments.deleteMany({});'

However, this is super slow when there are millions of records inside the collection.

In a relational db like Postgres I'd simply copy the structure of the collection, create a comments2 collection, drop the comments collection, and rename comments2 to comments.

Is this possible to do in MongoDB as well? Or are there any other tricks to speed up the progress?

CodePudding user response：

Thanks, the answers inspired my own solution. I forgot that MongoDB doesn't have a schema like a relationalDB.

So what I did is this:

1. dump an empty collection the indexes of the collection

mongodump --host=127.0.0.1 --port=7001 --db=coral --collection=comments --query='{"id": "doesntexist"}'  --out=./dump

This will create a folder ./dump with the contents comments.bson (empty) and comments.metadata.json

2. Drop the comments collection

mongo mongodb://127.0.0.1:7001/coral --eval 'db.comments.drop();'

3. Import new data new_comments.json (different from comments.bson)

mongoimport --uri=mongodb://127.0.0.1:7001/coral --file=new_comments.json --collection comments --numInsertionWorkers 12

This is way faster than first adding the indexes, and then importing.

4. Add indexes back

mongorestore --uri=mongodb://127.0.0.1:7001/coral --dir dump/coral --nsInclude coral.comments --numInsertionWorkersPerCollection 12

Note that --numInsertionWorkers speeds up to process by dividing the work over 12 cpus. How many cpus do you have can be found on OSx with:

sysctl -n hw.ncpu

CodePudding user response：

db.cities.aggregate([{ $match: {} }, { $out: "collection2" }]) in case you can login to the mongo prompt and simply drop the previous collection. Otherwise, the approach you have posted is the one.

mongoexport.exe /host: /port: /db:test /collection:collection1 /out:collection1.json mongoimport.exe /host: /port: /db:test /collection:collection2 /file:collection1.json

Thanks, Neha

CodePudding user response：

For mongodb version >=4.0 you can do this via db.comments.renameCollection("comments2") ,but it is kind of resource intensive operation and for bigger collections better you do mongodump/mongorestore. So the best action steps are:

 mongodump -d x -c comments -out dump.bson
 >use x
 >db.comments.drop()
 mongorestore -d x -c comments2  dump.bson

Plese, note deleteMany({}) is even more resource intensive operation since it will create oplog single entry for every document you delete and propagate to all replicaSet members.