Home > Software engineering >  Pymongo query to extract value of a matching key
Pymongo query to extract value of a matching key

Time:02-02

I have below document:

{
    "_id": "61f7d5cfd0c32b744d3f81c2",
    "_form": "61e66b8fd0c32b744d3e24a0",
    "_workflow": "61e54fe2d0c32b744d3e0b7c",
    "_appUser": "61e6b098d0c32b744d3e3808",
    "sectionResponse": [{
            "_id": "61f7d5cfd0c32b744d3f81c3",
            "name": "Project Details & Goals",
            "order": 2,
            "fieldResponse": [{
                    "_id": "61f7d5cfd0c32b744d3f81c4",
                    "fieldType": "Text",
                    "name": "Project Name",
                    "value": "TRT",
                    "order": 0
                },
                {
                    "_id": "61f7d5cfd0c32b744d3f81c5",
                    "fieldType": "Number",
                    "name": "Amount Requested",
                    "value": "20",
                    "order": 1
                },
                {
                    "_id": "61f7d5cfd0c32b744d3f81c6",
                    "fieldType": "Number",
                    "name": "Project Cost",
                    "value": "50",
                    "order": 1
                },
                {
                    "_id": "61f7d5cfd0c32b744d3f81c7",
                    "fieldType": "Comment",
                    "name": "Project Goals",
                    "value": "TRT",
                    "order": 3
                }
            ]
        },
        {
            "_id": "61f7d5cfd0c32b744d3f81c8",
            "name": "Section Heading",
            "order": 2,
            "fieldResponse": [{
                    "_id": "61f7d5cfd0c32b744d3f81c9",
                    "fieldType": "Multiselectdropdown",
                    "name": "Multiselectdropdown",
                    "value": "Y",
                    "order": 0
                },
                {
                    "_id": "61f7d5cfd0c32b744d3f81ca",
                    "fieldType": "Image_Upload",
                    "name": "Image Upload",
                    "value": "Y",
                    "order": 1
                }
            ]
        }
    ],
    "order": 2,
    "status": "Reviewed",
    "updatedAt": "2022-01-31T12:27:59.541Z",
    "createdAt": "2022-01-31T12:27:59.541Z",
    "__v": 0
}

Inside the document, there is a sectionResponse which contains response of multiple sections. Inside this, there is a fieldResponse which contains the name and value. I have to extract the value from all the documents where name is Amount Requested.

Can anyone help me in writing a query for such situation. Please help. Thanks

CodePudding user response:

find({"sectionResponse.fieldResponse.name" : "amount requested"})

It will return if there is only one match or many matches. But not only matched elements but all the array elements.

playground

CodePudding user response:

Here is a solution that returns only matching material and requires no $unwind.

 db.foo.aggregate([
    // This stage alone is enough to give you the basic info.                                         
    // You will get not only doc _id but also an array of arrays                                      
    // (fieldResponse within sectionResponse) containing the whole                                    
    // fieldResponse doc.  It is slight awkward but if you need structural data                                   
    // other than *just* the value, it is a good start:                                              
    {$project: {
        // outer filter removes inner filter results where size is 0                                  
        // i.e. no "Amount Requested" found.                                                          
        XX: {$filter: {input:
                       {$map: {
                           input: "$sectionResponse", as: "z1", in:
                           // inner filter gets only name = Amount Requested                          
                           {$filter: {input: "$$z1.fieldResponse",
                                      as: "z1",
                                      cond: {$eq:["$$z1.name","Amount Requested"]}
                                     }}
                       }},
                       as: "z2",
                       cond: {$ne: ["$$z2", [] ]}
                      }}
    }}

which yields (given a slightly expanded input set where subdocs were copied but the value and order changed for clarity):

{
    "_id" : 0,
    "XX" : [
        [
            {
                "_id" : "61f7d5cfd0c32b744d3f81c5",
                "fieldType" : "Number",
                "name" : "Amount Requested",
                "value" : "20",
                "order" : 1
            },
            {
                "_id" : "61f7d5cfd0c32b744d3f81c5",
                "fieldType" : "Number",
                "name" : "Amount Requested",
                "value" : "77",
                "order" : 18
            }
        ],
        [
            {
                "_id" : "61f7d5cfd0c32b744d3f81c5",
                "fieldType" : "Number",
                "name" : "Amount Requested",
                "value" : "99",
                "order" : 818
            }
        ]
    ]
}
{
    "_id" : 1,
    "XX" : [
        [
            {
                "_id" : "61f7d5cfd0c32b744d3f81c5",
                "fieldType" : "Number",
                "name" : "Amount Requested",
                "value" : "333",
                "order" : 1
            }
        ]
    ]
}
{ "_id" : 2, "XX" : [ ] }

If you don't want to know about top level docs that contained
NO fieldResponses where name = "Amount Requested" then append this stage:

    {$match: {XX: {$ne: [] }}}

Finally, if you really want just the values, append this reduce stage:

    ,{$addFields: {XX: {$reduce: {
        input: "$XX",
        initialValue: [],
            in: {$concatArrays: ["$$value",
                                 {$map: {input: "$$this",
                                         as:"z",  
                                         in: "$$z.value"
                                        }} ] }
        }}
    }}

which yields:

{ "_id" : 0, "XX" : [ "20", "77", "99" ] }
{ "_id" : 1, "XX" : [ "333" ] }

If you want a little more than just value(like order for example) then have $map return a doc instead of a scalar, e.g.:

                                 {$map: {input: "$$this",
                                         as:"z",
                                         in: {v:"$$z.value",o:"$$z.order"}            
                                 }} ] }

to yield:

{
    "_id" : 0,
    "XX" : [
        {
            "v" : "20",
            "o" : 1
        },
        {
            "v" : "77",
            "o" : 18
        },
        {
            "v" : "99",
            "o" : 818
        }
    ]
}
{ "_id" : 1, "XX" : [ { "v" : "333", "o" : 1 } ] }

Again, the input set provided by the OP was expanded with additional {name:"Amount Requested"} subdocs tossed into different sectionResponse arrays to generate a more complex structure.

CodePudding user response:

Maybe something like this which you may easy adapt to python supposing you need only value from sectionResponse.$[].fieldResponse.$[] elements having the name "Amount Requested":

 db.collection.aggregate([
 {
   $match: {
     "sectionResponse.fieldResponse.name": "Amount Requested"
    }
 },
 {
 "$project": {
  "sectionResponse": {
    "$filter": {
      "input": {
        "$map": {
          "input": "$sectionResponse",
          "as": "somesub",
          "in": {
            "_id": "$$somesub._id",
            "fieldResponse": {
              "$filter": {
                "input": "$$somesub.fieldResponse",
                "as": "sub",
                "cond": {
                  "$eq": [
                    "$$sub.name",
                    "Amount Requested"
                  ]
                }
              }
            }
          }
        }
      },
      "as": "some",
      "cond": {
        "$gt": [
          {
            "$size": "$$some.fieldResponse"
          },
          0
        ]
      }
     }
   }
  }
 },
 {
  $unwind: "$sectionResponse"
 },
 {
   $unwind: "$sectionResponse.fieldResponse"
 },
 {
  $project: {
  value: "$sectionResponse.fieldResponse.value"
 }
}
])
  1. Match the documents containing at least one element with sectionResponse.fieldResponse.name:"Amount Requested"
  2. project/map all sectionResponse.fieldResponse elements containing name="Amount Requested" ( non empty elements only )
  3. unwind the sectionResponse array
  4. unwind the fieldResponse array
  5. project only the value field.

playground

For best results index on "sectionResponse.fieldResponse.name" need to be added.

  •  Tags:  
  • Related