I am trying to get all the Key Ingredients for all the products under a category,

CodePudding user response：

One way could be to isolate the the key ingredients marker - move to the next tag (<br>) - then process all following tags until you reach the next <br> tag.

<strong>Key Ingredients:</strong>
<br>
<ul>
...
...
<ul>
<br>

code:

>>> for tag in description.find(string='Key Ingredients:').find_next('br').next_elements:
...     if tag.name == 'br': break
...     if tag.name == 'li': tag.get_text()
'Glycerin'
'Sodium Palmate'
'Sodium Palm Kemelate'
'Cymbopogon Flexuosus Oil'
'Linalool'
'Coumarin'
'Benzyl Salicylate'
'Citral'