Home > Software engineering >  Basic Error while web scraping using Selenium and Edge with Python
Basic Error while web scraping using Selenium and Edge with Python

Time:01-16

Code trials:

!pip install selenium
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from time import sleep
from datetime import datetime
import pandas as pd
errors = []
season = []

The code runs fine till loading. An error pops in when I write the following code:

for id in range(46605, 46985):
my_url = f'https://www.premierleague.com/match/{id}'
option = Options()
option.headless = True
driver = webdriver.Edge(options=option)
driver.get(my_url)

The following error pops up:

Error Screenshot 1

Error Screenshot 2:

Error Screenshot 2

CodePudding user response:

To use the Chromium based with Selenium v4.x you have to install the following packages:

  • msedge-selenium-tools:

    pip install msedge-selenium-tools
    
  • Code Block :

    from msedge.selenium_tools import Edge
    
    s = Service('/path/to/edge')
    driver = Edge(service=s)
    

To use the Chromium based with in mode you need the EdgeOptions class.

  • Code Block :

    from msedge.selenium_tools import EdgeOptions
    from msedge.selenium_tools import Edge
    
    # make Edge headless
    edge_options = EdgeOptions()
    edge_options.use_chromium = True  # required to make Edge headless
    s = Service('/path/to/edge')
    driver = Edge(service=s, options=edge_options)
    
  •  Tags:  
  • Related