how to scrape website with infinite scroll?
07:53 23 Sep 2023

I want to scrape website in which, when we scroll down scrollbar the new data starts loading. how to scrape this website?

hospitals2 = []

for i in range(1,10):
    try:     
        element = driver.find_element(By.XPATH, paths[i]) 
        hospital_name=element.text
        driver.implicitly_wait(5)
        driver.execute_script("arguments[0].scrollIntoView();",element)
        hospitals2.append(hospital_name)
    except AttributeError:
        hospitals2.append("hospital name not found")

I tried above code but new data is not loading.

python selenium-webdriver web-scraping