how to scrape website with infinite scroll?
I want to scrape website in which, when we scroll down scrollbar the new data starts loading. how to scrape this website?
hospitals2 = []
for i in range(1,10):
try:
element = driver.find_element(By.XPATH, paths[i])
hospital_name=element.text
driver.implicitly_wait(5)
driver.execute_script("arguments[0].scrollIntoView();",element)
hospitals2.append(hospital_name)
except AttributeError:
hospitals2.append("hospital name not found")
I tried above code but new data is not loading.