Web Scraping Using Selenium - Navigation Using Browser History

Selenium Navigation - History

Navigations in selenium can be done in different ways. In this tutorial, we will use the back and forward button of the browser as a form of navigation.

But before that, please make sure you have read the first blog on this series to do the prerequisites.

Selenium Navigation History - Back and Forward

1. Create a file seleniumnavhistory.py and paste the following codes

   from selenium import webdriver
   import time
   

   The codes above imports the required library that we will use.
2. Add this line

   driver = webdriver.Firefox(executable_path="geckodriver.exe")
   

   The code above will create a webdriver instance for Firefox.
3. Add this line

   driver.get("https://slackingslacker.github.io/seleniumindex")
   

   The line will execute a script to go to the webpage https://slackingslacker.github.io/seleniumindex.
4. Add this line

   time.sleep(3)
   

   We will pause the program for 3 seconds.
5. Add this line

   driver.get("https://slackingslacker.github.io/seleniumindex#/about")
   

   The line will execute a script to go to the webpage https://slackingslacker.github.io/seleniumindex#/about.
6. Add this line

   time.sleep(3)
   

   We will pause the program for 3 seconds. At this time we have history in the browser.
7. Add this line

   driver.back()
   

   The line will go back to the previous page which is https://slackingslacker.github.io/seleniumindex.
8. Add this line

   time.sleep(10)
   

   We will pause the program for 10 seconds. You may have noticed that the current page is the main page.
9. Add this line

   driver.forward()
   

   The line will go back to the next page which is https://slackingslacker.github.io/seleniumindex#/about.
10. Add this line

    time.sleep(10)
    

    We will pause the program for 10 seconds. You may have noticed that the current page is the about page.
11. Add this line

    driver.close()
    

    The line will close the webdriver as well as the browser.
12. Run the seleniumnavhistory.py. It should do the following:
    • Open the firefox browser
    • Browser goes to https://slackingslacker.github.io/seleniumindex
    • Halts for 3 seconds
    • Browser goes to https://slackingslacker.github.io/seleniumindex#/about
    • Halts for 3 seconds
    • Browser goes to https://slackingslacker.github.io/seleniumindex using the back of history
    • Halts for 10 seconds
    • Browser goes to https://slackingslacker.github.io/seleniumindex#/about using the forward of history
    • Halts for 10 seconds
    • Closes the browser

 

Final Selenium Code

from selenium import webdriver
import time

driver = webdriver.Firefox(executable_path="geckodriver.exe")
driver.get("https://slackingslacker.github.io/seleniumindex")
time.sleep(3)
driver.get("https://slackingslacker.github.io/seleniumindex#/about")
time.sleep(3)

driver.back()
time.sleep(10)

driver.forward()
time.sleep(10)

driver.close()

 

Conclusion

Navigation in selenium can be done using browser history.