MustaThis Tutorial about how to write a Search and save Google maps results in Excel Script using Python and Selenium, that searches for something within a location and stores results in Excel.
In this tutorial, you’ll learn:
- How to use selenium with Python
- about Gecko Driver
- How to select web-pages objects and different methods
- How to build a scripts that manipulates Google bot with Selenium
What You Need
- Installed Firefox browser
- Install Gecko: Firefox Web-driver from here
- Python 3.x
- Selenium Package from here
Writing the code
To begin, let’s import necessary package, where the most important is Selenium:
|
1 2 3 4 5 6 7 8 9 10 |
from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.common.keys import Keys from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support.expected_conditions import presence_of_element_located from selenium.common.exceptions import NoSuchElementException import time import string import openpyxl import os |
Then you need to define the webdriver and make it open google maps as shown below. Note that at the end of each move, you need to add a pause. "time.sleep(x seconds)".
|
1 2 3 4 5 6 7 |
#Loading Selenium Webdriver driver= webdriver.Firefox() wait = WebDriverWait(driver, 5) #Opening Google maps driver.get("https://www.google.com/maps") time.sleep(3) |
The next step is close the google consent form, and in order to do this, you need to tell the script to look for the “I accept” . Selenium provides a “switch to frame” tool when treating the widget as a independent object.
The default process within any script, you are telling the program to pick element by an identifier. Selenium provides many ways to identify page elements using a find_element_by or find_elements_by, your next step will be like this.
|
1 2 3 4 5 |
#Closing the google consent form widget=driver.find_element_by_tag_name("iframe") driver.switch_to_frame(widget) button=driver.find_element_by_xpath('.//*[@id="introAgreeButton"]') button.click() |
To get the necessary values that Identify each element, you need to copy it from the page HTML code by inspecting elements.
Afterwards, the code shall be written as follows:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
#Finding the search box driver.switch_to_default_content() searchbox=driver.find_element_by_id('searchboxinput') location= "Málaga" searchbox.send_keys(location) searchbox.send_keys(Keys.ENTER) time.sleep(2) cancelbut=driver.find_element_by_class_name('gsst_a') cancelbut.click() searchbox.send_keys("seguro") searchbox.send_keys(Keys.ENTER) time.sleep(3) |
Up to this point, the bot should be able to open Google Maps and look for something . The next step shall be store each element result to an excel file.
the next block of code is for locating the search results list and preparing the excel file.
|
1 2 3 4 5 6 7 8 9 |
#Locating the results section entries=driver.find_elements_by_class_name('section-result') #Prepare the excel file using the Openpyxl wb= openpyxl.load_workbook("comapnies.xlsx") sheetname=wb.get_sheet_names()[0] sheet=wb[sheetname] sheet.title ="companies" |
Finally, for each entry extracting the the name, address, phone number and website, and at the end save the excel file.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
#Extracting the information from the results for entry in entries: #Empty list labels=[] #Extracting the Name, adress, Phone, and website: name= entry.get_attribute("aria-label") adress= entry.find_element_by_class_name('section-result-location').text phone = entry.find_element_by_class_name('section-result-hours-phone-container').text.split(' · ')[-1] try: webcontainer= entry.find_element_by_class_name('section-result-action-container') website=entry.find_element_by_tag_name('a').get_attribute("href") except NoSuchElementException: website="No website could be found" print (name) print (website) #Try/except to write the extracted info in the Excel file pass if doessn't exist try: sheet.append([location,name,adress,phone,website]) except IndexError: pass #saving the excel file wb.save("companies.xlsx") |
Notes:
Google maps has a different layout design for special categories, so eventually the script won’t work correctly . also you should have your excel ready because the script doesn’t create the file
And that is it, good luck with your Search and save Google maps results in Excel Script . further reading would be the Selenium documentation. also below is the video for this tutorial.
Watch The video: Google Maps Collector to Excel Using Selenium Webdriver
checkout more python from Genius Panda
The Whole Code
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 |
#Importing libraries from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.common.keys import Keys from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support.expected_conditions import presence_of_element_located from selenium.common.exceptions import NoSuchElementException import time import string import openpyxl import os #Loading Selenium Webdriver driver= webdriver.Firefox() wait = WebDriverWait(driver, 5) #Opening Google maps driver.get("https://www.google.com/maps") time.sleep(3) #Closing the google consent form widget=driver.find_element_by_tag_name("iframe") driver.switch_to_frame(widget) button=driver.find_element_by_xpath('.//*[@id="introAgreeButton"]') button.click() #Finding the search box driver.switch_to_default_content() searchbox=driver.find_element_by_id('searchboxinput') location= "Málaga" searchbox.send_keys(location) searchbox.send_keys(Keys.ENTER) time.sleep(2) cancelbut=driver.find_element_by_class_name('gsst_a') cancelbut.click() searchbox.send_keys("seguro") searchbox.send_keys(Keys.ENTER) time.sleep(3) #Locating the results section entries=driver.find_elements_by_class_name('section-result') #Prepare the excel file using the Openpyxl wb= openpyxl.load_workbook("comapnies.xlsx") sheetname=wb.get_sheet_names()[0] sheet=wb[sheetname] sheet.title ="companies" #Extracting the information from the results for entry in entries: #Empty list labels=[] #Extracting the Name, adress, Phone, and website: name= entry.get_attribute("aria-label") adress= entry.find_element_by_class_name('section-result-location').text phone = entry.find_element_by_class_name('section-result-hours-phone-container').text.split(' · ')[-1] try: webcontainer= entry.find_element_by_class_name('section-result-action-container') website=entry.find_element_by_tag_name('a').get_attribute("href") except NoSuchElementException: website="No website could be found" print (name) print (website) #Try/except to write the extracted info in the Excel file pass if doessn't exist try: sheet.append([location,name,adress,phone,website]) except IndexError: pass #saving the excel file wb.save("companies.xlsx") |
