Scraping Data From Google Maps Using Python With Source Code
Introduction :
In this article, we are going to see how to scrap all data from Google Maps like Addresses , Ratings , Descriptions , Contact Numbers , Reviews etc . using Python. Scraping Data from Google Maps Becomes a Popular Technique to gathering informations about the location or Places . It is used in various applications , like market analysis , competitor research or location-based sevices . Google Maps is offering detailed information about businesses , landmarks and geographical locations in worldwide . Web scraping using python is a powerful technique for extracting data from websites . Python is a popular choice for web scraping due to the availability of libraries like BeautifulSoup , Scrapy and Selenium .
In this blog post, we’ll explore how to scrape data from Google Maps using Python. Web scraping allows us to extract useful information from web pages, and in this case, we’ll gather data such as restaurant names, ratings, descriptions, addresses, contact numbers, and reviews from Google Maps.
The Python script uses Selenium, a powerful tool for automating web browsers, to navigate through Google Maps URLs and extract the desired information. This approach can be incredibly useful for various purposes, such as collecting business details, analyzing user reviews, or creating a database of local establishments.
Before proceeding, please note that web scraping may violate the terms of service of certain websites, so always ensure you comply with the site’s policies.
Required Modules Or Packages:
To scrape data from Google Maps, you need the following Python modules and packages:
Selenium:
A web automation tool that allows you to interact with web browsers programmatically.
- Installation:
pip install selenium
ChromeDriver:
A separate executable that Selenium requires to interface with the Chrome browser. Download it from here and make sure to specify the correct path in the script.
Google Chrome Browser:
The script requires Google Chrome to be installed on your computer to interact with the webpages.
How To Run The Code :
To run the Python script for scraping data from Google Maps, follow these steps:
Download ChromeDriver:
- Visit the ChromeDriver page and download the version that matches your installed version of Chrome.
- Extract the downloaded file and copy the path to the
chromedriver.exeexecutable. For example,C:/chromedriver_win32/chromedriver.exe.
Install Selenium:
- Open the command prompt and run the following command to install Selenium:
pip install selenium
Set Up the Python Script:
Copy the provided code into a Python file, for example,
google_maps_scraper.py.Make sure to replace the
servicepath in the script with the path to yourchromedriver.exefile:
service = Service("C:/chromedriver_win32/chromedriver.exe")
Run the Script:
- Open the command prompt or your preferred Python environment (like PyCharm or VS Code) and run the script by typing:
python google_maps_scraper.pyCode Explanation :
Here’s a breakdown of the main components of the Python script:
1. Setting Up Selenium WebDriver:
# Import necessary libraries from Selenium
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# Configure Selenium options for headless mode
options = webdriver.ChromeOptions()
options.add_argument('headless')
# Specify the path to the ChromeDriver executable
service = Service("C:/chromedriver_win32/chromedriver.exe")
# Initialize the WebDriver
browser = webdriver.Chrome(service=service, options=options)
This section sets up the Selenium WebDriver with Chrome, enabling headless mode (running the browser in the background without a graphical interface) to make the scraping process more efficient.
2. List of Google Maps URLs:
# List of Google Map URLs to scrape
url = [
"https://www.google.com/maps/place/Papa+John's+Pizza/@40.7936551,-74.0124687,17z/data=!3m1!4b1!4m5!3m4!1s0x89c2580eaa74451b:0x15d743e4f841e5ed!8m2!3d40.7936551!4d-74.0124687",
"https://www.google.com/maps/place/Lucky+Dhaba/@30.653792,76.8165233,17z/data=!3m1!4b1!4m5!3m4!1s0x390feb3e3de1a031:0x862036ab85567f75!8m2!3d30.653792!4d76.818712"
]
This list contains the Google Maps URLs of the places from which data will be scraped.
3. Loop Through URLs and Scrape Data:
# Loop through the URLs to scrape data
for i in range(len(url)):
browser.get(url[i]) # Open the Google Map URL
# Wait for the title to load and get it
title = WebDriverWait(browser, 10).until(
EC.presence_of_element_located((By.CLASS_NAME, "x3AX1-LfntMc-header-title-title"))
)
print(i + 1, "-", title.text)
# Scrape the rating stars
stars = WebDriverWait(browser, 10).until(
EC.presence_of_element_located((By.CLASS_NAME, "aMPvhf-fI6EEc-KVuj8d"))
)
print("The stars of the restaurant are:", stars.text)
# Scrape the description of the place
description = WebDriverWait(browser, 10).until(
EC.presence_of_element_located((By.CLASS_NAME, "uxOu9-sTGRBb-T3yXSc"))
)
print("Description:", description.text)
# Scrape the address of the place
address = WebDriverWait(browser, 10).until(
EC.presence_of_all_elements_located((By.CLASS_NAME, "CsEnBe"))
)[0]
print("Address:", address.text)
# Scrape the contact number of the place
phone = WebDriverWait(browser, 10).until(
EC.presence_of_all_elements_located((By.CLASS_NAME, "CsEnBe"))
)[-2]
print("Contact Number:", phone.text)
# Scrape the reviews of the place
reviews = WebDriverWait(browser, 10).until(
EC.presence_of_all_elements_located((By.CLASS_NAME, "OXD3gb"))
)
print("------------------------ Reviews --------------------")
for review in reviews:
print(review.text)
print("\n")
This loop iterates over each Google Maps URL and performs the following actions:
- Opens the URL in the browser.
- Waits for the page elements (like title, rating stars, description, address, contact number, and reviews) to load using
WebDriverWait. - Scrapes the required data from the webpage and prints it to the console.
4. Closing the Browser:
# Close the browser after scraping is complete
browser.quit()
This command closes the browser window once all the data has been scraped.
Source Code:
main.py
# Import the necessary libraries from Selenium
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# Make browser open in the background
options = webdriver.ChromeOptions()
options.add_argument('headless')
# Specify the path to the chromedriver executable
service = Service("C:/chromedriver_win32/chromedriver.exe")
# Create the webdriver object
browser = webdriver.Chrome(service=service, options=options)
# List of Google Map URLs
url = [
"https://www.google.com/maps/place/Papa+John's+Pizza/@40.7936551,-74.0124687,17z/data=!3m1!4b1!4m5!3m4!1s0x89c2580eaa74451b:0x15d743e4f841e5ed!8m2!3d40.7936551!4d-74.0124687",
"https://www.google.com/maps/place/Lucky+Dhaba/@30.653792,76.8165233,17z/data=!3m1!4b1!4m5!3m4!1s0x390feb3e3de1a031:0x862036ab85567f75!8m2!3d30.653792!4d76.818712"
]
# Loop through the URLs
for i in range(len(url)):
# Open the Google Map URL
browser.get(url[i])
# Wait for the page to load and then get the title
title = WebDriverWait(browser, 10).until(
EC.presence_of_element_located((By.CLASS_NAME, "x3AX1-LfntMc-header-title-title"))
)
print(i + 1, "-", title.text)
# Get the rating stars
stars = WebDriverWait(browser, 10).until(
EC.presence_of_element_located((By.CLASS_NAME, "aMPvhf-fI6EEc-KVuj8d"))
)
print("The stars of the restaurant are:", stars.text)
# Get the description of the place
description = WebDriverWait(browser, 10).until(
EC.presence_of_element_located((By.CLASS_NAME, "uxOu9-sTGRBb-T3yXSc"))
)
print("Description:", description.text)
# Get the address of the place
address = WebDriverWait(browser, 10).until(
EC.presence_of_all_elements_located((By.CLASS_NAME, "CsEnBe"))
)[0]
print("Address:", address.text)
# Get the contact number of the place
phone = address = WebDriverWait(browser, 10).until(
EC.presence_of_all_elements_located((By.CLASS_NAME, "CsEnBe"))
)[-2]
print("Contact Number:", phone.text)
# Get the reviews of the place
reviews = WebDriverWait(browser, 10).until(
EC.presence_of_all_elements_located((By.CLASS_NAME, "OXD3gb"))
)
print("------------------------ Reviews --------------------")
for review in reviews:
print(review.text)
print("\n")
# Close the browser
browser.quit()
Output :
Find More Projects
Build a Quiz Game Using HTML CSS and JavaScript Introduction Hello coders, you might have played various games, but were you aware …
Emoji Catcher Game Using HTML CSS and JavaScript Introduction Hello Coders, Welcome to another new blog. In this article we’ve made a …
Typing Challenge Using HTML CSS and JavaScript Introduction Hello friends, all you developer friends are welcome to our new project. If you …
Breakout Game Using HTML CSS and JavaScript With Source Code Introduction Hello friends, welcome to today’s new blog post. All of you …
Digital and Analog Clock using HTML CSS and JavaScript Introduction : This project is a digital clock and stopwatch system, which allows …
Coffee Shop Website using HTML, CSS & JavaScript Introduction : This project is a website for coffee house business. It uses HTML …
Get Huge Discounts
More Python Projects
