Scraping Data From Google Maps Python

Scraping Data From Google Maps Using Python With Source Code

Introduction :

In this article, we are going to see how to scrap all data from Google Maps like Addresses , Ratings , Descriptions , Contact Numbers , Reviews etc . using Python. Scraping Data from Google Maps Becomes a Popular Technique to gathering informations about the location or Places . It is used in various applications , like market analysis , competitor research or location-based sevices . Google Maps is offering detailed information about businesses , landmarks and geographical locations in worldwide . Web scraping using python is a powerful technique for extracting data from websites . Python is a popular choice for web scraping due to the availability of libraries like BeautifulSoup , Scrapy and Selenium .

In this blog post, we’ll explore how to scrape data from Google Maps using Python. Web scraping allows us to extract useful information from web pages, and in this case, we’ll gather data such as restaurant names, ratings, descriptions, addresses, contact numbers, and reviews from Google Maps.

The Python script uses Selenium, a powerful tool for automating web browsers, to navigate through Google Maps URLs and extract the desired information. This approach can be incredibly useful for various purposes, such as collecting business details, analyzing user reviews, or creating a database of local establishments.

Before proceeding, please note that web scraping may violate the terms of service of certain websites, so always ensure you comply with the site’s policies.

Required Modules Or Packages:

To scrape data from Google Maps, you need the following Python modules and packages:

Selenium:

A web automation tool that allows you to interact with web browsers programmatically.

Installation:

pip install selenium

ChromeDriver:

A separate executable that Selenium requires to interface with the Chrome browser. Download it from here and make sure to specify the correct path in the script.

Google Chrome Browser:

The script requires Google Chrome to be installed on your computer to interact with the webpages.

How To Run The Code :

To run the Python script for scraping data from Google Maps, follow these steps:

Download ChromeDriver:

- Visit the ChromeDriver page and download the version that matches your installed version of Chrome.
- Extract the downloaded file and copy the path to the chromedriver.exe executable. For example, C:/chromedriver_win32/chromedriver.exe.

Install Selenium:

- Open the command prompt and run the following command to install Selenium:

pip install selenium

Set Up the Python Script:

Copy the provided code into a Python file, for example, google_maps_scraper.py.
Make sure to replace the service path in the script with the path to your chromedriver.exe file:

service = Service("C:/chromedriver_win32/chromedriver.exe")

Run the Script:

Open the command prompt or your preferred Python environment (like PyCharm or VS Code) and run the script by typing:

python google_maps_scraper.py

Code Explanation :

Here’s a breakdown of the main components of the Python script:

1. Setting Up Selenium WebDriver:

				
					# Import necessary libraries from Selenium
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# Configure Selenium options for headless mode
options = webdriver.ChromeOptions()
options.add_argument('headless')

# Specify the path to the ChromeDriver executable
service = Service("C:/chromedriver_win32/chromedriver.exe")

# Initialize the WebDriver
browser = webdriver.Chrome(service=service, options=options)

This section sets up the Selenium WebDriver with Chrome, enabling headless mode (running the browser in the background without a graphical interface) to make the scraping process more efficient.

2. List of Google Maps URLs:

				
					# List of Google Map URLs to scrape
url = [
    "https://www.google.com/maps/place/Papa+John's+Pizza/@40.7936551,-74.0124687,17z/data=!3m1!4b1!4m5!3m4!1s0x89c2580eaa74451b:0x15d743e4f841e5ed!8m2!3d40.7936551!4d-74.0124687",
    "https://www.google.com/maps/place/Lucky+Dhaba/@30.653792,76.8165233,17z/data=!3m1!4b1!4m5!3m4!1s0x390feb3e3de1a031:0x862036ab85567f75!8m2!3d30.653792!4d76.818712"
]

This list contains the Google Maps URLs of the places from which data will be scraped.

3. Loop Through URLs and Scrape Data:

				
					# Loop through the URLs to scrape data
for i in range(len(url)):
    browser.get(url[i])  # Open the Google Map URL
    
    # Wait for the title to load and get it
    title = WebDriverWait(browser, 10).until(
        EC.presence_of_element_located((By.CLASS_NAME, "x3AX1-LfntMc-header-title-title"))
    )
    print(i + 1, "-", title.text)
    
    # Scrape the rating stars
    stars = WebDriverWait(browser, 10).until(
        EC.presence_of_element_located((By.CLASS_NAME, "aMPvhf-fI6EEc-KVuj8d"))
    )
    print("The stars of the restaurant are:", stars.text)

    # Scrape the description of the place
    description = WebDriverWait(browser, 10).until(
        EC.presence_of_element_located((By.CLASS_NAME, "uxOu9-sTGRBb-T3yXSc"))
    )
    print("Description:", description.text)

    # Scrape the address of the place
    address = WebDriverWait(browser, 10).until(
        EC.presence_of_all_elements_located((By.CLASS_NAME, "CsEnBe"))
    )[0]
    print("Address:", address.text)

    # Scrape the contact number of the place
    phone = WebDriverWait(browser, 10).until(
        EC.presence_of_all_elements_located((By.CLASS_NAME, "CsEnBe"))
    )[-2]
    print("Contact Number:", phone.text)

    # Scrape the reviews of the place
    reviews = WebDriverWait(browser, 10).until(
        EC.presence_of_all_elements_located((By.CLASS_NAME, "OXD3gb"))
    )
    print("------------------------ Reviews --------------------")
    for review in reviews:
        print(review.text)
    print("\n")

This loop iterates over each Google Maps URL and performs the following actions:

Opens the URL in the browser.
Waits for the page elements (like title, rating stars, description, address, contact number, and reviews) to load using WebDriverWait.
Scrapes the required data from the webpage and prints it to the console.

4. Closing the Browser:

				
					# Close the browser after scraping is complete
browser.quit()

This command closes the browser window once all the data has been scraped.

Source Code:

Your download is starting now...

main.py

				
					# Import the necessary libraries from Selenium
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# Make browser open in the background
options = webdriver.ChromeOptions()
options.add_argument('headless')

# Specify the path to the chromedriver executable
service = Service("C:/chromedriver_win32/chromedriver.exe")

# Create the webdriver object
browser = webdriver.Chrome(service=service, options=options)

# List of Google Map URLs
url = [
    "https://www.google.com/maps/place/Papa+John's+Pizza/@40.7936551,-74.0124687,17z/data=!3m1!4b1!4m5!3m4!1s0x89c2580eaa74451b:0x15d743e4f841e5ed!8m2!3d40.7936551!4d-74.0124687",
    "https://www.google.com/maps/place/Lucky+Dhaba/@30.653792,76.8165233,17z/data=!3m1!4b1!4m5!3m4!1s0x390feb3e3de1a031:0x862036ab85567f75!8m2!3d30.653792!4d76.818712"
]

# Loop through the URLs
for i in range(len(url)):

    # Open the Google Map URL
    browser.get(url[i])

    # Wait for the page to load and then get the title
    title = WebDriverWait(browser, 10).until(
        EC.presence_of_element_located((By.CLASS_NAME, "x3AX1-LfntMc-header-title-title"))
    )
    print(i + 1, "-", title.text)

    # Get the rating stars
    stars = WebDriverWait(browser, 10).until(
        EC.presence_of_element_located((By.CLASS_NAME, "aMPvhf-fI6EEc-KVuj8d"))
    )
    print("The stars of the restaurant are:", stars.text)

    # Get the description of the place
    description = WebDriverWait(browser, 10).until(
        EC.presence_of_element_located((By.CLASS_NAME, "uxOu9-sTGRBb-T3yXSc"))
    )
    print("Description:", description.text)

    # Get the address of the place
    address = WebDriverWait(browser, 10).until(
        EC.presence_of_all_elements_located((By.CLASS_NAME, "CsEnBe"))
    )[0]
    print("Address:", address.text)

    # Get the contact number of the place
    phone = address = WebDriverWait(browser, 10).until(
        EC.presence_of_all_elements_located((By.CLASS_NAME, "CsEnBe"))
    )[-2]
    print("Contact Number:", phone.text)

    # Get the reviews of the place
    reviews = WebDriverWait(browser, 10).until(
        EC.presence_of_all_elements_located((By.CLASS_NAME, "OXD3gb"))
    )
    print("------------------------ Reviews --------------------")
    for review in reviews:
        print(review.text)
    print("\n")

# Close the browser
browser.quit()

Output :

Find More Projects

Drawing Ganesha Using Python Turtle Graphics [Drawing Ganapati Using Python]

Drawing Ganesha Using Python Turtle Graphics [Drawing Ganpati Using Python]

Yadnyesh (curious_coder)

•

September 7, 2024

•

projects, Python Projects

Drawing Ganesha Using Python Turtle Graphics[Drawing Ganapati Using Python] Introduction In this blog post, we will learn how to draw Lord Ganesha …

Contact Management System In PYTHON with complete source code

Yadnyesh (curious_coder)

•

September 5, 2024

•

projects, Python Projects

Contact Management System in Python with a Graphical User Interface (GUI) Introduction: The Contact Management System is a Python-based application designed to …