Google Trends Agent

A robust web scraping tool that retrieves top trending keywords from Google Trends across multiple categories and 15 countries, complete with related article links for each keyword.

Overview

This agent automates the process of monitoring trending topics across different regions and categories, making it perfect for content creators, marketers, and researchers who need to stay on top of current trends.

Key Features

Supported Categories

The agent can scrape trending searches from:

• Business and Finance - Market trends, company news, economic indicators
• Politics - Political events, elections, policy changes
• Newspapers - Breaking news and media coverage
• Sports News - Sports events, player news, game results

Technical Features

Dynamic Content Handling

Uses Selenium WebDriver to handle JavaScript-rendered content, ensuring reliable data extraction even from complex, dynamic pages.

Fallback Mechanisms

Implements robust fallback strategies to maintain reliability:

• Alternative scraping methods
• Cached data options
• Error recovery procedures

Multiple Output Formats

{
  "timestamp": "2025-03-03 04:45:12",
  "region": "US",
  "category": "Business & Finance",
  "keywords": [
    {
      "keyword": "xrp price",
      "search_volume": "200K+",
      "articles": [
        "https://edition.cnn.com/2025/03/02/business/...",
        "https://finance.yahoo.com/video/crypto-prices-...",
        "https://www.cnbc.com/2025/03/02/trump-announces-..."
      ]
    }
  ]
}

# Google Trends Report
**Date:** 2025-03-03 04:45:12
**Region:** US
**Category:** Business & Finance

## xrp price (200K+ searches)
- [CNN Article](https://edition.cnn.com/...)
- [Yahoo Finance](https://finance.yahoo.com/...)
- [CNBC Report](https://www.cnbc.com/...)

Technical Stack

• Python - Core programming language
• BeautifulSoup4 - HTML parsing and data extraction
• Requests - HTTP library for web requests
• Selenium WebDriver - Dynamic content handling
• ChromeDriver - Browser automation (must match Chrome version)

Command Line Interface

Basic Usage

python run_agent.py --region US --category business --output results.json

Available Options

Examples

# Scrape US business trends
python run_agent.py --region US --category business

# Get UK politics trends with verbose output

python run_agent.py --region UK --category politics --verbose

# Save German sports trends to file

python run_agent.py --region DE --category sports --output de_sports.json

# Use fallback data for testing

python run_agent.py --use-fallback --output test_data.json

Output Example

Here's a real-world example of the agent's output:

{
  "timestamp": "2025-03-03 04:45:12",
  "region": "US",
  "category": "Business & Finance",
  "keywords": [
    {
      "keyword": "xrp price",
      "search_volume": "200K+",
      "articles": [
        "https://edition.cnn.com/2025/03/02/business/trump-cryptocurrency-market-spike/index.html",
        "https://finance.yahoo.com/video/crypto-prices-jump-trump-sets-025315985.html",
        "https://www.cnbc.com/2025/03/02/trump-announces-strategic-crypto-reserve-including-bitcoin-solana-xrp-and-more.html"
      ]
    },
    {
      "keyword": "nasa moon landing blue ghost",
      "search_volume": "100K+",
      "articles": [
        "https://www.nasa.gov/missions/artemis/blue-ghost-moon-landing/",
        "https://www.space.com/blue-ghost-moon-lander-launch-date",
        "https://www.scientificamerican.com/article/nasa-blue-ghost-moon-mission/"
      ]
    },
    {
      "keyword": "anne wojcicki",
      "search_volume": "20K+",
      "articles": [
        "https://www.forbes.com/profile/anne-wojcicki/",
        "https://www.businessinsider.com/23andme-ceo-anne-wojcicki-profile",
        "https://www.cnbc.com/2025/03/01/23andme-ceo-anne-wojcicki-announces-new-health-initiative.html"
      ]
    }
  ]
}

Use Cases

Content Marketing

• Identify trending topics for blog posts
• Discover viral content opportunities
• Track competitor mentions

Market Research

• Monitor industry trends
• Track brand mentions
• Analyze consumer interest patterns

News Monitoring

• Stay updated on breaking news
• Track story development
• Find source articles quickly

SEO & Analytics

• Discover high-volume keywords
• Track search trend changes
• Identify content gaps

Installation

Prerequisites

# Install Python dependencies
pip install requests beautifulsoup4 selenium

# Download ChromeDriver (must match your Chrome version)
# Visit: https://chromedriver.chromium.org/downloads

Setup

1. Clone the repository
2. Install dependencies: pip install -r requirements.txt
3. Download and configure ChromeDriver
4. Run the agent: python run_agent.py

Configuration

Supported Regions

US, UK, CA, AU, DE, FR, IT, ES, BR, MX, IN, JP, KR, CN, RU

Rate Limiting

The agent implements respectful rate limiting to avoid overwhelming Google's servers:

• Configurable delay between requests
• Automatic retry with exponential backoff
• Request throttling

Future Enhancements

• Real-time trend monitoring
• Email/webhook notifications
• Historical trend analysis
• Sentiment analysis integration
• Custom category support
• API endpoint wrapper
• Dashboard visualization

Links

• GitHub Repository
• Documentation
• Report Issues