Skip to main content

​
Overview

The LinkedIn Scraper Agent Node uses an AI agent with tools that fetch LinkedIn data by URL (via Bright Data). The agent decides which tool to call based on your prompt. Capabilities:
  • Get profiles by URL — Retrieve profile data for one or more LinkedIn profile URLs (e.g. linkedin.com/in/username).
  • Get companies by URL — Retrieve company page data for one or more LinkedIn company URLs (e.g. linkedin.com/company/microsoft).
  • Get posts by URL — Retrieve post data for one or more LinkedIn post URLs.
  • Get posts by profile URL — Retrieve posts published by given profile URL(s), with optional start_date and end_date (input is JSON).
  • Get posts by company URL — Retrieve posts from given company page URL(s).
There is no keyword search: you (or upstream nodes) provide URLs; the agent fetches and returns the data for those URLs.
Finding people or companies by name? Use an Agent Swarm with (1) an Agent node that has a web search tool and (2) this LinkedIn Scraper Agent. The agent with web search can find LinkedIn profile or company URLs from the web, then hand off to this agent to fetch the LinkedIn profile or company data.

​
Configuration Parameters

To set up the LinkedIn Scraper Agent Node, configure the following parameters:

​
Node Configuration

  • Prompt: Describe what you want using LinkedIn URLs (profile, company, or post URLs). The agent will choose the right tool and fetch the data.
    Get profile data for https://www.linkedin.com/in/jane-doe and https://www.linkedin.com/in/john-smith
    Get company data for Microsoft and Google—here are the URLs: https://www.linkedin.com/company/microsoft, https://www.linkedin.com/company/google
    Get posts from https://www.linkedin.com/company/acme-corp
    Get posts from this profile between 2024-01-01 and 2024-06-01: https://www.linkedin.com/in/thought-leader
    If you have URLs from another node (e.g. a list of company URLs), reference them in the prompt with format strings (e.g. {company_urls}).
  • Tools: Select which tools the agent can use. All are URL-based:
    ToolDescription
    Get Profiles By UrlFetch profile data for comma-separated profile URLs
    Get Companies By UrlFetch company page data for comma-separated company URLs
    Get Posts By UrlFetch post data for comma-separated post URLs
    Get Posts By Profile UrlFetch posts by profile URL(s), with optional start/end date (JSON input)
    Get Posts By Company UrlFetch posts from company page URL(s)
    LinkedIn scraping is subject to rate limits and LinkedIn’s terms of service. Use responsibly.
  • System Prompt: Customize the agent’s LinkedIn scraping behavior.
  • Model: Select the language model for the agent (must support function calling).
  • Max Steps: Set the maximum number of steps the agent can take (1-20).
  • Enable Model Fallbacks: Allow automatic fallback to alternative models if the primary model fails.

​
Expected Inputs and Outputs

  • Inputs:
    • The node accepts input data that can be referenced within the prompt.
  • Outputs:
    • output: The final result from the agent’s LinkedIn scraping operations
    • conversation: Detailed log of the agent’s scraping activities

​
Use Case Examples

  1. Profile and company enrichment: You have a list of LinkedIn profile or company URLs (e.g. from a CRM or spreadsheet). Use the agent to fetch profile data or company page data for those URLs and feed the result into downstream nodes.
  2. Company and post monitoring: Provide company page URLs; the agent fetches company details and posts from those pages. Use for competitive intelligence or content aggregation when you already know which companies to track.
  3. Posts by profile or date range: You have profile URLs and optionally a date range. Use Get Posts By Profile Url (with JSON input for URLs and start/end dates) to retrieve their posts for analysis or reporting.

​
Error Handling and Troubleshooting

  • Rate Limits: LinkedIn scraping services have rate limits. The agent will manage requests to avoid hitting limits.
  • Profile Access: Some profiles may be private or have limited public information. The agent will indicate when data is unavailable.
  • Model Compatibility: The selected model must support function calling. Switch to a compatible model if you encounter errors.
If you encounter any issues not covered in this documentation, please reach out to our support team for assistance.

​
Relevant Nodes

Hunter Agent Node

Find and verify professional email addresses

Crunchbase Agent Node

Research company funding and business information

Reddit Scrape Agent Node

Scrape Reddit posts and comments