Let’s be honest: when you’re receiving a phone call or an email about a topic that is completely unrelated to your interests, you feel like it is wasting a non-renewable resource - your time. But receiving insightful information that will solve what you are trying to solve, improve your team’s performance, and actually give you a bit of time back is like a breath of fresh air.
And to achieve this with your prospect, one thing is paramount: research. You look up a profile, read their recent posts, search for company news, cross-reference with what you know about their values and tech stack, then stitch together a first message that references what is real. It might take hours, even days per person. Most of it is mechanical, and the quality depends entirely on how much time you actually have before the next meeting.
This article is about building a LinkedIn Prospecting Research MCP Engine that automates that loop for you - and about what that process looks like when IBM Bob is both the tool you are using to build and the agent that will eventually run the engine (MCP Server).
The architecture in one paragraph
The server is a Node.js MCP server written in TypeScript, using the @modelcontextprotocol/sdk package. It exposes seven tools over stdio: two for managing a seller context file, three for scraping LinkedIn (login, connections list, individual profile), one for web research via Brave Search API, and one that orchestrates all the above into a saved markdown prospect card. Playwright drives a persistent Chromium profile so the LinkedIn session survives between tool calls without requiring a fresh login every time. The whole thing registers with Bob as a local MCP server and from that point forward Bob can call any of the tools natively in conversation.
Starting with Bob as the builder
The practical experiment here was using Bob to build the server rather than building it and then connecting it. That distinction matters because the feedback loop is different. When you describe what you want in conversation and Bob scaffolds the code, you can test each tool immediately, see what breaks, and iterate in the same session rather than switching between editor, terminal and documentation. The build-mcp-server skill in Bob carries the structural knowledge - registerTool signatures, the inputSchema convention, how stdio transport connects - so the scaffolding phase was fast. The interesting work was in the details that no skill covers: what LinkedIn’s DOM actually looks like in headless Playwright in 2026.
The DOM problem
LinkedIn’s class names are fully obfuscated hashes. There are no semantic selectors like .profile-headline or .connection-name left in the markup. Every class is a short random string like _3b309dbd or b5287586 that changes with each deployment. The initial scraper used selectors from older documentation and returned empty strings across the board.
The fix required live DOM inspection: running a headless Playwright session, dumping the HTML of the connections page, and working out what stable attributes remained. The answer was componentkey - a custom attribute LinkedIn attaches to its component tree with meaningful names like ProfileVerificationTriggerRef-username, com.linkedin.sdui.profile.card.refXXXXAbout, and ConnectionsPage_ConnectionsList. These are stable enough to use as selectors, and the topcard section ([componentkey$='Topcard']) reliably contains name, headline, location, and company in a predictable <p> element structure regardless of the class names around them.
The connections list had a separate problem: it renders inside a fixed-height scrollable container, not the window. window.scrollBy does nothing. The correct trigger for lazy-loading more results is scrollIntoView on the last rendered card, which fires the intersection observer LinkedIn uses to load the next batch. Each call loads approximately ten more profiles.
Bob ran the debug scripts, read the DOM output, identified the patterns, and updated the selectors iteratively - the same investigation loop a developer would run, but without leaving the conversation.
Filtering with two-phase topcard verification
The connections scraper supports excludeCompanies and filterLocations parameters. The naïve implementation checked headline text only, which misses people whose employer is not mentioned in their headline - a common pattern for executives whose title gives no company signal.
The robust implementation uses two phases. Phase one is a fast substring match against the headline to drop obvious cases before any profile page loads. Phase two visits each candidate’s profile page and reads the topcard section text - which always contains the current employer and location - and applies both the exclusion and location filters there. This costs one page load per candidate but catches cases the headline alone cannot. Since the topcard fetch is also the source of the clean location field in the output, the extra load serves double duty.
The location extraction had its own subtlety: the topcard text is one long concatenated string with no clear delimiter between company name and city. The solution was DOM-based rather than text-parsing: the location lives in the <p> element immediately before the lone · separator element in the topcard, and querying for that structural position returns a clean "Sydney, New South Wales, Australia" string rather than requiring any regex against the surrounding noise.
Research and card generation
The research_prospect tool runs a set of structured queries against the Brave Search API - company news, technology signals, named-person mentions - and classifies the results into highlights, signals and articles. This is straightforward HTTP fetch work, though rate limits on the Brave API mean it is worth building in graceful degradation: if all queries fail, the prospect card still generates with whatever profile data was scraped, rather than throwing an error.
The generate_prospect_card tool takes the scraped profile, the research results, and the seller’s stored context (name, company, value proposition, target personas) and builds a structured markdown file. The card includes a persona summary table, a GTM brief with the research signals, a placeholder outreach draft, and conversation starters. Cards are written to a prospect-cards/ directory with a kebab-cased filename. The LLM that generates the narrative sections is Bob itself, called implicitly when the tool prompt asks it to synthesise the inputs.
What this pattern actually demonstrates
The interesting thing about this project is not the specific tools - LinkedIn scrapers exist, Brave Search wrappers exist, prospect card templates exist. The interesting thing is the build pattern: using an AI agent’s native MCP infrastructure to extend that same agent’s capabilities in real time, with the agent doing most of the investigative and implementation work.
The loop looks like this: describe a capability gap in conversation, Bob scaffolds the code, you test it through the same MCP interface Bob will eventually use, something breaks in a way the scaffold did not anticipate, Bob investigates the live environment (DOM inspection, debug scripts, API responses), updates the implementation, and the tool is ready. The context that makes this work is continuity - Bob knows the entire build history, all the failed selector attempts, the scroll behaviour discovery, the two-phase filter reasoning - because it was present for all of it.
That continuity is what makes the debugging fast. When a tool returns empty results, you do not have to re-explain the codebase; you describe the symptom and Bob already has the context to find the cause.
What it does not solve
There are things this approach does not fix. LinkedIn’s DOM will change again, and the componentkey attribute is not a public API contract - it is an implementation detail that could disappear in the next rewrite. The scraper will need updating. The research tool is only as good as the Brave Search index for the companies you care about, which is patchy for smaller firms. The prospect card template generates structure but the quality of the outreach draft depends on how much real signal was scraped - a sparse profile with no posts and no company news produces a card full of placeholders.
The right mental model for this kind of tool is a research assistant that handles the mechanical parts of the pre-call loop reliably, not an end-to-end automation that replaces judgement about what to actually say.
Running it yourself
The server runs as a local stdio MCP server registered in Bob’s configuration. It requires Node.js, Playwright with Chromium installed, and a Brave Search API key. The LinkedIn session is persisted in a local browser profile directory and survives restarts. First run requires a manual login through a visible browser window; subsequent runs are headless.
The full source is a single TypeScript project with five tool files, a browser context module, and a straightforward tsconfig targeting ESM output. Nothing about the architecture is specific to LinkedIn - the same pattern of a persistent browser context behind MCP tools works for any web platform that requires an authenticated session and does not expose a proper API.
IBM Bob built it with a single simple prompt
Hi mate. We will work in this directory today to try to build an awesome tool for LinkedIn Prospecting. I want to show the benefits of building an MCP with IBM Bob. Let's say, I want to build a Prospecting Crawler with IBM Bob, based on my LinkedIn Connections.
My LinkedIn profile: url_to_your_profile
I want to make this Prospecting tool universal for every users, so it should be aware about user's context. Prospecting use-cases: AI adoption for large Enterprises across your_location, Legacy technology modernisation (like Java 8)
Focus audience: C-Suite (CTO, CFO), Principal-level Influencers
Prospecting should be based on market trends, latest news posts (both from company's or individual's LinkedIn posts and general online news), and should help curating a personal go-to-market strategy that will benefit both the user of the Prospecting Tool and the prospecting target persona. Use your_preferred_spelling spelling.
Add following filters: by_company, by_location
Exclude personas from: your_own_company
Filter personas by: your_location
Ask me required questions.