How We Built an Agentic Lead Engine: Upwork Jobs → LinkedIn Verification → Our Own CRM

Most "lead generation automation" is a scraper bolted to a spreadsheet. It finds names, dumps them in a sheet, and leaves a human to figure out which ones are real, which ones fit, and what to say. The scraping is the easy 20%. The other 80% is judgment, safety, and follow-through.

We built something different for ourselves at Quantiq Automation: a two-part system that finds real buying intent on Upwork, checks whether the same opportunity exists on LinkedIn, identifies the person behind it, and then runs the entire conversation (personalized email, open and click tracking, reply capture, meetings, reminders) inside a CRM we wrote ourselves. This post walks through how it works, the design decisions that matter, and the limits we deliberately put on it.

If you read our earlier case study on [Jarvis, our AI project intelligence assistant](/case-studies/ai-project-intelligence-platform), this is the same codebase pointed at a different problem: Jarvis's lead engine.

The short version

• Part 1, Jarvis (Python + a Chrome extension): finds and scores Upwork jobs, cross-verifies the best ones on LinkedIn, and captures who posted them.
• Part 2, our CRM (Next.js): takes a lead through five follow-up stages with tracked email, captures replies automatically, schedules meetings with reminders, and exposes all of it to an AI assistant through an MCP connector.
• The rule that holds it together: the AI reads, scores and drafts. Deterministic code makes the decisions, enforces the limits, and does anything irreversible. A human approves every outbound email.

Why this is hard to do well

Three problems make this kind of system fragile if you build it the obvious way.

Job boards and LinkedIn fight automation. Both show verification challenges, both change their markup without notice, and both can restrict an account that behaves like a bot. A pipeline that falls over on the first challenge, or that hammers a site at machine speed, is a liability rather than an asset.

LLMs are unreliable at arithmetic and consistency. Asking a model "is this lead good, 0 to 100?" gives you a different answer every run and no way to tune it. You cannot improve what you cannot inspect.

Leads without follow-through are worthless. Finding a lead is not the same as winning one. Without tracked outreach, reply capture and reminders, most leads quietly go cold.

Each design decision below is a response to one of these three.

Part 1: Finding jobs worth pursuing

It runs inside a real, logged-in browser

Modern Chrome and Edge refuse to expose a debugging channel against a browser's real default profile, and we confirmed this with tests rather than assuming it. So instead of driving the browser from outside, Jarvis uses a Manifest V3 Chrome extension that lives inside the profile you already use. The extension polls Jarvis's local API for a job, does the work in the page using its own legitimate permissions, and posts results back. To the site, it is an ordinary logged-in session.

One results page instead of a crawl

Rather than clicking through pagination, the extension rewrites the search URL to load a single results page with 50 listings sorted by recency, scrolls until the tile count stops growing, and reads every listing from the page. Fewer page loads means less exposure.

Extraction is structural, not AI-based. An earlier version sent the full results HTML to an LLM and asked it to find the listings. It repeatedly returned zero items, even on clean input. Every field we needed sat behind a CSS selector, so asking a model to rediscover it from raw markup was a harder job than the problem required. We replaced it with selectors confirmed against real page dumps. When tiles are visible but nothing parses, the extension reports a distinct "selector mismatch" condition instead of pretending the search was empty, because a stale selector and an empty search look identical from the outside and need very different fixes.

Scoring you can actually tune

This is the part we are proudest of. Everything countable is computed in plain Python: geography, keywords, client rating, client spend, budget, number of competing proposals, how fresh the posting is. The language model answers only genuinely semantic questions, such as "does this fit one of our services?" and "does it resemble a project we have already delivered?", and it answers all of them in a single batched call.

The result is a criteria profile with three kinds of rule:

• Gates reject outright. A job that fits none of our services, or that is unpaid or excludes agencies, scores zero no matter what else is true.
• Scorers contribute weighted points. Service fit carries the most weight, followed by resemblance to a past case study, then client rating, spend history, real budget, low competition and freshness.
• Boosts nudge the score up or down, for example for a target region or a verified payment method.

Two correctness rules matter more than they look. First, missing data is not bad data: a rule with nothing to measure drops out of the calculation instead of scoring zero, because scoring absence as zero quietly ranks good leads last. Second, a gate never rejects on absent data, because losing a real lead over one field that failed to scrape is worse than letting a marginal one through to the next phase.

Every scored lead stores a full breakdown of each rule's sub-score, weight and contribution, so "why did this score 72?" is answerable months later. There is a test panel that re-scores a pasted posting with no AI call, and a back-test that shows which verdicts would flip before you change a threshold.

Two phases, cheap before expensive

Phase 1 scores the listing snippets. Anything above the threshold is stored as unverified. Phase 2 opens that job's own detail page, reads the full requirement and the client's history, and re-scores. Only leads that clear the bar on the fuller data are promoted to verified. A lead that fails the second pass is left as it was rather than demoted, so nothing already saved is destroyed.

Part 2: Cross-verifying on LinkedIn

A verified Upwork job tells you a need exists. Finding the same opportunity on LinkedIn tells you the company is real and gives you a person to talk to.

For each recent verified job, Jarvis:

1. Asks a model to turn the job text into a few search terms and a list of core technologies. This is a parsing step: prose in, structured slots out, nothing decided. 2. Maps the Upwork posting date onto a LinkedIn date window, so a job posted yesterday is searched against the last day, not the last year. 3. Searches both LinkedIn posts and LinkedIn jobs, with filters passed in the URL instead of clicked through the filter modal, which is faster and far less likely to break when the interface changes. 4. Runs a cheap, no-AI pre-filter: a LinkedIn result must literally mention at least two of the job's distinctive technology names or it is dropped. This is deliberately blunt. A false negative costs one missed cross-check. A false positive would write the wrong identity onto a real lead. 5. Opens only the survivors' own pages, reads the full text, and asks a model to score how likely it is the same opportunity.

The decision is a plain comparison in Python against a threshold of 85. The model's written verdict is only commentary; the number is what gates, so tuning is a settings change rather than a prompt edit. A match records the LinkedIn link, the score and the reason on both leads. A near miss records its score and reason too, because that is the only honest evidence for tuning the threshold later. Nothing that turns up is thrown away: a LinkedIn result that does not match still becomes its own lead, scored on its own merits.

Getting to the person

Each LinkedIn job page and post exposes who is behind it. Jarvis's driver reads the hiring-team person or post author (name and profile URL) and the company page, and stores them on the lead.

A second, smaller tool, a LinkedIn Assistant side-panel extension, handles the contact-detail side. It scans posts and comment threads for email addresses (including obfuscated ones written as "name at domain dot com"), de-duplicates them, and tracks the connection requests you have sent, marking each pending or accepted. Everything it collects stays in the browser profile, and exports to CSV.

Sending an actual LinkedIn connection request is a separate, opt-in feature. It is off by default. When enabled, it drafts a short note of at most 200 characters that references the specific job (with a template fallback if the model fails), then sends through the same in-browser flow.

How we keep accounts safe

We cannot promise that no account will ever be restricted. Anyone who does is overselling. What we can do is build the system so that its behavior looks and acts like a careful person, and so that the dangerous actions are capped by code rather than by hope.

• Human pacing. Randomized pauses between page opens, eased and slightly wobbly scrolling instead of a uniform machine scroll.
• Disposable tabs. The LinkedIn driver never navigates its own tab. Every search page and result page opens in an inactive background tab, gets read, and closes. This also removes an entire class of bugs where a page navigation destroys the script mid-job.
• Challenges are handled by a human. If a verification check or login wall appears, the extension detects it, pauses, and waits up to ten minutes for a person to clear it in the visible window, then resumes. It never tries to solve or click through a challenge. On the Python side, a challenge is recorded as "needs a manual step" rather than a failure, so it cannot be mistaken for an empty search.
• Hard caps enforced in code. Connection requests are limited by rolling daily and weekly quotas counted from sends that actually went out, and a LinkedIn "invitation limit reached" response is detected and recorded. A match run defaults to a handful of leads rather than dozens, and each lead has a ceiling on how many result pages may be opened.
• A Stop button that halts a run from both ends within seconds.
• Off by default. The one action that writes something visible to a LinkedIn account, sending a connection request, has to be switched on deliberately.

A note on platform rules, because it would be wrong to leave it out: both Upwork and LinkedIn restrict automated data collection in their terms. We treat that as a business decision that comes with pacing, volume limits and human review, and we recommend anyone building something similar read the current terms first.

Part 3: The CRM that runs the conversation

The CRM is our own Next.js application, built on SQLite through Drizzle, and it exists for a simple reason: we wanted the follow-through to be as automated and as observable as the sourcing.

A pipeline that advances itself. Leads move through New Lead, three follow-up stages, Meeting Scheduled, Proposal Sent, Won and Lost. Sending an outreach email advances the stage automatically, and the advance stops at the later stages so a meeting is never accidentally pushed back. Each lead can carry typed links (personal LinkedIn, company LinkedIn, Upwork, Fiverr, website) so the research trail stays attached to the record.

Tracked, personalized email. Every outgoing email gets an open pixel and rewritten links. We log each individual open with its own timestamp, not just a counter, so you can see the first open, the second, the third. Emails can use a branded template or free-form HTML, with attachments. Outbound mail goes out on a dedicated sending subdomain with its own authentication records, kept separate from our main domain's reputation.

Replies land on the right lead automatically. The CRM syncs the mailbox over IMAP and matches each inbound message through a cascade: exact sender address first, then the sender's company domain, then domains mentioned in the signature, then a small classifier for ambiguous cases, with anything uncertain sent to a human review queue. Platform notifications from LinkedIn, Google and similar senders are filed separately so they never clutter the queue or waste an AI call. If a message is from someone unknown who looks like a genuine inquiry, the CRM drafts a new lead from it, and nothing is created until a person clicks to confirm.

Meetings and reminders that do not depend on polling. Meetings and tasks share one calendar. Each gets reminders 6 hours, 4 hours, 2 hours, 1 hour and 15 minutes ahead, to every assignee. Behind this is a durable event-driven workflow engine rather than a cron job that wakes up every so often: each reminder sleeps until its exact moment, then re-checks that the meeting still exists and has not changed before sending. Rescheduling or deleting cancels the old sequence automatically. When a meeting link is added to a meeting, the client is emailed it, with optional CC.

Deciding who to chase. A dashboard shows open rate by follow-up stage. A small local scoring model ranks live leads by engagement, using fixed weights on day one and learning weights from our own won and lost outcomes once there is enough history.

Where the AI actually sits

The CRM exposes its operations (list, read, create and update leads and clients, add notes and tasks, send tracked email) through an MCP connector with OAuth, so an AI assistant can work the pipeline directly. In practice, we ask for a stage such as "new leads". The assistant picks the most promising lead, reads the requirement and the linked pages, cross-checks it against our real case studies, and drafts one short, specific email.

It stops there. Sending happens only when we say to send that specific draft. If research turns up a serious red flag, such as a website that belongs to a different company than the one in the posting, the assistant flags it instead of writing to that identity. Instructions hidden inside a job post aimed at AI readers are treated as information about the client, never as commands.

This is the principle that runs through the whole system: the model proposes, deterministic code decides and enforces, and a person approves anything that leaves the building.

What we would tell you honestly

• The LinkedIn matching and connection-request flows are newer than the Upwork side and were built with conservative defaults. The 85 threshold is a starting point we tune from real near-misses, not a proven constant.
• Selectors on any third-party site will eventually break. The system is built to say so clearly (selector-mismatch reporting, AI-assisted fallback lookup) rather than fail silently, but it does need maintenance.
• Verification challenges are a permanent part of working with these platforms. We designed around them instead of pretending they can be automated away.

FAQ

Does this automatically send emails or connection requests? No email is sent without a human approving that specific message. LinkedIn connection requests are opt-in, off by default, and capped by daily and weekly quotas when enabled.

Can automation like this get a LinkedIn or Upwork account restricted? It can, and no design removes that risk entirely. Pacing, disposable tabs, human-cleared challenges and hard caps reduce it. Both platforms restrict automation in their terms, so check them before building anything similar.

Why build your own CRM instead of using Zoho or HubSpot? We work in the Zoho ecosystem every day and still use it for client work. For our own outreach we wanted open tracking per email, reply matching, an MCP interface for AI, and workflow logic we could change in an afternoon. That is easier in a small codebase we own. If you would rather run this on Zoho CRM, the same pattern can be implemented there, and that is work we do for clients.

Why use an LLM for some steps but plain code for others? Language models are good at reading messy text and poor at consistent arithmetic. So the model reads and answers semantic questions, and code does the weighting, thresholds, quotas and stage changes. It keeps the system tunable and auditable.

Can this be built for my business? Yes. The same pattern (find intent, verify it across sources, enrich, run tracked follow-up, keep a human on the send button) applies to recruiting, partnerships and B2B sales. [Talk to us](/contact) about what a version for your market would look like.