> ## Documentation Index
> Fetch the complete documentation index at: https://docs.automindz.co/llms.txt
> Use this file to discover all available pages before exploring further.
# Scrape career pages
> Discover and extract job listings from a company's career page
Triggers a career page scrape for a given company URL. The scraper maps the company's website, identifies career pages, extracts job listings, and returns structured job posting data.
The pipeline runs asynchronously. Use the returned `run_id` to [poll for results](/api-reference/endpoint/get-run).
`target_titles` and `target_titles_prompt` are two ways to express the same title filter and are **mutually exclusive** — sending both returns `422`. Use `target_titles` for a fixed list, or `target_titles_prompt` for open-ended roles.
**Timing guidance:** Most scrapes complete within 2-5 minutes. Companies with many pages or ATS-hosted job boards may take longer. Poll every 15-30 seconds.
## How it works
1. Maps the company website to discover URLs
2. Identifies career/jobs pages using pattern matching and AI filtering
3. Scrapes each career page and extracts job listings
4. Applies optional title/location filters (pass `target_titles_prompt` to drive AI title filtering for open-ended role sets, e.g. "any physician role")
5. Analyzes each job posting in detail (salary, requirements, hiring manager, etc.)
If the company uses an external ATS (Greenhouse, Lever, Workday, Ashby, etc.), the scraper follows the redirect and paginates through all listings automatically.
## Example responses
```json theme={null}
{
"request_id": "run_abc123xyz",
"status": "COMPLETED",
"result": {
"company": "Acme Corp",
"url": "https://acme.com",
"career_page_urls": ["https://jobs.lever.co/acme"],
"pages_scraped": 3,
"jobs_returned": 1,
"jobs_total": 1,
"jobs_filtered_out": 0,
"limit": 25,
"offset": 0,
"has_more": false,
"jobs": [
{
"job_title": "Senior Software Engineer",
"location": "London, UK",
"city": "London",
"country": "United Kingdom",
"job_type": "full-time",
"remote_policy": "hybrid",
"department": "Engineering",
"salary_min": 90000,
"salary_max": 120000,
"salary_currency": "GBP",
"salary_period": "annual",
"min_years_of_experience": 5,
"requirements": ["5+ years TypeScript", "React", "Node.js"],
"job_url": "https://jobs.lever.co/acme/abc-123",
"ats_system": "lever",
"posted_at": "2026-05-01"
}
]
},
"started_at": "2026-05-15T10:00:00Z",
"finished_at": "2026-05-15T10:03:42Z"
}
```
```json theme={null}
{
"request_id": "run_def456uvw",
"status": "COMPLETED",
"result": {
"company": "acme.com",
"url": "https://acme.com",
"pages_scraped": 8,
"jobs_returned": 0,
"jobs_total": 0,
"jobs_filtered_out": 0,
"limit": 25,
"offset": 0,
"has_more": false,
"jobs": []
},
"started_at": "2026-05-15T10:00:00Z",
"finished_at": "2026-05-15T10:04:10Z"
}
```
```json theme={null}
{
"run_id": "run_ghi789rst",
"status": "QUEUED",
"result": null,
"started_at": null,
"finished_at": null
}
```
## OpenAPI
````yaml openapi.json POST /v1/scrape-career-pages/async
openapi: 3.1.0
info:
title: Automindz API
version: 0.1.0
servers:
- url: https://api.automindz.co
description: Production
security: []
paths:
/v1/scrape-career-pages/async:
post:
tags:
- scrape-career-pages
summary: Scrape Career Pages
operationId: scrape_career_pages_v1_scrape_career_pages_async_post
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ScrapeJobsRequest'
example:
company_url: inriver.com
company_name: inRiver
max_pages: 10
target_titles_prompt: >-
Only keep Principal Investigator, any 'Investigator'/'PI' role,
any physician/clinician role, Medical Director, Physician
Investigator, Clinical Research Physician. Reject everything
else.
target_locations:
- Finland
- Sweden
- Denmark
- Norway
- Belgium
- Netherlands
known_career_page_url: https://career.inriver.com
required: true
responses:
'202':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/ScrapeJobsResponse'
example:
request_id: run_abc123xyz
status: QUEUED
'422':
description: Validation Error
content:
application/json:
schema:
$ref: '#/components/schemas/HTTPValidationError'
security:
- HTTPBearer: []
components:
schemas:
ScrapeJobsRequest:
properties:
company_url:
type: string
title: Company Url
company_name:
anyOf:
- type: string
- type: 'null'
title: Company Name
max_pages:
anyOf:
- type: integer
- type: 'null'
title: Max Pages
default: 10
target_titles:
anyOf:
- items:
type: string
type: array
- type: 'null'
title: Target Titles
description: >-
Explicit list of target job titles. Mutually exclusive with
target_titles_prompt — passing both returns HTTP 422.
target_titles_prompt:
anyOf:
- type: string
- type: 'null'
title: Target Titles Prompt
description: >-
Natural-language instruction telling the AI filter which job titles
to keep and which to reject. Use instead of target_titles when the
target roles are open-ended (e.g. 'any physician role'). When set,
the title keyword pre-filter is bypassed and the LLM decides per the
prompt. Mutually exclusive with target_titles — passing both returns
HTTP 422.
target_locations:
anyOf:
- items:
type: string
type: array
- type: 'null'
title: Target Locations
filters:
anyOf:
- $ref: '#/components/schemas/ScrapeFilters'
- type: 'null'
known_career_page_url:
anyOf:
- type: string
- type: 'null'
title: Known Career Page Url
known_career_page_urls:
anyOf:
- items:
type: string
type: array
- type: 'null'
title: Known Career Page Urls
type: object
required:
- company_url
title: ScrapeJobsRequest
ScrapeJobsResponse:
properties:
request_id:
type: string
title: Request Id
status:
type: string
title: Status
default: QUEUED
type: object
required:
- request_id
title: ScrapeJobsResponse
HTTPValidationError:
properties:
detail:
items:
$ref: '#/components/schemas/ValidationError'
type: array
title: Detail
type: object
title: HTTPValidationError
ScrapeFilters:
properties:
titles:
anyOf:
- items:
type: string
type: array
- type: 'null'
title: Titles
locations:
anyOf:
- items:
type: string
type: array
- type: 'null'
title: Locations
job_types:
anyOf:
- items:
type: string
type: array
- type: 'null'
title: Job Types
remote_only:
anyOf:
- type: boolean
- type: 'null'
title: Remote Only
type: object
title: ScrapeFilters
ValidationError:
properties:
loc:
items:
anyOf:
- type: string
- type: integer
type: array
title: Location
msg:
type: string
title: Message
type:
type: string
title: Error Type
type: object
required:
- loc
- msg
- type
title: ValidationError
securitySchemes:
HTTPBearer:
type: http
scheme: bearer
````