Pricemancer

How product URL discovery works

Pricemancer probes each site and runs every discovery method that works for it — sitemaps, category pages, site search and store-platform APIs — merged into one product URL list.

Select a tracked site and click Find URLs. Pricemancer probes the site and runs every discovery method that works for it, then merges the results into a single list of product pages.

  • Sitemaps — the fastest route when a site publishes a complete one.
  • Store-platform APIs — shops built on common platforms expose their whole catalog through a bulk endpoint, which returns thousands of products in seconds.
  • Category pages — walking the navigation tree, page by page.
  • Site search — used to reach products the navigation does not link directly.

Methods are not tried one at a time until something works; the ones that suit the site all run and their results are merged and de-duplicated. A sitemap that covers 80% of a catalog plus a category walk that covers a different 70% together give better coverage than either alone.

Depending on the size of the site this takes from a minute to an hour or more. Progress is shown live and the run continues even if you leave the page.

Re-run discovery every few weeks, or after a site redesign, to pick up new products. Already-known URLs are kept and only new ones are added.

← All docs topics

Try it on your own catalog

Pricemancer is free: every feature, unlimited usage and up to 3 tracked competitor sites.

Start free