About the WebToAPI crawler

Dernière mise à jour le 26 août 2026

Ce document n'est pas encore disponible dans votre langue ; la version anglaise est affichée. La traduction est en cours.

This page is linked from our user agent so site owners can find out who we are.

Who we are

WebToAPI turns user-selected elements of a web page into a JSON endpoint. Our crawler only visits pages that a user has explicitly configured — we do not spider links or discover pages on our own.

How to recognise us

Our requests carry the user agent:

WebToAPI/1.0 (+https://webtoapi.net/en/bot)

Requests come from Google Cloud Platform in region us-central1. Because we run on serverless infrastructure the source IP is not stable, so please identify us by user agent rather than by address.

How we behave

We fetch a page at most once per second per domain.
We cache each result for 60 seconds, so repeated API calls do not translate into repeated visits.
We read robots.txt before creating an API and refuse to create one for a disallowed path.
We do not attempt to bypass logins, paywalls, CAPTCHAs or bot protection.
We do not submit forms and do not follow links beyond the exact page configured.

How to block us

Add the following to your robots.txt:

User-agent: WebToAPI
Disallow: /

We re-read robots.txt at least once a day. If you need something removed sooner, or want to report abuse, email support@webtoapi.net with the URL and we will disable the affected APIs.

Ce texte est un projet et ne constitue pas un conseil juridique. Faites-le relire par un avocat avant de vous y fier.