A development site that never calls the outside APIs itself

A development site for a content publisher that relies on outside APIs has an awkward problem. The development copy needs realistic data to be useful, but every request it makes to a provider costs quota, may break a provider's terms, and doubles the traffic coming from your servers. On one client's site, a data provider had already blocked the development server for making too many requests.

The rule I set was simple: dev never calls an outside API itself.

The development site reads what live records

Production already fetches everything it needs and caches the responses. So production now also records the raw response of each GET to an allowed list of provider hosts, in its object cache, for 24 hours. When dev wants that data, it asks production for the cached copy.

Two details matter:

• Production never fetches on dev's behalf. If live has not fetched something recently, the answer is a miss, and a miss stays a miss. Dev cannot cause a single extra request to a provider.
• Misses are remembered briefly. A miss is cached on dev for a minute, and a failed round trip backs off for five, so a broken link does not turn into a flood of retries.

A tunnel, not the public site

The obvious route would be an HTTPS endpoint on the live site. That failed straight away: production's firewall challenges server traffic, as it should. Weakening the firewall for dev was not an option.

Instead, dev keeps an SSH tunnel open to production as a small systemd service. The key it uses can do exactly one thing:

restrict,port-forwarding,permitopen="127.0.0.1:443",command="/bin/false" ssh-ed25519 AAAA... relay

No shell, no commands, and forwarding to a single local port. On production, the relay endpoint only accepts requests from loopback, and each request must carry an HMAC signature made with a key held outside the repository on both servers.

What it gives us

• Dev shows the same data as live, so layout and logic are tested against reality.
• No extra quota is spent and no provider sees a second server.
• Production's firewall stays as strict as it was.
• Credentials for the providers never need to exist on dev at all.

When the development site later moved to a new server, the same arrangement moved with it: a new tunnel service and the same restricted key line on production.

If dev shows empty data

Check the tunnel service first. Then check whether live has fetched that data recently. An empty panel on dev usually means production simply has not needed it today, which is the system working as designed.