For the past few months, ScrapingDuck has been a “headless” product. We focused entirely on the engine, optimizing proxy rotation, handling headless browsers, and overcoming anti-bot protections.
While the API was powerful, the developer experience (DX) was strictly terminal-based. If you wanted to test a request, for example, you had to write a cURL command. If you wanted to check your usage, you had to estimate it.
Today, we are launching the full ScrapingDuck Web Dashboard.
We built this interface to provide total visibility into your scraping operations and significantly speed up the process of integrating our API into your codebase.
Here’s what’s new:
1. The Interactive Playground
Writing HTTP requests from scratch is error-prone. We added a Request Builder directly inside the app.
- Test Live: Enter a target URL and send the request. Rotating premium proxies and JavaScript rendering are built in.
- Code Generation: Once your request is working properly, we will automatically generate the ready-to-run code for Python, Node.js, Go, and PHP.
- Visual Response: See the rendered HTML or Markdown response instantly to verify the data quality.
2. Real-Time Analytics and Transparency
In the world of web scraping, “blind spots” are dangerous. You need to know if a target site has started blocking you or if your latency is spiking.
Our new analytics dashboard tracks:
- Success rates: Instantly see the percentage of
2xxversus4xx/5xxresponses. - Latency Monitoring: Track the average time (in milliseconds) for your requests to complete.
- Credit Consumption: Get a transparent view of your billing usage (standard requests vs. browser rendering).
3. Integrated documentation and logs
We have embedded our Swagger/OpenAPI documentation directly into the dashboard. You no longer need to switch tabs to find a parameter definition.
We have also added recent activity logs. Now, you can view a real-time feed of your last 25 requests, including their timestamps, status codes, and execution time. This feature is a game-changer for debugging specific URLs.
4. Better Security (Key Management)
You can now generate and revoke API keys with a single click. We recommend using separate keys for your development, staging, and production environments. If a key is compromised, you can instantly revoke it without shutting down your entire operation.
Try it out!
The dashboard is now live for all users (free and paid) starting today. Log in, generate a new key, and try the Playground.
