Thank you for your interest in our rotating residential proxies! As we mention on our ethics page, we are super proud and excited to be able to enter the market of residential proxies in a way that has never been seen before: transparent and ethical from the get-go.
This article goes into detail on the steps necessary for you to set up your software to use our residential proxies. In order to use our residential proxies, you must add a small line of code to your software to “ignore a certificate” (which we’ll go into detail below) so that we can:
Provide you amazing insights that you may not be getting currently.
Ensure our residential users are not being harmed, adhering to our standard of ethics.
We call this proxy middleman: Proxy Pilot
What is it?
To use our proxies you need to add a line of code to your software to “ignore” a certificate warning (recommended option), or install our secure certificate on your server. This certificate allows us to intercept your traffic and decrypt it, which emulates a man-in-the-middle-attack with your connection.
This certificate then stores your data insights in the powerful ELK stack, fully accessible by you and your team. You’ll be able to see data like the list below, and much more:
How many connections were sent to domain.com?
How many non-200 HTTP response codes?
How much bandwidth have I consumed?
How many total connections in the past 7 days?
How much bandwidth on a specific domain?
Look at the full header sent to the target website to understand why you may have received a non-200 HTTP response (a ban, in many cases)
Why do I need to do this? I don’t have to with other providers…
There are two reasons you must work with this certificate:
By working with this certificate nuance, you will get access to these amazing statistics at your fingertips. What we have seen with many customers is that many of them do not have such statistics even on their side, or at least not to the level of detail we are able to provide. Without such data, we believe customers are just “spraying” connections at our competitors’ pools and not knowing really what’s going on or what they’re charged for; with our solution, you will know everything!
Check out this video of our VP of Technology explaining a variety of use cases you can see information on, all because of this simple certificate usage!
Video (10m): https://screencast-o-matic.com/watch/crhwXNVhrBQ
Here are some screenshots as well:
2) Adhering to our ethical standards
The second reason for needing this certificate is so that we can ensure that customers are not abusing our residential users’ devices. With normal residential proxy provider companies, they cannot technologically prohibit a customer from opening a tunnel and sending 100k requests to a single device (which would unethically harm that residential user).
The reason: without decrypting the HTTPS traffic, they do not know if there is 1 request in that tunnel, or 100k, and thus they are unable to round-robin the connections to different devices in the pool.
Therefore, by giving our software the insights to know about each connection being sent, we are able to ethically spread out connections to our residential devices and prevent harming our end residential users from getting banned/blocked from a certain website.
Although we recognize the overhead needed to implement this certificate handling (below), we believe the statistical insights, and trust from our customers about our ethics is worth the effort.
How do I work with this certificate?
We have prepared examples across many programming languages listed below. If your language is not listed, please reach out and we’d be happy to help!
Note: most implementations require <2 lines of code to be changed to make it work in your current software.
We recognize this may be new to some people, so please do not hesitate to reach out to our support team at: firstname.lastname@example.org