Broker

The Broker is the entry point to the library: it turns a set of providers into a stream of proxies. Two operations sit on top of it:

grab returns immediately (the work runs in a spawned task). find and check do their fail-fast setup up front — discovering the host’s external IPs and verifying at least one judge — so [Error::NoTypes], [Error::ExtIpUnknown], and [Error::NoJudges] surface from the await, not as a silently-empty stream.

The crypto provider note

This crate builds reqwest with rustls-no-provider (to keep aws-lc-rs out of the dependency graph, which is the musl cross-compile blocker). The trade-off: reqwest bakes in no crypto provider, so a bare reqwest::Client::new() panics until one is installed.

BrokerBuilder::build and Resolver::new call install_default_crypto_provider() for you, so the normal paths need nothing. You only call it yourself when you build a custom reqwest::Client to pass to BrokerBuilder::client:

#![allow(unused)]
fn main() {
use proxybroker::install_default_crypto_provider;

install_default_crypto_provider(); // idempotent; call before building your own reqwest::Client
let client = reqwest::Client::builder()
    .timeout(std::time::Duration::from_secs(10))
    .build()?;
let broker = proxybroker::Broker::builder().client(client).build();
Ok::<(), Box<dyn std::error::Error>>(())
}

install_default_crypto_provider() is idempotent (a std::sync::Once), so calling it more than once is harmless.

BrokerBuilder

Broker::builder() returns a BrokerBuilder. Every setter is consuming (returns Self); unset fields fall back to defaults. build() is infallible.

With default features (geo-bundled), build() auto-attaches the bundled DB-IP Country-Lite database unless you supplied one or called without_geo(). See feature flags for what each feature pulls in.

#![allow(unused)]
fn main() {
use proxybroker::Broker;

let broker = Broker::builder().build(); // bundled providers + bundled geo
}

FindQuery

FindQuery describes what to find and check. types is required — an empty types makes find return [Error::NoTypes]. Every other field has a default. You can construct it as a struct literal (with ..Default::default()) or through FindQueryBuilder.

FindQueryBuilder

FindQuery::builder() returns a FindQueryBuilder. build() is infallible — the NoTypes guard lives in find/check, not the builder, so the builder stays composable. The builder covers the common fields; the A4/A6 capability flags (require_cookie, require_connect25, trust_check, …) have no builder setter — set those public fields directly on the struct.

GrabQuery

GrabQuery is much smaller — grabbing does not check, so there are no judge/timeout/retry knobs.

GrabQuery derives Default, so GrabQuery::default() grabs everything the providers return.

ProxyStream

Both grab and find return a ProxyStream, which implements futures_util::Stream<Item = Proxy>. Consume it like any stream:

#![allow(unused)]
fn main() {
use futures_util::StreamExt;
async fn f(stream: &mut proxybroker::ProxyStream) {
while let Some(proxy) = stream.next().await {
    println!("{}", proxy.addr());
}
}
}

The stream ends when the source is exhausted, the limit is reached, or the stream is dropped — dropping fires a cancellation token that aborts in-flight checks, so there is no detached-task leak.

For find (not grab), the stream also carries running statistics over every checked proxy, working or not. Read them after the stream is fully drained for a complete picture:

#![allow(unused)]
fn main() {
fn f(stream: &proxybroker::ProxyStream) {
if let Some(stats) = stream.stats() {
    // aggregate over all checked proxies — see the Stats type
    let _ = stats;
}
}
}

stats() returns Some only for find; it is None for grab (nothing is checked).

A real find example

This mirrors examples/find.rs. It finds up to ten working HTTP or HTTPS proxies and prints each as it is confirmed.

use futures_util::StreamExt;
use proxybroker::{Broker, FindQuery, Proto, TypeSpec};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let broker = Broker::builder().build();

    let mut proxies = broker
        .find(FindQuery {
            types: vec![TypeSpec::any(Proto::Http), TypeSpec::any(Proto::Https)],
            limit: Some(10),
            ..Default::default()
        })
        .await?;

    while let Some(proxy) = proxies.next().await {
        // schemes() is HTTP/HTTPS support; types() has the per-protocol anonymity level.
        println!(
            "Found proxy: {:<21} {:?}  {:.2}s",
            proxy.addr(),
            proxy.schemes(),
            proxy.avg_resp_time(),
        );
    }
    Ok(())
}

The same query via the builder, which normalizes limit(0) to unlimited:

#![allow(unused)]
fn main() {
use proxybroker::{FindQuery, Proto, TypeSpec};

let query = FindQuery::builder()
    .types(vec![TypeSpec::any(Proto::Http), TypeSpec::any(Proto::Https)])
    .limit(10)
    .build();
let _ = query;
}

rotating (feature connector)

Broker::rotating is the one-call convenience for the whole discover → pool → rotate pipeline. It runs find, feeds the resulting stream into a background-warming Pool, and wraps that pool in a ready RotatingProxyConnector:

#![allow(unused)]
fn main() {
pub async fn rotating(
    &self,
    query: FindQuery,
    cfg: RotateConfig,
) -> Result<RotatingProxyConnector, Error>
}

The pool fills as find streams; the connector routes each connection through a healthy proxy (dead ones self-eject via the pool’s health thresholds). It hands hyper a raw tunnel to the target and does not terminate TLS — for an https:// target, layer TLS over the connector (e.g. a hyper-rustls HttpsConnector).

#![allow(unused)]
fn main() {
use proxybroker::{Broker, FindQuery, Proto, TypeSpec};
use proxybroker::connector::RotateConfig;

async fn f() -> Result<(), Box<dyn std::error::Error>> {
let broker = Broker::builder().build();
let connector = broker
    .rotating(
        FindQuery {
            types: vec![TypeSpec::any(Proto::Http)],
            ..Default::default()
        },
        RotateConfig::default(),
    )
    .await?;
// Hand `connector` to a hyper_util Client — see the connector page.
let _ = connector;
Ok(())
}
}

For a pre-seeded pool or a custom PoolConfig, compose the pieces yourself: find → Pool::spawn → RotatingProxyConnector::from_pool.

See also

• Proxy — the value type each ProxyStream yields.
• Pool — feed a ProxyStream into a rotating proxy pool.
• Rotating connector — what rotating returns; a drop-in hyper Service.
• feature flags — geo, server, and friends.