Skip to content

v0.4.8

v0.4.8: Solve Imperva in place when the challenged host serves a sensor
@Averyy Averyy tagged this 02 Aug 18:42 
support.lutron.com returned "Protected by imperva bot detection and could not
be bypassed" through the normal get() path, with a browser solver attached and
every log line reporting success:

  Imperva embedder solve (20 cookies, solved=True, passthrough=False)
  Browser solved challenge_type=imperva (cookie_count=15)
  Challenge detected (body): imperva interstitial   <- the replay, still blocked

The solve was earning cookies for a different WAF tenant. Imperva sites are
provisioned per host: support.lutron.com is site 2477703 (visid_incap_2477703)
while www.lutron.com is site 1555970 (nlbi_1555970) and never challenges at
all. imperva_embedder()'s "non-www subdomain must be an API host" heuristic
sent the browser to that sibling, where cookies appear on any page load whether
or not anything was challenged, so solve_imperva_embedder() reported success
having proven nothing. The reese84 token for 2477703 was never minted.

That heuristic exists for api2.realtor.ca, where it is correct: a top-level
navigation to an API host earns Imperva's interactive edet=15 block, so there
is nothing to solve at the URL and a sibling origin is the only way in. The
mistake was assuming every non-www subdomain is such a host. Normal content
hosts -- support., help., docs., shop. -- are the common case.

The challenge body already distinguishes them, and the request loop already
has it. Measured on both live sites:

                                  support.lutron.com   api2.realtor.ca
  status                          200                  403
  body                            ~6.2KB interstitial  ~960B edet=15 block
  reese sensor hooks              present              absent
  solvable by navigating the URL  yes                  no

Imperva serving the sensor interstitial is the invitation to run it on that
host, so imperva_embedder() now takes the body and returns None -- solve in
place -- when the hooks are present, keeping the sibling guess for the hookless
block it was built for. The marker set moved to is_imperva_interstitial() in
_challenge.py so detection and this decision cannot drift apart.

The predicate is deliberately narrower than detection, which also treats any
tiny (<5KB) hookless body as IMPERVA. Detection asks "is this a challenge",
where a false positive costs one solve attempt; the predicate asks "will
navigating this URL produce a sensor I can run", where a false positive aims
the solve at a page with nothing on it.

Live: support.lutron.com returns 200 and the real 110,875-byte article via
session.get(), sync and async, solved in ~3s on the challenged URL itself.

api2.realtor.ca is unaffected, which is the regression that mattered. Its body
is hookless cold and stays hookless under escalation (8 rapid requests, edet=15
every time), so the predicate is False and the derivation is byte-for-byte what
it was; both real bodies are pinned in tests. www.realtor.ca 200/283KB,
amadeus, hkbea and anz.com.au all still pass on TLS alone. An explicit
solve_origin= still overrides the derivation, as before.
Assets 2
Loading