I made a mistake on the "Steps to reproduce."
Here are the corrected instructions to reproduce.
Steps to reproduce
Be on an http proxy you can verify works with OpenWRT's opkg's http_proxy option.
Have said proxy be the only internet.
Add the "option http_proxy..." setting as you normally would.
Run "opkg update"