No documentation on how to use as transparent proxy

[tb3088]

It's listed as a feature but not actually explained. Maybe some examples?
Should it be assumed the user intuitively knows to install Tiny on the network's gateway? Or resort to tricks like IPtables to intercept and Dest-NAT?
Can it be used for HTTPS? In which case, any special configuration?
How to play nice with AWS Elastic-loadbalancers' HTTP health checks? (aside: doesn't appear to work)

[obnoxxx]

Good point!
HTTPS does currently not work in transparent mode.

Who would like to contribute documentation about transparent?

[pellucida]

The iptables REDIRECT rule(s) are buried in
https://www.gypthecat.com/a-tinyproxy-transparent-installation-on-ubuntu-12-04-with-https-support

I couldn't get transparent proxying to work without editing src/transparent-proxy.c
replacing
"getsockname (connptr->client_fd, (struct sockaddr *)..."
with (Linux specific?)
"getsockopt (connptr->client_fd, SOL_IP, SO_ORIGINAL_DST, (char *)&dest_addr..."

Even then it doesn't work for SSL/TLS.

[ghost]

Hello, does tinyproxy has transparent proxy mode?

[pellucida]

Apparently.
In short you configure the netfilter nat/PREROUTING to redirect any traffic to port 80/tcp
(or whatever) to a local port on which tinyproxy is listening.

I could only get it to work by the substituting the getsockopt() call in my comment above.
This call queries the socket to discover where the packet was originally going before netfilter
redirected it and tinyproxy makes a new connection to the original destination and shuffles
packet between the two connections.
As I didn't need or want a transparent proxy I didn't pursue this further.
Try replacing in do_transparent_proxy() / src/transparent-proxy.c

if (getsockname
                    (connptr->client_fd, (struct sockaddr *) &dest_addr,
                     &length) < 0)

with

if (getsockopt (connptr->client_fd, SOL_IP, SO_ORIGINAL_DST, (char *)&dest_addr, &length) <  0)

Some years ago I hacked apart transproxy (http://transproxy.sourceforge.net/) for a particular
purpose now long gone... that code might assist understanding what is happening.
(http://toves.freeshell.org/tproxy/, http://toves.freeshell.org/tproxy/files/tproxy.c)

[pellucida]

I have put the Linux patch for transparent proxying and some information on
http://toves.freeshell.org/tinyproxy/
http://toves.freeshell.org/tinyproxy/files/tinyproxy-1.9.0-p1.txt

[rofl0r]

@pellucida would you mind to describe what the issue with transparent proxying is, that your patch is trying to address ?

[pellucida]

@pellucida would you mind to describe what the issue with transparent proxying is, that your patch is trying to address ?

Please read http://toves.freeshell.org/tinyproxy/

Its three years ago and the source has probably been updated and/or the linux kernel support
has changed
The platform was a dual homed rhel7 box and the code wasn't picking up the original destination
ip from the REDIRECTED packets.

If you are interested I can probably pull the syslog entries and elaborate.
Email toves@sdf.org

The OP's missing documentation I believe was the netfilter stuff.

[rofl0r]

thanks.
we have

CONNECT   May 05 15:35:50 [14971]: Request (file descriptor 10): GET / HTTP/1.0
INFO      May 05 15:35:51 [14971]: process_request: trans IP GET http://0.0.0.0:0/ for 10

vs

CONNECT   May 05 17:14:50 [15601]: Request (file descriptor 10): GET / HTTP/1.1
INFO      May 05 17:14:50 [15601]: process_request: trans Host GET http://www.uq.edu.au:80/ for 10

this looks like tinyproxy failed to extract the Host: header from the original request and insert it as the destination host, right ?

[pellucida]

I added a comment on github. I had a quick look at the latest version v1.11 The penny dropped that it really only supports HTTP/1.1 ie requires the 'Host: ' header. My requirement at the time was for a HTTP/1.0 client. Hence the getsockopt() syscall to pickup the original destination IP. I would note that a 1.1 client connecting to the wrong IP address would still get the right page as tinyproxy uses the Host: header. The technique of determining the address family from socket size doesn't work on my rhel7 box. Why not just use dest_addr.v4.sin_family? The family field is common to all families. I haven't used tinyproxy for years. Once the cluster hosts had to access an external license server I was allowed to enable forwarding and nat the connections.

…

On Tue, 15 Sep. 2020, 20:05 rofl0r, ***@***.***> wrote: @pellucida <https://github.com/pellucida> would you mind to describe what the issue with transparent proxying is, that your patch is trying to address ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#45 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADJNOR63O35LLCWBIW2QTYLSF43X5ANCNFSM4CYISKMQ> .

[rofl0r]

I added a comment on github.

you mean elsewhere ? i can't see anything, please point me there

I would note that a 1.1 client connecting to the wrong IP address would still get the right page as tinyproxy uses the Host: header.

i'd think so, but it is kinda hard to test the transparent functionality without a test network setup

The technique of determining the address family from socket size doesn't work on my rhel7 box. Why not just use dest_addr.v4.sin_family? The family field is common to all families.

in which file/line ?

[pellucida]

you mean elsewhere ? i can't see anything, please point me there

Sorry I didn't realize email fed into this.

I would note that a 1.1 client connecting to the wrong IP address would still get the right page as tinyproxy uses the Host: header.

i'd think so, but it is kinda hard to test the transparent functionality without a test network setup

I tested it on real hardware but a VM setup would be just as good.
But basically tinyproxy doesn't use the IP address to which the client intended to
connect but only uses the Host header to determine the destination IP and port.
This confused me originally and hence the patch to get the original destination.

The technique of determining the address family from socket size doesn't work on my rhel7 box. Why not just use dest_addr.v4.sin_family? The family field is common to all families.

in which file/line ?

src/transparent-proxy.c
..
  55    int
  56    do_transparent_proxy (struct conn_s *connptr, orderedmap hashofheaders,
  57                          struct request_s *request, struct config_s *conf,
  58                          char **url)
..  
  86                    af = length == sizeof(dest_addr.v4) ? AF_INET : AF_INET6;
  87                    if (af == AF_INET) dest_inaddr = &dest_addr.v4.sin_addr;
  88                    else dest_inaddr = &dest_addr.v6.sin6_addr;

At line 86 debugging gives (on RHEL7.8)

DEBUG: transparent-proxy.c(85) sizeof(dest_addr.v4) = 16, length = 28

This works but there are better ways.

af = dest_addr.v4.sin_family;

[tb3088]

corporate and ISP networks are known to hijack name resolution so a log.notice() for 'dst_iP' != "what HOST_HEADER actually resolves to" would be helpful IMO.

[rofl0r]

The technique of determining the address family from socket size doesn't work on my rhel7 box.

This works but there are better ways.

@pellucida now does it actually work or not? according to posix manual, it should:

getsockname() returns the current address to which the socket  sockfd  is
bound,  in the buffer pointed to by addr.  The addrlen argument should be
initialized to indicate the amount of space  (in  bytes)  pointed  to  by
addr.   On return it contains the actual size of the socket address.  The
returned address is truncated if the buffer provided  is  too  small;  in
this  case,  addrlen will return a value greater than was supplied to the
call.

@tb3088

corporate and ISP networks are known to hijack name resolution so a log.notice() for 'dst_iP' != "what HOST_HEADER actually resolves to" would be helpful IMO.

assuming the proxy runs in the corporate/ISP network, we'd get the same value for both, wouldn't we?

[tb3088]

no the proxy would be located where TRUTH would be available. If name lookups on Tinyproxy were also lies, then the product wouldn't work.

As to hacking based on sizeof a structure when sin_family has pretty much ALWAYS been correct, just get rid of the hack, yes? Even microsoft stack fills in sin_family correctly so I don't know what prompted going about it all back-asswards.

[pellucida]

@pellucida now does it actually work or not? according to posix manual, it should:
This is what you get on rhel7.8 latest distro glibc/kernel - posix or not.

DEBUG: transparent-proxy.c(85) sizeof(dest_addr.v4) = 16, length = 28

The effect is the test for ipv4 fails ie af is set to AF_INET6.
You can always cast to the generic socket structure
((struct sockaddr*)(&dest_addr))->sa_family

Personally I would use sockaddr_storage (rather the sockaddr_union) and
use ss_family to descriminate between address types.
Probably not portable outside glibc?

The glibc bindings have an elaborate transparent union arrangement which
addresses part of this.
Possibly using getaddrinfo and getnameinfo could avoid dealing with v4 and v6
separately.

[rofl0r]

c74fe57

[pellucida]

c74fe57

I tend to avoid macros as far as possible in favour of inline functions
eg

static inline sa_family_t getfamily (__CONST_SOCKADDR_ARG sa) {
           return sa->sa_family;
}

[rofl0r]

I tend to avoid macros as far as possible in favour of inline functions

tinyproxy is compiled with hardcore pedantic compiler options, including -std=c89. means there's no inline. apart from that, we can discuss C style on freenode if you so wish, but i don't think it belongs here.