[2.1.5] Upload BlockBlob doesn't use proxy, and a risk of HttpProxy

[yuxisun1217]

Hi,

In 2.1.5, it seems that the put block/page blob request doesn't use proxy(wire.py Line 390 and 410), which do use proxy in 2.0.16. Is it a change or an issue? I'm not quite sure if it is necessary to use proxy.

And there might be a risk of the connection between WALA and Azure Server(168.63.129.16). If I configure a Network Security Group, add rules like this:

The rules only allow proxy server(172.20.0.254:3128) and deny all others. So that the WALA fails to connect to Azure Server even if enable HttpProxy(Because some of the wala http requests never use proxy). I know this is not a good policy but at least the Azure Portal doesn't forbidden customers to do that and even has no warning messages on UI...
Is there any reason that cannot let all the wala http request use proxy if enable HttpProxy in waagent.conf?(Perhaps when proxy service and the client VM are in different regions? I guess)

[brendandixon]

@yuxisun1217 Thank you for finding this. We'll address it in v2.1.7.

[hglkrijger]

There are some additional details in the Bugzilla bug, listing here as well:

Description of problem:

"HttpProxy" configuration in /etc/waagent.conf doesn't work. The waagent connect to Azure Server directly.

Version-Release number of selected component (if applicable):
WALinuxAgent-2.1.5 (upstream, package by tester)

RHEL Version:
RHEL-7.3-20160729.1

How reproducible:
100%

Steps to Reproduce:

1. Prepare a VM on Azure with squid installed. Prepare a RHEL7.3 VM on Azure. These 2 VMs are in the same Virtual Network. Make sure that the client VM can connect to the Internet through the squid proxy, such as:

wget -e "http_proxy=[squid VM private IP]:3128" http://www.baidu.com

Can download index.html successfully.

1. Change HttpProxy configurations in /etc/waagent.conf at the client VM:

HttpProxy.Host=[squid VM private IP]
HttpProxy.Port=3128

1. restart waagent service

systemctl restart waagent

1. Catch packages at the squid VM and the client VM:

tcpdump -s 0 -A 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420'

Actual results:

Client VM: Can catch packages sent by waagent.
Squid VM: No packages come from client VM are catched.

Expected results:

Squid VM: Can capture upload blob packages at proxy.

Additional info:

1. It works well in WALA-2.0.16.
2. The root cause is that the PUT block/page blob request doesn't use proxy(wire.py Line 390 and 410), which do use proxy in 2.0.16.

[hglkrijger]

fix is merged, closing

[yuxisun1217]

Hi @hglkrijger ,

I install the v2.1.6.8 and try to verify this. It seems that if I set HttpProxy, waagent cannot send blob status. Could you please help to check this? Thanks!

OS: RHEL-7.3 internal build
WALA version: 2.1.6.8

/var/log/waagent.log:
2016/09/26 14:26:46.390242 VERBOSE HTTP Req: GET http://168.63.129.16/machine/?comp=goalstate
2016/09/26 14:26:46.406912 VERBOSE Data=None
2016/09/26 14:26:46.414903 VERBOSE Header={'x-ms-version': '2012-11-30', 'x-ms-agent-name': 'WALinuxAgent'}
2016/09/26 14:26:46.433542 VERBOSE HTTP Resp: Status=200
2016/09/26 14:26:46.442239 VERBOSE Header=[('date', 'Mon, 26 Sep 2016 06:26:46 GMT'), ('content-length', '2107'), ('content-type', 'text/xml; charset=utf-8'), ('server', 'Microsoft-IIS/8.5')]
2016/09/26 14:26:46.465387 VERBOSE Load GoalState.xml
2016/09/26 14:26:46.473473 VERBOSE Handle extensions updates for incarnation 1
2016/09/26 14:26:46.482715 VERBOSE No ext handler config found
2016/09/26 14:26:46.490295 VERBOSE Report vm agent status
2016/09/26 14:26:46.497096 VERBOSE Upload status blob
2016/09/26 14:26:46.503760 VERBOSE Check blob type.
2016/09/26 14:26:46.509827 VERBOSE HTTP Req: HEAD https://walaautoasmeastus.blob.core.windows.net/vhds/nay-67-ond-squid.nay-67-ond-squid.walaautos73small-proxy.status?sr=b&sp=rw&se=9999-01-01&sk=key1&sv=2014-02-14&sig=HSuTGMDspHXds1O56J%2FrlCM0nj8qnzEA5QwuRpc4F2w%3D
2016/09/26 14:26:46.535872 VERBOSE Data=None
2016/09/26 14:26:46.541229 VERBOSE Header={'x-ms-version': '2014-02-14', 'x-ms-date': '2016-09-26T06:26:46Z'}
2016/09/26 14:26:46.553064 WARNING Socket IOError getaddrinfo() argument 2 must be integer or string, args:('getaddrinfo() argument 2 must be integer or string',)
2016/09/26 14:26:46.569303 INFO Retry=0, HEAD https://walaautoasmeastus.blob.core.windows.net/vhds/nay-67-ond-squid.nay-67-ond-squid.walaautos73small-proxy.status?sr=b&sp=rw&se=9999-01-01&sk=key1&sv=2014-02-14&sig=HSuTGMDspHXds1O56J%2FrlCM0nj8qnzEA5QwuRpc4F2w%3D
2016/09/26 14:26:56.603822 WARNING Socket IOError getaddrinfo() argument 2 must be integer or string, args:('getaddrinfo() argument 2 must be integer or string',)
2016/09/26 14:26:56.620361 INFO Retry=1, HEAD https://walaautoasmeastus.blob.core.windows.net/vhds/nay-67-ond-squid.nay-67-ond-squid.walaautos73small-proxy.status?sr=b&sp=rw&se=9999-01-01&sk=key1&sv=2014-02-14&sig=HSuTGMDspHXds1O56J%2FrlCM0nj8qnzEA5QwuRpc4F2w%3D
2016/09/26 14:26:59.684508 VERBOSE Found event file: /var/lib/waagent/events/1474871180359187.tld
2016/09/26 14:26:59.694624 VERBOSE Processed event file: /var/lib/waagent/events/1474871180359187.tld
2016/09/26 14:26:59.705175 VERBOSE HTTP Req: POST http://168.63.129.16/machine?comp=telemetrydata
2016/09/26 14:26:59.714779 VERBOSE Data=]]>
2016/09/26 14:26:59.848919 VERBOSE Header={'Content-Type': 'text/xml;charset=utf-8', 'x-ms-version': '2012-11-30', 'x-ms-agent-name': 'WALinuxAgent'}
2016/09/26 14:26:59.866292 VERBOSE HTTP Resp: Status=200
2016/09/26 14:26:59.872255 VERBOSE Header=[('date', 'Mon, 26 Sep 2016 06:27:00 GMT'), ('content-length', '0'), ('content-type', 'text/xml; charset=utf-8'), ('server', 'Microsoft-IIS/8.5')]
2016/09/26 14:27:06.656307 WARNING Socket IOError getaddrinfo() argument 2 must be integer or string, args:('getaddrinfo() argument 2 must be integer or string',)
2016/09/26 14:27:06.673037 ERROR Event: name=WALinuxAgent, op=, message=Failed to report vm agent status: (000008)Failed to get status blob type: (000009)HTTP Err: HEAD https://walaautoasmeastus.blob.core.windows.net/vhds/nay-67-ond-squid.nay-67-ond-squid.walaautos73sm
2016/09/26 14:27:06.698397 VERBOSE Successfully reported vm agent status

[hglkrijger]

@yuxisun1217 - could you provide your /etc/waagent.conf? It looks like perhaps the port is not configured correctly.

[yuxisun1217]

Hi @hglkrijger ,

My proxy configuration is:
HttpProxy.Host=172.20.0.254
HttpProxy.Port=3128

Environment:
# waagent -version
WALinuxAgent-2.1.6.8 running on redhat 7.3
Python: 2.7.5
Goal state agent: 2.1.6.8


This is the who waagent.conf file:

#
# Microsoft Azure Linux Agent Configuration
#

# Enable instance creation
Provisioning.Enabled=y

# Password authentication for root account will be unavailable.
Provisioning.DeleteRootPassword=y

# Generate fresh host key pair.
Provisioning.RegenerateSshHostKeyPair=y

# Supported values are "rsa", "dsa" and "ecdsa".
Provisioning.SshHostKeyPairType=rsa

# Monitor host name changes and publish changes via DHCP requests.
Provisioning.MonitorHostName=y

# Decode CustomData from Base64.
Provisioning.DecodeCustomData=n

# Execute CustomData after provisioning.
Provisioning.ExecuteCustomData=n

# Algorithm used by crypt when generating password hash.
#Provisioning.PasswordCryptId=6

# Length of random salt used when generating password hash.
#Provisioning.PasswordCryptSaltLength=10

# Allow reset password of sys user
Provisioning.AllowResetSysUser=n

# Format if unformatted. If 'n', resource disk will not be mounted.
ResourceDisk.Format=y

# File system on the resource disk
# Typically ext3 or ext4. FreeBSD images should use 'ufs2' here.
ResourceDisk.Filesystem=ext4

# Mount point for the resource disk
ResourceDisk.MountPoint=/mnt/resource

# Create and use swapfile on resource disk.
ResourceDisk.EnableSwap=y

# Size of the swapfile.
ResourceDisk.SwapSizeMB=2048

# Comma-seperated list of mount options. See man(8) for valid options
ResourceDisk.MountOptions=None

# Enable verbose logging (y|n)
Logs.Verbose=y

# Root device timeout in seconds.
OS.RootDeviceScsiTimeout=300

# If "None", the system default version is used.
OS.OpensslPath=None

# If set, agent will use proxy server to access internet
HttpProxy.Host=172.20.0.254
HttpProxy.Port=3128

# Detect Scvmm environment, default is n
# DetectScvmmEnv=n

#
# Lib.Dir=/var/lib/waagent

#
# DVD.MountPoint=/mnt/cdrom/secure

#
# Pid.File=/var/run/waagent.pid

#
# Extension.LogDir=/var/log/azure

#
# Home.Dir=/home

# Enable RDMA management and set up, should only be used in HPC images
# OS.EnableRDMA=y

# Enable or disable goal state processing auto-update, default is enabled
AutoUpdate.Enabled=n

# Determine the update family, this should not be changed
# AutoUpdate.GAFamily=Prod

[hglkrijger]

Thanks @yuxisun1217, I will investigate further.

[yuxisun1217]

Thanks Hans! I change the "conf.get" to "conf.get_int" follow your fix #451 and it works well now. :) I'll verify it again in v2.1.7.

[hglkrijger]

Great, thanks @yuxisun1217.