Bug fix with CONFIG_VXLAN is not included in the latest image build

[al-sabr]

Hi @docbobo I've downloaded the latest release and it seems that you didn't integrate the bug fix concerning the last bug we worked on together : #38

Testes image : https://github.com/hypriot/image-builder-odroid-c1/releases/tag/v0.4.2

After a fresh install on my Odroid C1 check-config.sh shows :

$ ./check-config.sh
info: reading kernel config from /proc/config.gz ...

Generally Necessary:
- cgroup hierarchy: properly mounted [/sys/fs/cgroup]
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: enabled
- CONFIG_CGROUP_FREEZER: enabled
- CONFIG_CGROUP_SCHED: enabled
- CONFIG_CPUSETS: enabled
- CONFIG_MEMCG: enabled
- CONFIG_KEYS: enabled
- CONFIG_VETH: enabled (as module)
- CONFIG_BRIDGE: enabled
- CONFIG_BRIDGE_NETFILTER: enabled
- CONFIG_NF_NAT_IPV4: enabled (as module)
- CONFIG_IP_NF_FILTER: enabled (as module)
- CONFIG_IP_NF_TARGET_MASQUERADE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_IPVS: enabled (as module)
- CONFIG_IP_NF_NAT: missing
- CONFIG_NF_NAT: enabled (as module)
- CONFIG_NF_NAT_NEEDED: enabled
- CONFIG_POSIX_MQUEUE: enabled
- CONFIG_DEVPTS_MULTIPLE_INSTANCES: enabled

Optional Features:
- CONFIG_USER_NS: enabled
- CONFIG_SECCOMP: missing
- CONFIG_CGROUP_PIDS: missing
- CONFIG_MEMCG_SWAP: enabled
- CONFIG_MEMCG_SWAP_ENABLED: enabled
    (cgroup swap accounting is currently enabled)
- CONFIG_MEMCG_KMEM: missing
- CONFIG_RESOURCE_COUNTERS: enabled
- CONFIG_BLK_CGROUP: enabled
- CONFIG_BLK_DEV_THROTTLING: missing
- CONFIG_IOSCHED_CFQ: enabled
- CONFIG_CFQ_GROUP_IOSCHED: missing
- CONFIG_CGROUP_PERF: enabled
- CONFIG_CGROUP_HUGETLB: missing
- CONFIG_NET_CLS_CGROUP: enabled (as module)
- CONFIG_NETPRIO_CGROUP: missing
- CONFIG_CFS_BANDWIDTH: enabled
- CONFIG_FAIR_GROUP_SCHED: enabled
- CONFIG_RT_GROUP_SCHED: enabled
- CONFIG_IP_VS: enabled (as module)
- CONFIG_IP_VS_NFCT: enabled
- CONFIG_IP_VS_RR: enabled (as module)
- CONFIG_EXT3_FS: missing
- CONFIG_EXT3_FS_XATTR: missing
- CONFIG_EXT3_FS_POSIX_ACL: missing
- CONFIG_EXT3_FS_SECURITY: missing
    (enable these ext3 configs if you are using ext3 as backing filesystem)
- CONFIG_EXT4_FS: enabled
- CONFIG_EXT4_FS_POSIX_ACL: enabled
- CONFIG_EXT4_FS_SECURITY: enabled
- Network Drivers:
  - "overlay":
    - CONFIG_VXLAN: missing
      Optional (for encrypted networks):
      - CONFIG_CRYPTO: enabled
      - CONFIG_CRYPTO_AEAD: enabled
      - CONFIG_CRYPTO_GCM: missing
      - CONFIG_CRYPTO_SEQIV: enabled
      - CONFIG_CRYPTO_GHASH: missing
      - CONFIG_XFRM: enabled
      - CONFIG_XFRM_USER: enabled (as module)
      - CONFIG_XFRM_ALGO: enabled
      - CONFIG_INET_ESP: enabled
      - CONFIG_INET_XFRM_MODE_TRANSPORT: enabled
  - "ipvlan":
    - CONFIG_IPVLAN: missing
  - "macvlan":
    - CONFIG_MACVLAN: enabled (as module)
    - CONFIG_DUMMY: missing
  - "ftp,tftp client in container":
    - CONFIG_NF_NAT_FTP: enabled (as module)
    - CONFIG_NF_CONNTRACK_FTP: enabled (as module)
    - CONFIG_NF_NAT_TFTP: enabled (as module)
    - CONFIG_NF_CONNTRACK_TFTP: enabled (as module)
- Storage Drivers:
  - "aufs":
    - CONFIG_AUFS_FS: enabled
  - "btrfs":
    - CONFIG_BTRFS_FS: enabled (as module)
    - CONFIG_BTRFS_FS_POSIX_ACL: enabled
  - "devicemapper":
    - CONFIG_BLK_DEV_DM: enabled
    - CONFIG_DM_THIN_PROVISIONING: enabled (as module)
  - "overlay":
    - CONFIG_OVERLAY_FS: missing
  - "zfs":
    - /dev/zfs: missing
    - zfs command: missing
    - zpool command: missing

Limits:
- /proc/sys/kernel/keys/root_maxkeys: 1000000

- "overlay":
- CONFIG_VXLAN: missing

[al-sabr]

This is related with #42

[docbobo]

There hasn't been any new release since #42, as I've been waiting for an official Odroid kernel to pick up my submission. If you want to help, you could run an update of the kernel on your instance, then verifying if CONFIG_VXLAN is present.

[al-sabr]

I've did that already yeah and it works for VXLAN with docker. But the problem is after the IPVLAN problem with ticket #42.

How can I integrate both of them?

[al-sabr]

It means that everytime I get a new Odroid C1 I will have to do this :

$ sudo apt-get install -y bc curl gcc git libncurses5-dev lzop make
$ git clone --depth 1 --single-branch -b odroidc-3.10.y https://github.com/hardkernel/linux
$ cd linux
$ make odroidc_defconfig
$ sed -ie 's/# CONFIG_VXLAN is not set/CONFIG_VXLAN=m/g' .config
$ make -j 4 uImage dtbs modules
$ sudo cp arch/arm/boot/uImage arch/arm/boot/dts/*.dtb /boot
$ sudo make modules_install
$ sudo make firmware_install
$ sudo make headers_install INSTALL_HDR_PATH=/usr
$ kver=`make kernelrelease`
$ sudo cp .config /boot/config-${kver}
$ cd /boot
$ sudo update-initramfs -c -k ${kver}
$ sudo mkimage -A arm -O linux -T ramdisk -a 0x0 -e 0x0 -n initrd.img-${kver} -d initrd.img-${kver} uInitrd-${kver}
$ sudo cp uInitrd-${kver} /boot/uInitrd

Not practical at all what do you think ?
Is there a shortcut ?

[docbobo]

Okay. So you are saying that the current C1 kernel does include the VXLAN module. That's good. In addition, you want a kernel that also includes IPVLAN. In theory, you could follow the same approach as outlined in #38, just adding something like

# sed -ie 's/# CONFIG_IPVLAN is not set/CONFIG_IPVLAN=m/g' .config

directly before or after

# sed -ie 's/# CONFIG_VXLAN is not set/CONFIG_VXLAN=m/g' .config

Once you've confirmed that this fixes the problem, we could try to create a PR for the official kernel, so that this module will be included in the future. However, I haven't seen anything in #42 that suggests that it does fix the problem. Or do you have additional insights?

[al-sabr]

I've checked in the drivers/net/ if there is ipvlan folder for this kernel and it does not.

I tried to add the missing folder + missing config manually and I'm compiling now let's see if it works.

[al-sabr]

Do you have any Odroid C1? If you have then make a simple test in swarm mode like It is described in the ticket as suggested from the Moby team.

You can read it here : moby/moby#33531

If your docker is able to pass the request in the 2nd node with its own load balancer then it means it is my setup which is broken and it also means that the fix you and I have worked on is not swarm ready.

Let me know

[al-sabr]

Unfortunately your proposition for

# sed -ie 's/# CONFIG_IPVLAN is not set/CONFIG_IPVLAN=m/g' .config

It's not as easy as you think.
Is not working because it is not in the source code for 3.10.y

[al-sabr]

I'm closing this ticket!

The issue is not the missing CONFIG_IPVLAN. Please check the following link : #42 (comment)

The problem is not Traefik nor Odroid C1 with HypriotOS.

It seems to be Docker the problem.

[firecyberice]

@gdeverlant If I remeber correctly you need kernel 3.18++ for completely working swarm mode (incl. vxlan) and the odroid c1 build uses 3.10 right now.
So it is not a docker problem.
Maybe you can try to install the raspberry pi docker debian package on a c1 to verify it.

[al-sabr]

And can you point me how to do that or give ma link or tutorial?

Wait a second I use the latest Docker 17.05ce from apt-get update your idea of using a debian package I'm already doing so.

In the meantime I've found this concerning the progress for mainline kernel on the C1 : http://linux-meson.com/doku.php

[al-sabr]

Fixed the problem as described at : moby/moby#33531

[al-sabr]

Docker swarm will never work on Odroid C1