mvebu: add support for Turris Omnia #1210
Conversation
|
Wow! Thank you so much for this! |
target/linux/mvebu/config-4.9
Outdated
| @@ -60,6 +60,8 @@ CONFIG_BLK_DEV_LOOP=y | |||
| CONFIG_BLK_DEV_SD=y | |||
| CONFIG_BLK_MQ_PCI=y | |||
| CONFIG_BOUNCE=y | |||
| CONFIG_BTRFS_FS=y | |||
There was a problem hiding this comment.
No, this shouldn't be enabled by default for mvebu. Use the package instead if you need support however I don't see why this should be in by default.
There was a problem hiding this comment.
btrfs is used for the rootfs, it can't be distributed as a module. The only way around this is to use a different filesystem which will complicate the flashing process somewhat.
ext4 and F2FS are already in there just for the ClearFog, so there is some precedent. I will test the impact this has on the kernel image size.
There was a problem hiding this comment.
Quick comparison
ext4 + F2FS + btrfs: 2.426 MiB
ext4 + F2FS (current image): 2.103 MiB
(without ClearFog support): 1.927 MiB
I had hoped to maintain a simple flashing process, however seeing this I will look at the possibility of using a ClearFog-like eMMC layout and performing a somewhat more manual initial flash via built-in OpenWrt-based Omnia recovery partition. I would at least like to avoid requiring use of a serial cable for this process (though at least the pin header is populated in the factory!)
There was a problem hiding this comment.
I would at least like to avoid requiring use of a serial cable for this process (though at least the pin header is populated in the factory!)
Does the bootloader have an automated tftp recovery mode? If it does you can just send it an initramfs image and then you can ssh in and do a sysupgrade (which will reformat if needed).
If you have a sysupgrade working, anyway.
There was a problem hiding this comment.
There is no automated tftp recovery mode. Serial is required to access u-boot for tftp and also as it turns out to use the built-in OpenWrt recovery image.
https://www.turris.cz/doc/en/howto/omnia_factory_reset
The only way I can think to get the Omnia to do anything other than boot from internal btrfs without opening it/serial is to use fw_setenv from the factory OS to set tftp for next boot.
(thinking out loud) It may also be possible to write a script to run on Turris OS that prepares fw_setenv for booting a ClearFog-like image, downloads the LEDE eMMC image to /tmp/ and then dd's it on top of the mounted btrfs filesystem. A dirty hack, but it would avoid having to tftp from another machine.
I quite liked the concept of using the Omnia's built-in recovery and rollback built on btrfs, but writing sysupgrade scripts (and probably also adding packages for btrfs userland tools) is increasingly seeming like it will be a nuisance all while adding bloat to the mvebu kernel for the sake of bending to one device's factory partition layout.
I will experiment over the coming days with tftp booting my Omnia, flashing FAT/squashfs image using ClearFog sysupgrade scripts and then u-booting from this, and test flashing from a running btrfs environment.
There was a problem hiding this comment.
If the rootfs is BTRFS this is fine. Anything else does not make sense.
There was a problem hiding this comment.
It's not fine as it bloats all other targets and btrfs isn't really a great idea on nand either.
There was a problem hiding this comment.
This is not raw NAND where UBIFS should be used, eMMC devices have a flash translation layer. F2FS may be optimised for this case and I would choose it if starting from scratch, but that does not really make Btrfs (or ext4) a bad idea.
To be clear, I did not choose Btrfs because I think it's a neat FS. Btrfs is the only FS the Omnia's default u-boot environment will attempt to boot from, and it is the only FS that built-in recovery will format.
I would prefer to reach some consensus on this matter before doing any further work towards the current (bloat-inducing) medkit/Btrfs approach or alternative (and more complicated for end users) TFTP flash, u-boot environment update, ClearFog-like sysupgrade approach. I still not even sure which I prefer...
I do not know the ins and outs of the build system, but would it not be possible to build a separate kernel for the Omnia in order to contain the Btrfs bloat?
target/linux/mvebu/image/Makefile
Outdated
| KERNEL := dtb | kernel-bin | ||
| DEVICE_DTS := armada-385-turris-omnia | ||
| DEVICE_TITLE := Turris Omnia | ||
| DEVICE_PACKAGES := swconfig wpad-mini kmod-ath9k kmod-ath10k ath10k-firmware-qca988x |
There was a problem hiding this comment.
Not sure if we should include wifi drivers by default as not all(?) devices shipped with Atheros/QCA wifi?
There was a problem hiding this comment.
Case for exclusion: It increases the size of images and take some space for users that don't have WiFi modules.
Case for inclusion: The Omnia has a very large amount of storage and I assume that a majority shipped with the WiFi modules. <1MB out of 8GB is negligible.
I think it is more likely that space-conscious users will be building their own images, while those downloading images with the default configuration will be expecting some level of functionality out of the box.
There, I have expressed my opinion. Hopefully it's settled faster than the last time I was involved in this discussion. #931
There was a problem hiding this comment.
Maybe @oerdnj can give some numbers of how many Omnia shipped without wifi so we have some data added to the discussion.
There was a problem hiding this comment.
Enough storage, so let's not block the PR over this.
There was a problem hiding this comment.
I have notified the Turris team about this PR, hopefully someone will pop-up and comment.
| @@ -26,6 +26,9 @@ armada-385-linksys-rango) | |||
| armada-xp-linksys-mamba) | |||
| ubootenv_add_uci_config "/dev/mtd1" "0x0" "0x40000" "0x20000" | |||
| ;; | |||
| armada-385-turris-omnia) | |||
There was a problem hiding this comment.
Please maintain alphabetical order.
target/linux/mvebu/config-4.9
Outdated
| @@ -60,6 +60,8 @@ CONFIG_BLK_DEV_LOOP=y | |||
| CONFIG_BLK_DEV_SD=y | |||
| CONFIG_BLK_MQ_PCI=y | |||
| CONFIG_BOUNCE=y | |||
| CONFIG_BTRFS_FS=y | |||
There was a problem hiding this comment.
If the rootfs is BTRFS this is fine. Anything else does not make sense.
| @@ -39,6 +39,11 @@ armada-388-clearfog-*) | |||
| ucidef_add_switch "switch0" \ | |||
| "0:lan:5" "1:lan:4" "2:lan:3" "3:lan:2" "4:lan:1" "5u@eth1" "6:lan:6" | |||
| ;; | |||
| armada-385-turris-omnia) | |||
There was a problem hiding this comment.
Same here. One could argue that it currently isn't ordered alphabetically, but let's not make things worse. Ideally, fix the order in a separate commit first.
There was a problem hiding this comment.
Fixed + added separate commit to fix initial order.
target/linux/mvebu/image/Makefile
Outdated
| KERNEL := dtb | kernel-bin | ||
| DEVICE_DTS := armada-385-turris-omnia | ||
| DEVICE_TITLE := Turris Omnia | ||
| DEVICE_PACKAGES := swconfig wpad-mini kmod-ath9k kmod-ath10k ath10k-firmware-qca988x |
There was a problem hiding this comment.
Enough storage, so let's not block the PR over this.
|
Another issue I have is the permissions are off. Only root has permission for anything, making LuCI unusable. Also, does sysupgrade work with the medkit file? |
|
LuCI appears to work fine here. Permissions are preserved, only owner/group is changed to root. Or am I missing something else? As noted in the commit message sysupgrade is not currently implemented, but it is possible. The medkit file is already a metadata image, I plan to add upgrade scripts and btrfs userland tools in a later commit. |
|
All the permissions under /www are X00 where X is 6 or 7. |
|
Where are you seeing this? Permissions appear fine in the medkit tar and the same after installation. |
|
I compile my own build with luci built in. I don't use opkg if that helps.
It's the medkit file that comes with the bad permissions.
…On Tue, Jul 11, 2017, 21:05 Ryan Mounce ***@***.***> wrote:
Where are you seeing this? Permissions appear fine in the medkit tar and
the same after installation.
drwxr-xr-x 0/0 0 2017-07-10 00:51 ./www/
drwxr-xr-x 0/0 0 2017-07-10 00:51 ./www/cgi-bin/
-rwxr-xr-x 0/0 135 2017-07-10 00:51 ./www/cgi-bin/luci
-rw-r--r-- 0/0 495 2017-07-10 00:51 ./www/index.html
drwxr-xr-x 0/0 0 2017-07-10 00:51 ./www/luci-static/
drwxr-xr-x 0/0 0 2017-07-10 00:51 ./www/luci-static/bootstrap/
-rw-r--r-- 0/0 50527 2017-07-10 00:51 ./www/luci-static/bootstrap/cascade.css
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1210 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ACyJ2dp4iNkDTqmz739i9B7smYgOFrepks5sNEXwgaJpZM4OSHcC>
.
|
|
Perhaps there is some issue with your build environment? The above listing is from my own medkit build with LuCI. I built the above image on Debian 9 (unprivileged LXC container under Proxmox 5). I am building now on macOS to confirm that this is also fine. Edit: Same result on macOS (permissions are correct). @neheb could you please provide more information about your build environment? |
|
I'm running on fedora 25. I happen to be building on an NTFS drive but oddly enough, i've never encountered this issue. |
|
I also started poking at the Turris Omnia (before I was aware of your work), and went the modified u-boot bootargs / resuse of the clearfog code way (which does work fine). Obviously this is no good solution, as it requires serial access. My best idea so far to get both easy installation and a LEDE layout (readonly-fs + overlay) is to create a second stage u-boot and the following partition layout: p0: btrfs + boot/zimage - second stage u-boot + boot/dtb - so original u-boot doesn't complain (if actually needed) p1: vfat (or any other FS already supported by the kernel) + zImage - actual kernel - could also very well be a fit image + dtb p2: squashfs p3: f2fs (overlay) So the sd-card image would span p0 to p2. The second stage u-boot in p0 is in this case responsible for booting the kernel with a modified commandline to make sure the rootfs points to the right partition. For easy flashing, my idea is to include the sdcard image in the turris-medkit*.tar.gz, and let the second stage u-boot check for it and write the image to the eMMC if it finds it. That way we don't need to modify the u-boot env, don't need serial access, can easily implement sysupgrade and don't need btrfs in the kernel. Of course writing a custom u-boot is easier said then done, as the omnia support isn't upstream. Then we also need to create a host-build for the btrfs-tools to create the btrfs partition. But mkfs.btrfs does allow creating file system images with custom contents, so it should be possible to create a sd-card image that boots with the original u-boot commandline. Comments welcome, especially those that show how we don't need a second stage u-boot ;-) |
|
FWIW the omnia is geared towards relatively advanced users. Requiring serial access is not that much of a stretch. BTW, u-boot args can be modified in userspace using fw_setenv. |
|
neheb <notifications@github.com> writes:
FWIW the omnia is geared towards relatively advanced users. Requiring
serial access is not that much of a stretch.
Even for advanced users, requiring serial access to flash is a major
PITA and a barrier for adoption. I would absolutely hate to have to
bring a laptop and a serial cable to where my router is just to be able
to flash it; even if it's just the one time...
|
|
I was pitching the idea of having netconsole in the U-boot to avoid that but they argued it's a security risk since there's no switch driver in U-boot. In any case, fw_setenv should accomplish the goal |
|
Question is if we really want to pollute the tree for a handful of users or just tell them that serial access is needed. I'd vote for the serial access route in that case. |
|
I only can say that I prefer any method no requiring open the router and use serial console. But you are the developers, only is my opinion as Turris Omnia user. |
|
The current way is implemented and working, it is only lacking the failsafe/overlay-fs functions. But that is no problem when we are able to use the btrfs snapshots. In Turris-OS you are able to rollback snapshots, mount the "broken" snapshot, mend your config and then activate the repaired snapshot state. So I think there is no need to fiddle with serial consoles, additional u-boots etc. because it has no advantage. |
|
I think I have come up with a relatively clean and safe approach that avoids serial console and TFTP. Inspiration comes from the Debian wiki, just hold the reset button until 6 LEDs are illuminated to boot the LEDE ramdisk. From there you can SSH in and download/flash a sysupgrade image. Obviously the variables will have to be changed again after flashing LEDE. It suppose it would also be possible to create a kernel+initramfs that includes btrfs support and a script to copy the sysupgrade image from Btrfs /boot/ to /tmp/, unmount Btrfs, flash sysupgrade and set uboot variables automatically. At that point the user experience could be a matter of copying some files into Turris OS /boot/, executing a script, waiting a few minutes and then LEDE is fully flashed. I am sorry to those who enjoy Omnia recovery features with Btrfs but this is looking like the path of least resistance to please those that want to keep the kernel lean, use established LEDE partition layout+sysupgrade, and avoid packaging extra Btrfs userland stuff. |
|
So ext4 is ok for clearfog but btrfs is not ok for omnia? Is that the correct interpretation? |
ext4 is compiled in kernel in many other devices that don't have space issues (like say kirkwoods) as it is a commonly used filesystem for external storage. f2fs is used already in LEDE images for sdcard or for x86 images (that usually go on sdcard or usb flash storage), so again it is not uncommon. btrfs is a kind of white elephant here, and is not truly required. Then again, I don't see such hard space issues in mvebu target anyway, there is only one board with a kernel size of 3072k (linksys-wrt1900ac), while others have 6 MB or 8 MB or even no space constraints at all. |
|
I have no clear preference. On one hand I agree that there are no hard kernel space constraints on mvebu so we can likely affort the few hundred KB for btrfs support. On the other hand, btrfs is a white elephant here, no other target uses it and it introduces even more variations into the already quite diverse image format zoo. I think I would have a slight preference on consistency, means ext4/f2fs or squash+overlay for this target. |
Same here |
I think ext4/f2fs does not allow easy "reset to default" or failsafe so I would prefer squash+overlay (with overlay being ext4 or f2fs as this is emmc, not raw flash) between these two. |
|
I have updated my patch to re-use ClearFog image building and sysupgrade, squash+f2fs overlay. Flashing process feels like a bit of a hack to me, but it works. |
|
it's better if you write installing instructions in the commit description. Also because afaik the LEDE wiki does not currently have pages for device-specific instructions, only for hardware specs. |
|
I began doing that but the commands are too long to fit in the commit without wrapping, unless the 75-char limit only applies to prose. I am also going to have to have a quick look now and see if there is an easy way to get initramfs and dtb as build artifacts. |
|
Install process simplified slightly by appending DTB to ramdisk, and enabling ramdisk image generation by default for mvebu (so that users can download ramdisk from LEDE mirror rather than infected image from someone's Github releases). Install process documented in commit message. |
|
To make the installation easier, how about we just wrap the ramdisk zImage in a turris-mediakit*.tar.gz so you can use the factory recovery flash method to boot it. After that you can sysupgrade to a full sdcard image. This way you don't need to touch the u-boot environment manually at all. And for the "new" boot scripts, my suggestion would be something like this (only the LEDE part and switch on failure tested; I didn't try to flash & boot a TurrisOS image again): bootargs_lede=earlyprintk console=ttyS0,115200 root=/dev/mmcblk0p2 rootfstype=auto rootwait bootargs_turris=earlyprintk console=ttyS0,115200 rootfstype=btrfs rootdelay=2 root=b301 rootflags=subvol=@,commit=5 rw mmcload_lede=setenv bootargs "$bootargs_lede cfg80211.freg=$regdomain"; fatload mmc 0 0x01000000 zImage @ && fatload mmc 0 0x02000000 armada-385-turris-omnia.dtb @ mmcload_turris=setenv bootargs "$bootargs_turris cfg80211.freg=$regdomain"; btrload mmc 0 0x01000000 boot/zImage @ && btrload mmc 0 0x02000000 boot/dtb @ mmcboot=run mmcload_lede || run mmcload_turris; bootz 0x01000000 - 0x02000000 The idea is to use the return value of fatload to dynamically detect whether we boot LEDE (FAT) or turrisOS (btrfs), and allow going back to TurrisOS easily without ever having to touch the environment again. |
| CONFIG_I2C_BOARDINFO=y | ||
| CONFIG_I2C_CHARDEV=y | ||
| CONFIG_I2C_MUX=y | ||
| CONFIG_I2C_MUX_GPIO=y |
There was a problem hiding this comment.
why do we need this? I don't see this used anywhere
There was a problem hiding this comment.
Could someone please reply to this comment.
There was a problem hiding this comment.
The SFP autodetect script shipped with the omnia uses i2c to probe the EEPROM of inserted SPF modules, as well as some GPIO pins to detect changes. Not sure if this option is related to that.
If it is, I don't think there's any reason to keep it enabled for now: the image I built from this pull request seems to be lacking driver support for some of the features that script is using anyway, so updates will have to be made to add the SFP switch support...
|
@bobafetthotmail thank you, very good explication. @wackejohn for me it's working: My current U-Boot bootcmd: I copied it and only added But my U-Boot does not has the command |
|
@localnet |
|
@wackejohn |
|
@localnet |
|
I don't know, but my BusyBox I used: Can be interesting know if it work in TurrisOS... |
|
@localnet |
the uboot in turris omnia is customized (for example mainline uboot does not support booting from btrfs), and in general most uboots in embedded devices don't have many commands to save space or whatever. turris omnia uboot is opensourced here https://gitlab.labs.nic.cz/turris/turris-omnia-uboot |
I wish the remaining issues(are there any?) with this commit could be finally resolved. |
|
I found that someone's wrt1900acs also have the rtc issue,see: |
|
Same here: I did a reboot at monday: The date time seems continue ok: |
|
@localnet |
|
@wackejohn My U-Boot environment is most complicated... Shutdown, open the router, connect the serial console... I have few time and my Omnia is in production. The only change I did was: Try my LEDE image and if the RTC continue without working for you, I will try to get the U-Boot environment. But I will need time... |
I was under the impression that Turris Omnia had fw_printenv tool you can use from SSH to get a dump of the uboot envs. |
You are right, it is selected by PS: I have edited |
|
@localnet I think there were something misconfigured by myself,can you upload your |
|
@wackejohn I has done changes only in LEDE packages: openwrt/luci#1422 and openwrt/packages#5040 But I haven't done changes in LEDE core: Can be more easy if you discard your changes and you do: PS: Edit turris-omnia, add/remove packages, as you want. |
|
@localnet |
RGMII internal delay must be enabled to support some devices where PCB trace length is short. There is no harm in enabling if unnecessary. Patch authored by Michael Altizer <xiche@verizon.net>, rewritten to comply with LEDE submission requirements. Signed-off-by: Ryan Mounce <ryan@mounce.com.au>
ramdisk images must be used in the installation process for upcoming Turris Omnia support. Signed-off-by: Ryan Mounce <ryan@mounce.com.au>
Adds support for the Turris Omnia and builds an eMMC sysupgrade image in the same format as the SolidRun ClearFog. An initramfs image in the simple yet Omnia-specific 'medkit' image format is also built in order to ease the initial flashing process. armada-385-turris-omnia.dts was added to the mainline kernel after 4.9. It has been backported as-is and patched separately so that the dts file may be removed for later kernels. Notable hardware support omissions are support for switching between SFP cage and copper PHY, and RGB LED control. Specifications: - Marvell Armada 385 1.6GHz dual-core ARMv7 CPU - 1GB DDR3 RAM - 8GB eMMC Flash - 5x Gigabit LAN via Marvell 88E6176 Switch (2x RGMII CPU ports) - 1x switchable RJ45 (88E1514 PHY) / SFP SGMII WAN - 2x USB 3.0 - 12x dimmable RGB LEDs controlled by independent MCU - 3x Mini PCIe slots - Optional Compex WLE200N2 Mini PCIe AR9287 2x2 802.11b/g/n (2.4GHz) - Optional Compex WLE900VX Mini PCIe QCA9880 3x3 802.11ac (2.4 / 5GHz) - Optional Quectel EC20 Mini PCIe LTE modem Flash instructions: If the U-Boot environment has been modified previously (likely manually via serial console), first use serial to reset the default environment. => env default -a => saveenv Method 1 - USB 'medkit' image w/o serial - Copy lede-mvebu-turris-omnia-sysupgrade.img.gz and omnia-medkit-lede-initramfs.tar.gz to the root of a USB flash drive formatted with FAT32 / ext2/3/4 / btrfs / XFS. - Disconnect other USB devices from the Omnia and connect the flash drive to either USB port. - Power on the Omnia and hold down the rear reset button until 4 LEDs are illuminated, then release. - Wait approximately 2 minutes for the Turris Omnia to flash itself with the temporary image, during which LEDs will change multiple times. - Connect a computer to a LAN port of the Turris Omnia with a DHCP client - (if necessary) ssh-keygen -R 192.168.1.1 - ssh root@192.168.1.1 $ mount /dev/sda1 /mnt $ sysupgrade /mnt/lede-mvebu-turris-omnia-sysupgrade.img.gz - Wait another minute for the final LEDE image to be flashed. The Turris Omnia will reboot itself and you can remove the flash drive. Method 2 - TFTP w/ serial - Extract omnia-medkit-lede-initramfs.tar.gz and copy dtb + zImage to TFTP server (rename if desired) - Connect Turris Omnia WAN port to DHCP-enabled network with TFTP server - Connect serial console and interrupt U-Boot => dhcp => setenv serverip <tftp_server_ip_here> => tftpboot 0x01000000 zImage => tftpboot 0x02000000 dtb => bootz 0x01000000 - 0x02000000 - LEDE will now boot from ramdisk - Download lede-mvebu-turris-omnia-sysupgrade.img.gz to /tmp/ $ sysupgrade /tmp/lede-mvebu-turris-omnia-sysupgrade.img.gz - Wait another minute for the final LEDE image to be flashed. The Turris Omnia will reboot itself. Signed-off-by: Ryan Mounce <ryan@mounce.com.au>
|
I've successfully tested the RTC fix, included it in the u-boot environment that is set during sysupgrade, and updated this in the PR. So that's one minor issue fixed (not that I considered it a blocker)... it would be nice to get this merged in 2017 :) Note for people directly following my 'mine' branch, feel free but proceed at your own risk. For your own builds, best to re-base this PR's 'omnia' branch on top of LEDE master and skip any other stuff I may be tinkering with personally. If I remember, I will upload images incorporating the RTC fix in the next few days. Otherwise, I trust that localnet has not backdoored their images or you can do your own build. |
|
A bit late to the party but SolidRun just added RTC stuff to their repo. |
|
I have also managed to compile and flash an image based on this pull request. The USB-stick flashing procedure seemed to work, but the router didn't boot afterwards. Retrying with the tftp/serial approach worked. It may have had something to do with the uboot environment as I also needed to reset that to get the image to boot over tftp. Tested-by: Toke Høiland-Jørgensen toke@toke.dk |
|
There are some open questions mostly from @KanjiMonster could someone please take care about them, otherwise this looks ok to me. |
|
One issue I've seen since I flashed this is that having receive offloads (GRO) enabled on the wan interface breaks my he.net 6in4 tunnel (I will only get around 1mbps of by throughput). Turning off GRO immediately unbreaks things. I've seen this same behaviour before, but I don't recall if it was on the stock Omnia firmware or if it was on a different device... |
|
With the remerge in progress, all PRs on the lede-project organisation will be closed. Please help getting this merged or rebase/post it on the openwrt project page (https://github.com/openwrt/openwrt/pulls). All remaining PRs will be closed in 30 days. |
|
I took the liberty of fixing the few outstanding nits and reopening this against the openwrt repo. See openwrt/openwrt#684 Not trying to take credit, just trying to speed this along. @rmounce, let me know if you object :) |
Adds support for the Turris Omnia and builds an image in its native
'medkit' format. This is a rootfs tar, initial flash is via built-in
recovery which formats the internal eMMC storage as btrfs and extracts the
image. sysupgrade is not currently implemented. It is planned to use the
same 'medkit' image with metadata for sysupgrade.
armada-385-turris-omnia.dts was added to the mainline kernel after 4.9.
It has been backported as-is and patched separately so that the dts file
may be removed for later kernels.
Notable hardware support omissions are support for switching between SFP
cage and copper PHY, and RGB LED control.
Specifications:
Flash instructions:
(in a state of flux depending on image approach taken)