Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Add BFS to RPi kernel. #444

Merged
merged 1 commit into from

8 participants

@graysky2

Rationale

Con Kolivas' BFS improves desktop interactivity (latency) and feels snappier on my RPi. Feelings are nice but here are some hard data I acquired on my RPi using the stock linux-rasperrypi package and the one patched with BFS v0.425.

Experiment

The test was compiling a preconfigured htop from source on my RPi. See the following little script that automates the testing: https://github.com/graysky2/bin/blob/master/make_pi_bench

Data

Here are the data using a one way anova. As you can see, I ran it with -j2 under each kernel. In each case, the BFS patched kernel shows statistical significance at the p=0.05 level.
Link to raw data
ANOVA

Conclusion

Although small, the BFS-patched kernel is statistically significantly faster than the current kernel when using this compilation endpoint. The BFS was never designed for superior performance, just superior latency. I cannot quantitate the latency, but I can quantitate the differences in compile times :p

More Reading

Here is a paper I wrote comparing the BFS on desktop and server hardware if you are interested.

@xenoxaos
Collaborator

This doesn't change the default scheduler, which is still set to cfq...correct? I don't want to build this in and have it cause a problem for people that don't know what it is.

As long as you need to manually specify the scheduler on the cmdline/bootargs, I would be ok with this.

@graysky2

Don't think of it as an I/O scheduler that can be change like BFQ, noop, deadline, etc.. BFS is a CPU scheduler and its patch is hardcoded into the kernel as-is CFS in mainline. There is no need for any special cmdline/bootargs; I built the kernel from the pull request I sent you and booted into it with no modification to anything. I am happy to share the build if you want to try it out:

http://repo-ck.com/armv6h/linux-raspberrypi-3.6.11-9-armv6h.pkg.tar.xz
http://repo-ck.com/armv6h/linux-raspberrypi-3.6.11-9-armv6h.pkg.tar.xz.sig
http://repo-ck.com/armv6h/linux-headers-raspberrypi-3.6.11-9-armv6h.pkg.tar.xz
http://repo-ck.com/armv6h/linux-headers-raspberrypi-3.6.11-9-armv6h.pkg.tar.xz.sig

Using CFS on a multicore desktop is silly if you asked me since it scales to thousands of CPUs... using CFS on a mono core ARM chip is beyond silly :p

@kmihelich kmihelich merged commit d8023ed into archlinuxarm:master
@graysky2

Nice... I didn't bump the pkgver in my pull request so you want wanna do that. I actually don't know how you guys handle the formal releases vs what is in git.

@gtmanfred
Collaborator

why isn't it in the actual kernel source code then?

@KaiSforza

He really should try and merge it again instead of maintainging a 'better' out of tree patch.

@falconindy

I'd like to see this pull request reconsidered. While "statistical significance" is a nice phrase to throw around, the test used to show this is utter hogwash (who the hell compiles anything of significance on a RPi??). If you want a low latency desktop, use the already mainline functionality of CONFIG_SCHED_AUTOGROUP.

@graysky2

@dave - No, it is not utter hogwash, it is a mathematical fact. In answer to your rhetorical question: people who own the hardware with no other ARM PC are those of us who do compile on it.

@falconindy

Yes, and it's mathematical fact that 0.0001 is greater than 0.00009.

If you want to build for ARM, you use a cross toolchain on a machine that consumes more than 2 watts. Please show a real world example showing noticeable gains on generic workloads -- something actually representative of what you would do with a tiny little ARM board.

@KaiSforza

And if you can, get ck to resubmit his work to the kernel if it's so much better.

@Kurlon
Collaborator

I use a GoFlex Net as a desktop and kernel development workstation. Less RAM, USB connected graphics, no floating point, I don't even have a functional Distcc setup right now. I can even stream MST3K while surfing and chatting on #archlinux-arm... you'd be amazed what people do on tiny little ARM boards on a regular basis.

@graysky2

@Dave - Thanks for the tip; I am using a cross toolchain.

Please re-read the first few lines of my pull request. Perhaps CK's interbench might be a better means to assess the differentiation that the BFS gives users on the Pi. I am interested to hear your suggestions.

@KaiSforza - CK did a pretty effective job communicating his views about your suggestion. Source: BFS-faq.

@KaiSforza

So when you gang together ARM boards and have 4096 CPUs, then there's no reason to use bfs. Why bother using it for 1, or 2 cores? If he is unwilling to rewrite it to support the range of hardware that Linux supports, why use this as a distribution's default kernel package? That seems to make very little sense to me.

@graysky2

@KaiSforza - I would challenge your assertion that RPi users are likely to reach into the full hardware range that Linux supports; likely >99% of Archlinux ARM users have a single board. I think the data I presented shows BFS offers tangibles to the target users of this distro.

@WarheadsSE
Collaborator
@Kurlon
Collaborator

KaiSforza: Your test case would only be valid if there was a single image linux kernel available that supported discrete ARM boards/systems and used the kernel scheduler at the macro level. So far the only SSI kernel varient I know of with any active devel is Kerrighed but it's strictly amd64 ISA only at this time. It's too bad, as I'd love to play with it on ARM, a SSI cluster of kirkwoods could be amusing given the decent interconnect (dual GigE / dual SATA) options present compared to the cpu's relative power.

Also note, Kerrighed is a useful kernel feature being developed outside of mainline. Such things do happen.

@kmihelich
Owner

I'll admit I barely looked at this before merging. So here's the question: is there a significant (preferably, order of magnitude) improvement by using this patch?

If yes, then we can stop here.

If no, then what, exactly is the benefit, in nontechnical terms if at all possible. If you can't get outside of a 10% margin of error, this isn't worth discussing and should be pulled back out. If we're talking in differences of a second on esoteric tasks, same deal.

@falconindy

The graphs graysky posted show that you're saving fractions of a second over a 90s compile. Even being generous, this is only ~1% faster.

Consider that you immediately lose out on cgroup integration with the scheduler (meaning your cpu cgroup does nothing) by using BFS.

@kmihelich
Owner

That would be along the lines of what I concluded as well. The raw data shows variances of at least +/- .5s in the samples, so it's really not showing anything. Using a better disk over USB will get you better compile times.

I'll take this patch out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Apr 5, 2013
  1. @graysky2

    add bfs to kernel

    graysky2 authored
This page is out of date. Refresh to see the latest.
Showing with 12 additions and 15 deletions.
  1. +6 −3 core/linux-raspberrypi/PKGBUILD
  2. +6 −12 core/linux-raspberrypi/config
View
9 core/linux-raspberrypi/PKGBUILD
@@ -16,7 +16,8 @@ license=('GPL2')
makedepends=('xmlto' 'docbook-xsl' 'uboot-mkimage' 'git' 'python2')
options=('!strip')
source=('config'
- 'change-default-console-loglevel.patch'
+ 'http://ck.kolivas.org/patches/bfs/3.0/3.6/3.5-sched-bfs-425.patch'
+ 'change-default-console-loglevel.patch'
'usb-add-reset-resume-quirk-for-several-webcams.patch'
'args-uncompressed.txt'
'boot-uncompressed.txt'
@@ -39,6 +40,7 @@ build() {
#nxp-pcf8523 RTC patch
patch -Np1 -i "${srcdir}/usb-add-reset-resume-quirk-for-several-webcams.patch"
+ patch -Np1 -i "${srcdir}/3.5-sched-bfs-425.patch"
# add latest fixes from stable queue, if needed
# http://git.kernel.org/?p=linux/kernel/git/stable/stable-queue.git
@@ -264,12 +266,13 @@ package_linux-headers-raspberrypi() {
rm -rf "${pkgdir}"/usr/src/linux-${_kernver}/arch/{alpha,arm26,avr32,blackfin,cris,frv,h8300,ia64,m32r,m68k,m68knommu,mips,microblaze,mn10300,parisc,powerpc,ppc,s390,sh,sh64,sparc,sparc64,um,v850,x86,xtensa}
}
-md5sums=('116a0c854e31bc0cdca41e490b2b26ea'
+md5sums=('d9ae544dcfa61e19cd4b2dc9a3f37b02'
+ '4616e99c6dc6da5962786465b19ddf18'
'9d3c56a4b999c8bfbd4018089a62f662'
'd00814b57448895e65fbbc800e8a58ba'
'9335d1263fd426215db69841a380ea26'
'a00e424e2fbb8c5a5f77ba2c4871bed4'
'2f82dbe5752af65ff409d737caf11954'
'ca74031c9e9bfc9f4a668924dcb37f4c'
- 'd9ae544dcfa61e19cd4b2dc9a3f37b02')
+ '1e46f207dcf2dd9392a099a437bb9b3e')
View
18 core/linux-raspberrypi/config
@@ -1,6 +1,6 @@
#
# Automatically generated file; DO NOT EDIT.
-# Linux/arm 3.6.11-8 Kernel Configuration
+# Linux/arm 3.6.11-9 Kernel Configuration
#
CONFIG_ARM=y
CONFIG_SYS_SUPPORTS_APM_EMULATION=y
@@ -28,6 +28,7 @@ CONFIG_IRQ_WORK=y
#
# General setup
#
+CONFIG_SCHED_BFS=y
CONFIG_EXPERIMENTAL=y
CONFIG_BROKEN_ON_SMP=y
CONFIG_INIT_ENV_ARG_LIMIT=32
@@ -95,17 +96,12 @@ CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CPUSETS=y
CONFIG_PROC_PID_CPUSET=y
-CONFIG_CGROUP_CPUACCT=y
CONFIG_RESOURCE_COUNTERS=y
CONFIG_MEMCG=y
CONFIG_MEMCG_SWAP=y
CONFIG_MEMCG_SWAP_ENABLED=y
# CONFIG_MEMCG_KMEM is not set
CONFIG_CGROUP_PERF=y
-CONFIG_CGROUP_SCHED=y
-CONFIG_FAIR_GROUP_SCHED=y
-# CONFIG_CFS_BANDWIDTH is not set
-# CONFIG_RT_GROUP_SCHED is not set
CONFIG_BLK_CGROUP=y
# CONFIG_DEBUG_BLK_CGROUP is not set
# CONFIG_CHECKPOINT_RESTORE is not set
@@ -114,7 +110,6 @@ CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
-CONFIG_SCHED_AUTOGROUP=y
CONFIG_MM_OWNER=y
CONFIG_SYSFS_DEPRECATED=y
# CONFIG_SYSFS_DEPRECATED_V2 is not set
@@ -156,10 +151,9 @@ CONFIG_PERF_USE_VMALLOC=y
CONFIG_PERF_EVENTS=y
# CONFIG_DEBUG_PERF_USE_VMALLOC is not set
CONFIG_VM_EVENT_COUNTERS=y
+CONFIG_SLUB_DEBUG=y
# CONFIG_COMPAT_BRK is not set
-CONFIG_SLAB=y
-# CONFIG_SLUB is not set
-# CONFIG_SLOB is not set
+CONFIG_SLUB=y
CONFIG_PROFILING=y
CONFIG_TRACEPOINTS=y
CONFIG_OPROFILE=m
@@ -3403,7 +3397,8 @@ CONFIG_SCHED_DEBUG=y
CONFIG_SCHEDSTATS=y
CONFIG_TIMER_STATS=y
# CONFIG_DEBUG_OBJECTS is not set
-# CONFIG_DEBUG_SLAB is not set
+# CONFIG_SLUB_DEBUG_ON is not set
+# CONFIG_SLUB_STATS is not set
# CONFIG_DEBUG_KMEMLEAK is not set
CONFIG_DEBUG_PREEMPT=y
# CONFIG_DEBUG_RT_MUTEXES is not set
@@ -3430,7 +3425,6 @@ CONFIG_DEBUG_MEMORY_INIT=y
# CONFIG_DEBUG_NOTIFIERS is not set
# CONFIG_DEBUG_CREDENTIALS is not set
CONFIG_BOOT_PRINTK_DELAY=y
-# CONFIG_RCU_TORTURE_TEST is not set
# CONFIG_RCU_TRACE is not set
# CONFIG_KPROBES_SANITY_TEST is not set
# CONFIG_BACKTRACE_SELF_TEST is not set
Something went wrong with that request. Please try again.