Adding setcap-wrapper functionality to Nix #16654

ixmatus · 2016-07-01T19:35:42Z

Motivation for this change

Resolves #6768

Setting capabilities on programs is desirable over setuid because it improves the granularity of security. Nix wasn't able to support setcap-wrappers in the style of setuid-wrappers until recently when the Linux 4.4 kernel was added. Linux 4.3 included a capability named Ambient which enables a wrapper program to propagate its capabilities to an execv'ed program.

This PR adds support for setcap-wrappers (with notable improvements to the wrapper program over setuid-wrapper.c) and adds a conditional task that sets ping and ping6 to be setcap wrapped if the kernel is at-least 4.3, if not then we default to the setuid behavior.

Things done

Tested using sandboxing
(nix.useChroot on NixOS,
or option build-use-chroot in nix.conf
on non-NixOS)
Built on platform(s)
- NixOS
- OS X
- Linux
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/), NOTE: only tested ping and ping6
Fits CONTRIBUTING.md.

I tested both a sub 4.3 kernel (it defaulted to the setuid'ed ping and ping6) and with a 4.4 kernel (it defaulted to setcap'ed ping and ping6).

mention-bot · 2016-07-01T19:35:44Z

@ixmatus, thanks for your PR! By analyzing the annotation information on this pull request, we identified @wkennington, @edolstra and @fpletz to be potential reviewers

bjornfor · 2016-07-01T19:55:45Z

Interesting!

I think there are at least 3 commits that are better squashed (MHO).

bjornfor · 2016-07-01T19:59:13Z

Sign in to view

+    pkgs.stdenv.mkDerivation {
+      name         = "setcap-wrapper";
+      unpackPhase  = "true";
+      buildInputs  = [ linuxHeaders_4_4 libcap libcap_ng ];


It doesn't strike me as a particularly good idea to hardcode the linux header version here (linuxHeaders_4_4). Isn't there an attribute for the "current" kernel header package?

Yes, you're right, that should just be linuxHeaders I believe. I had to specify a hard kernel header version because stable doesn't have the 4.4 headers in it (yet) I had to copy & inject the headers. I'll fix this.

ixmatus · 2016-07-01T20:03:58Z

Yeah a bunch can be squashed but I didn't do it not knowing how you guys
felt. Also we could squash merge the PR once it's approved (unless the
process is merge, not squash)...

On Fri, Jul 1, 2016 at 2:55 PM, Bjørn Forsman notifications@github.com
wrote:

Interesting!

I think there are at least 3 commits that are better squashed (MHO).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16654 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAB3-otwQGsZgiOe9rXYsa5eB82QbCkpks5qRXDFgaJpZM4JDbKB
.

Parnell Springmeyer
parnell@digitalmentat.com | digitalmentat.com | 0xDCCF89258EAD874A
http://pgp.mit.edu/pks/lookup?op=get&search=0xDCCF89258EAD874A

fpletz · 2016-07-02T07:57:22Z

Awesome! Thanks a lot for your work! Will test later and give some more feedback.

fpletz · 2016-07-02T08:00:18Z

Sign in to view

+            owner   = "nobody";
+            group   = "nogroup";
+            setcap  = true;
+            capabilities = "cap_net_raw+ep";


Could we maybe use a nix list here? Is the format to specify the capabilities documented somewhere? May be worth mentioning where to find it in the description.

@fpletz the format is documented in cap_from_text(3)

I'll add some documentation about where one would find the capabilities format cap_from_text and setcap expect, great point.

fpletz · 2016-07-02T08:03:19Z

Sign in to view

+{
+  options = {
+    security.setcapCapabilities = mkOption {
+      type    = types.listOf types.attrs;


To validate the the attribute sets you could to use types.listOf types.submodule here.

edolstra · 2016-07-04T07:49:57Z

This should be merged with the setuid wrapper so that we don't have to maintain two very similar programs, and bloat $PATH with yet another directory. Capabilities could be specified as an attribute in security.setuidOwners (which should probably be renamed).

ixmatus · 2016-07-05T15:24:37Z

@edolstra that's a great idea. It was separated originally simply because we needed the feature and just wanted to inject it into our modules tree but now that you're saying this I definitely agree it should be unified.

Additionally, we made significant improvements to the C wrapper program that the setuid C wrapper program should benefit from so I think it's very reasonable to merge these two features together.

@edolstra would you like me to begin making those changes in this PR? Or close this one and open a new PR with the two features merged? I could also do them in a separate branch and open a PR to this branch so we can review the changes required to merge to the features, then once approved approve this PR.

ttuegel · 2016-07-15T17:04:50Z

Just adding my 👍 for KDE, which could use a setcap wrapper (currently using a setuid wrapper).

ixmatus · 2016-07-15T17:21:07Z

Okay cool. Because I haven't had a response on how maintainers want the PR for merging setuid-wrapper and setcap-wrapper I'm just going to branch off of this branch and submit a PR for THIS branch.

fpletz · 2016-07-15T17:28:27Z

@ixmatus Both options seem fine to me. Please just go ahead. Thanks a lot for your work. 👍

grahamc · 2016-07-21T01:33:54Z

Just want to put a bump on this and say I look forward to having it in mainline.

ixmatus · 2016-07-21T03:16:48Z

Thanks 😃 I'm almost done with merging the two wrapper functionalities
(what's left is testing, actually) so soon!

On Jul 20, 2016 8:33 PM, "Graham Christensen" notifications@github.com
wrote:

Just want to put a bump on this and say I look forward to having it in
mainline.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16654 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAB3-ljiLeUZ8ox_kqN-V9VRuTRUez-oks5qXsyGgaJpZM4JDbKB
.

ixmatus · 2016-08-01T20:31:58Z

Sorry I've been MIA everyone - I have this 90% complete, just haven't found a few hours to work on this amongst all of the travel for work I've been doing. I should be able to finish this week so everyone can review.

ixmatus · 2016-08-25T16:04:10Z

@joachifm how much time do I have before the 16.09 release? We've been swamped here but would love to see this in 16.09 so I can prioritize time for this if it's coming soon.

joachifm · 2016-08-25T20:38:13Z

@ixmatus the branch-off is scheduled for next week (I don't recall if branch-off is at last day of the month or 1st of the next one, so either wednesday or thursday). Is there anything we can do to help move things along?

ixmatus · 2016-08-26T02:13:13Z

@joachifm okay thanks. I just saw the email message hiding in my folder from a month ago!

Well, as this branch stands, it works. But I'm right in the middle of addressing @edolstra's feedback; I should be able to get to that this weekend. Though I will say that there's some production mileage on our own systems of the current formulation that his PR represents.

As a suggestion, it might be "safer" to merge this one and run with it, then come back in after release and refactor. Though I know that's not preferable because it's easy to forget to do that...

I'm proceeding with addressing his feedback, will try to knock it out over the weekend, and merging the two functionalities but if you feel like the above strategy (merging what is here now and works) is better, just let me know.

joachifm · 2016-08-26T12:25:39Z

@ixmatus so the only thing that's currently missing is unifying setcap and setuid? If so, my preference would be to merge it, but that's not my call to make.

domenkozar · 2016-08-28T12:00:13Z

It's hard to make that trade-off.

I've seen many times such refactorings don't happen in future, because bugs are just fixed and there's never enough initiative to rewrite/merge.

This PR should follow the boyscout rule and refactor setuid-wrapper to introduce new functionality.

ixmatus · 2016-08-28T14:38:38Z

👍 I'm working on finishing up the merged functionality today.

On Aug 28, 2016 7:00 AM, "Domen Kožar" notifications@github.com wrote:

It's hard to make that tradeoff.

I've seen many times such refactorings don't happen in future, because
bugs are just fixed and there's never enough initiative to rewrite/merge.

This PR should follow the boyscout rule
http://programmer.97things.oreilly.com/wiki/index.php/The_Boy_Scout_Rule
and refactor setuid-wrapper to introduce new functionality.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16654 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAB3-sVa2Lk-FQYqj9nxj4XvkoGVRSmuks5qkXhRgaJpZM4JDbKB
.

grahamc · 2016-08-29T02:40:14Z

This looks really great. How did the merging go?

Reviewed 6 of 7 files at r1, 1 of 1 files at r2.
Review status: all files reviewed at latest revision, 3 unresolved discussions.

Comments from Reviewable

ixmatus · 2016-08-29T04:15:28Z

Honestly, very slowly. I also moved to a new permanent residence this
weekend so I'm hoping to tackle it tomorrow night.

On Sun, Aug 28, 2016 at 9:40 PM, Graham Christensen <
notifications@github.com> wrote:

This looks really great. How did the merging go?

Reviewed 6 of 7 files at r1, 1 of 1 files at r2.
Review status: all files reviewed at latest revision, 3 unresolved

discussions.

Comments from Reviewable
https://reviewable.io:443/reviews/nixos/nixpkgs/16654#-:-KQJ9Tt9zoI8Px3GH06L:b-cgdg9z

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16654 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAB3-vygx_AhQAFsbe_VofLNz9_IypI8ks5qkkaTgaJpZM4JDbKB
.

Parnell Springmeyer
parnell@digitalmentat.com | digitalmentat.com | 0xDCCF89258EAD874A
http://pgp.mit.edu/pks/lookup?op=get&search=0xDCCF89258EAD874A

ixmatus · 2016-08-30T19:13:43Z

I'm being given a day by my employer tomorrow (Wednesday) to knock this out, I'll be working on it then. Sorry for the delay everyone.

ixmatus · 2016-09-01T15:15:05Z

Well, the 1st of this month has come - I made progress but I don't feel confident yet in the merged functionality, I want to refactor the code a bit more and do more testing. I think unless the freeze gets extended, this will have to wait until the next release or make it into a patch release.

ixmatus · 2016-09-01T15:18:58Z

I'm continuing to work on it today so hopefully I can get something I feel confident about in today - but not sure if that's cutting it too close for the freeze.

domenkozar · 2016-09-01T15:24:43Z

If the feature is complete and you're willing to fix any bugs in upcoming month, we can merge tgis today.

ixmatus · 2016-09-01T15:27:00Z

@domenkozar yeah I am willing, I'll push to this branch in a few hours after a bit more testing.

bjornfor · 2017-02-15T07:42:36Z

Why did ping change permissions?

New:

-r-x--x--x 1 root root       17680 Feb 15 08:36 ping

vs old (on release-16.09):

-r-s--x--x 1 root root       12856 Feb 15 08:36 ping

I guess this reflects a change in the defaults of the wrappers?

globin · 2017-02-15T07:49:23Z

Probably ping is using the raw network capability now instead of setuid before or similar

teh · 2017-02-15T11:21:48Z

@bjornfor

Yea, what Robin said. Removing the s bit when possible is the main idea behind this branch.

You can set before running ping WRAPPER_DEBUG=1 and it should print some debug info on what caps it sets.

bjornfor · 2017-02-15T14:58:19Z

Thanks for the info.

ixmatus · 2017-02-15T15:47:16Z

@bjornfor thank you for catching those mistakes! Is the test suite on Hydra something that can be run, say, on my own EC2 machine so that I could catch these mistakes next time while developing? Do we have developer documentation outlining how to run such a test suite? It would be nice to catch these things earlier.

The list to attrset improvement was suggested by @edolstra in this PR and I like it better than the list-based approach as it keeps it consistent with the other APIs in NixOS that expose a similar interface.

And yes, the only two programs I updated to use the new setcap feature are ping and ping6 since those two are the canonical example of using Linux capabilities. This PR paved the way and I expect more of the programs currently being setuid wrapped need to be setcap wrapped with the appropriate capabilities on them.

bjornfor · 2017-02-15T16:43:03Z

Is the test suite on Hydra something that can be run, say, on my own EC2 machine [...]

I know very little about how Hydra works, but I know a bunch of nixos tests exist in the nixos/tests directory. A single test can be run with nix-build nixos/tests/<suite>. Not sure if the test coverage is sufficient to catch the errors I faced. (I was simply doing nixos-rebuild boot on my system, unfortunately.)

ixmatus · 2017-02-15T18:00:01Z

Okay, I'll investigate using the test suite next time, thanks. Hmm I was also running nixos-rebuild boot on my EC2 instance, did you have a different configuration.nix with some things enabled that a stock configuration.nix might not?

bjornfor · 2017-02-15T18:25:29Z

@ixmatus: Yes, I have enabled extra services. And because I don't have enabled "all", there may very well be some places in nixos that are still broken due to this change... (Hence I wish this API change would have been handled a bit more carefully...)

bjornfor · 2017-02-15T18:31:02Z

@edolstra, @ixmatus: The documentation for security.wrappers say that 'source' is optional, and if unset the program will be looked up in $PATH. That is not true. If 'source' is not set nixos-rebuild fails...

ixmatus · 2017-02-15T19:04:56Z

@bjornfor this API change / PR has been around for more than six months and I'm completely engaged in helping to fix things. What do you suggest I should do different now or next time to make this more graceful or careful?

Yes that documentation is stale, I will fix it ASAP. Thanks for pointing it out.

bjornfor · 2017-02-15T19:07:17Z

Already done: ce0a52f

ixmatus · 2017-02-15T19:12:14Z

Also, maintainers appeared unphased that travis reports a red X on builds of PRs so I was assuming that the builds are broken somewhere and it is acknowledged but not blocking merges, or builds are happening elsewhere (on hydra?) that validate work to be merged.

I'm very open to helping with project short-comings (to make merges safer) and receiving constructive feedback on how I could have gone about something better or different.

ixmatus · 2017-02-15T19:16:18Z

I will try enabling all services and building on my EC2 instance to see if other places fail; I'm going to ag for the wrapper interface and then enable all the services I find and build to see if there are more failures.

ixmatus · 2017-02-15T19:20:37Z

Also, I'm sorry my change is breaking a few things. I will try a wider build and clean up whatever fails, I will take ownership of not having thought to do that entirely for all my dev test builds :)

bjornfor · 2017-02-15T19:31:45Z

@ixmatus: Unfortunately Hydra doesn't build PRs (yet). So the (real) CI only catches problems after the fact. (Travis CI doesn't check much compared to Hydra.) This is IMHO one of the biggest problems with nixpkgs development. (Along with "who decides what"...) There is no automation to prevent untested features entering master, and people generally don't care enough (IMHO) about stability on master. (And there are many committers -- both good and bad!)

I think NixOS/hydra#95 is very important. And whatever we need to get to a place where Hydra is more or less a "gateway" for code to enter master branch.

Some might argue that problems are caught within "reasonable" time with the current setup. But I think we should strive for better quality on master branch :-)

ixmatus · 2017-02-15T19:46:51Z

@bjornfor: okay I ran a nixos-rebuild build on my EC2 machine with this configuration.nix:

{
  imports = [ <nixpkgs/nixos/modules/virtualisation/amazon-image.nix> ];
  ec2.hvm = true;

  virtualisation.virtualbox.host.enable = true;
  services.locate.enable = true;
  services.exim.enable = true;
  services.xserver.desktopManager.kde5.enable = true;
  services.xserver.desktopManager.enlightenment.enable = true;
  services.cron.enable = true;
  services.fcron.enable = true;
  services.atd.enable = true;

# trace: warning: You must run gale-install in order to generate a domain key.
#  services.gale.enable = true;
  services.smokeping.enable = true;

# - PAM support is currently not implemented.
#  security.duosec.pam.enable = true;
  security.chromiumSuidSandbox.enable = true;

# A Nix error is happening here unrelated to security wrappers I think
#  security.pam.enableEcryptfs.enable = true;
  security.polkit.enable = true;
  security.pam.usb.enable = true;
  programs.light.enable = true;
  programs.kbdlight.enable = true;
}

My build succeeded with those lines commented out and I left a comment with the failure. I believe gale can be fixed so I can enable it and test that. I'm not positive what the PAM support is currently not implemented assertion message is, I will try digging into that to see what's going on there.

And finally pam.enableEcryptfs.enable = true; generated an error unrelated to the wrapper API change, I'll put a PR up to fix that ASAP.

I think I'm covering the build pretty well here. I will switch my system to this config and check all of the executables being wrapped to make sure it went the way it was supposed to. A couple of modules have some tricky logic for conditionally including certain executables in the wrappers attrset so I'll to spend some time tonight picking that apart and making sure those code paths are exercised.

Also, yes, I think better CI would help NixOS a lot; I have experience working with CI for NixOS systems because we've had to make it as smooth as possible at work where we're using NixOS. I'm more than happy to help with this but not quite sure where to start or if I have permissions to do this kind of work or what is currently being worked on.

rycee · 2017-02-15T19:50:12Z

Umm, for future merges I would appreciate some kind of commit cleanup. Ideally into a sequence of reasonably small commits that can be individually reviewed and with clear descriptive commit messages.

I happened to think of this while browsing the log since my last pull to keep myself up to date. Was having a difficult time with commits such as

a26a796 Merging against master - updating smokingpig, rebase was going to be messy
ad8fde5 Andddd more derp
ce36b58 Derp
f64b06a Hmmm
fd97408 It's clearly quite late
61fe8de Silly, should just have one activation script
48a0c5a More fixing
21368c4 Hmm, unnecessary
a4f905a Enhhh I think compile time macros are gross
785684f Ahhh, my compile-time macros confused me...of course they did...
1ad5411 Hmm
e8bec4c Implicit declared function...
a20e657 Fixing

teh · 2017-02-15T19:57:27Z

+1 for flatting commit history next time, this is also hard to back out

bjornfor · 2017-02-15T19:57:50Z

I'm more than happy to help with this but not quite sure where to start or if I have permissions to do this kind of work or what is currently being worked on.

Good! I don't know where to start either. IMHO, another big pain point of NixOS is exactly the lack of knowing who has power to do what, and who has the final say in matters... and that some core people are overloaded (or so I've heard).

ixmatus · 2017-02-15T19:59:33Z

Yeah, do we squash-merge PRs (or is there a community guideline against that?) That would have at least cleaned it up going into master...

That feedback for general PR hygiene is totally fair and I completely understand the reason, it's painful for me when I have to review PRs that aren't cleaned up or broken into smaller pieces.

This PR has seen a lot of churn (the initial PR had a single clean commit that was squashed) and my personal and work life are such that I don't always have the time I want to devote to OSS projects and I was worried I'd lose momentum again if I started juggling multiple PRs and maintaining those PRs against master.

So, I appreciate people's patience with the messy PR and commit history and I'll try to do better next time, free-time and momentum permitting.

7c6f434c · 2017-02-15T22:08:32Z

(And there are many committers -- both good and bad!)

I take offense with the statement that any of the committers can be classified into either of these categories! (well, except Eelco, obviously, he is good by definition)

I think NixOS/hydra#95 is very important. And whatever we need to get to a place where Hydra is more or less a "gateway" for code to enter master branch.

Hm, what will be the live non-gridlocked branch then? Some updates have large but rarely used reverse dependencies…

bjornfor · 2017-02-16T07:54:33Z

I take offense with the statement that any of the committers can be
classified into either of these categories! (well, except Eelco,
obviously, he is good by definition)

Oh, that's not what I meant! (Sorry for bad wording.) I mean that many committers to the main repo is both a blessing and a curse. (I think a hierarchical approach, like the topic specific sub-trees of Linux development is a better model. AFAIK, only Linus has commit access to the main repo.)

Hm, what will be the live non-gridlocked branch then? Some updates have
large but rarely used reverse dependencies…

You mean where to push updates that will cause some regressions that negatively affect some, however few, people? Difficult question, but I think we have to start by knowing the effects of a change before it enters master. Then we can decide on whether that change should go in or not.

We already have a definition of a "working" master branch: it's where Hydra builds pass and a new channel is produced. And this "working" master branch still has many packages in a broken state, but those packages are not blockers.

7c6f434c · 2017-02-16T09:56:38Z

Oh, that's not what I meant! (Sorry for bad wording.) I mean that many committers to the main repo is both a blessing and a curse. (I think a hierarchical approach, like the topic specific sub-trees of Linux development is a better model. AFAIK, only Linus has commit access to the main repo.)

I think hierarchical model is nice, it also needs more contributors than we have now — I have recently checked one of the Linux releases, and the number of people directly sending changes to Linus Torvalds is larger than the number of committers in NixPkgs. The total number of contributors to a single Linux kernel release is higher than the total number of contributors across NixPkgs history. When some area does have multiple people who can be reasonably declared maintainers, I am all for encouraging the maintainers to set the rules. For most packages I maintain I support people committing improvements if they use the package, tested the improvements and do not know any negatives.

You mean where to push updates that will cause some regressions that negatively affect _some_, however few, people?

You mean known bad effects. In that case we usually provide multiple options in Nixpkgs until nobody admits using them. I ask about running Maxima on fresh SBCL without waiting for Hydra to check that ACL2 didn't get broken by the update. Also, how do we update libxml2 without drowning in merge conflicts.

The replacements matched to much due to wrapperDir having `/bin` in its path now. cc #16654

This makes setuid wrappers not fail after upgrading. references #23641, #22914, #19862, #16654

This makes setuid wrappers not fail after upgrading. references #23641, #22914, #19862, #16654 (cherry picked from commit e82baf0)

bjornfor reviewed Jul 1, 2016
View changes

fpletz added the 0.kind: enhancement label Jul 2, 2016

fpletz reviewed Jul 2, 2016
View changes

joachifm added this to the 16.09 milestone Aug 19, 2016

7c6f434c mentioned this pull request Feb 16, 2017

nixos: i3lock/xlockmore can no longer verify my password through PAM #22862

Closed

Mic92 mentioned this pull request Feb 21, 2017

add config.services.xserver.windowManager.sway #19663

Closed

4 of 7 tasks complete

ixmatus mentioned this pull request Mar 7, 2018

nixos: setuid-wrapper: simplify readlink logic #35893

Merged

hedning mentioned this pull request Oct 19, 2019

Support launching gnome-shell with RT scheduling #71381

Closed

NixOS / nixpkgs

Sponsor NixOS/nixpkgs

Join GitHub today

Adding setcap-wrapper functionality to Nix #16654

Adding setcap-wrapper functionality to Nix #16654

Conversation

ixmatus commented Jul 1, 2016 • edited

Motivation for this change

Things done

mention-bot commented Jul 1, 2016

bjornfor commented Jul 1, 2016

This comment has been minimized.

bjornfor Jul 1, 2016 Contributor

This comment has been minimized.

ixmatus Jul 1, 2016 Author Contributor

ixmatus commented Jul 1, 2016

fpletz commented Jul 2, 2016 • edited

This comment has been minimized.

fpletz Jul 2, 2016 Member

This comment has been minimized.

joachifm Jul 2, 2016 Contributor

This comment has been minimized.

ixmatus Jul 5, 2016 • edited Author Contributor

This comment has been minimized.

fpletz Jul 2, 2016 Member

edolstra commented Jul 4, 2016

ixmatus commented Jul 5, 2016 • edited

ttuegel commented Jul 15, 2016

ixmatus commented Jul 15, 2016

fpletz commented Jul 15, 2016

grahamc commented Jul 21, 2016

ixmatus commented Jul 21, 2016

ixmatus commented Aug 1, 2016

ixmatus commented Aug 25, 2016

joachifm commented Aug 25, 2016

ixmatus commented Aug 26, 2016

joachifm commented Aug 26, 2016

domenkozar commented Aug 28, 2016 • edited

ixmatus commented Aug 28, 2016

grahamc commented Aug 29, 2016

ixmatus commented Aug 29, 2016

This looks really great. How did the merging go?

discussions.

ixmatus commented Aug 30, 2016

ixmatus commented Sep 1, 2016 • edited

ixmatus commented Sep 1, 2016

domenkozar commented Sep 1, 2016

ixmatus commented Sep 1, 2016

bjornfor commented Feb 15, 2017

globin commented Feb 15, 2017

teh commented Feb 15, 2017

bjornfor commented Feb 15, 2017

ixmatus commented Feb 15, 2017

bjornfor commented Feb 15, 2017

ixmatus commented Feb 15, 2017

bjornfor commented Feb 15, 2017

bjornfor commented Feb 15, 2017

ixmatus commented Feb 15, 2017 • edited

bjornfor commented Feb 15, 2017

ixmatus commented Feb 15, 2017

ixmatus commented Feb 15, 2017

ixmatus commented Feb 15, 2017

bjornfor commented Feb 15, 2017

ixmatus commented Feb 15, 2017

rycee commented Feb 15, 2017 • edited

teh commented Feb 15, 2017

bjornfor commented Feb 15, 2017

ixmatus commented Feb 15, 2017

7c6f434c commented Feb 15, 2017

bjornfor commented Feb 16, 2017

7c6f434c commented Feb 16, 2017

Essential cookies

Always active

Analytics cookies

ixmatus commented Jul 1, 2016 •

edited

bjornfor Jul 1, 2016
Contributor

ixmatus Jul 1, 2016
Author Contributor

fpletz commented Jul 2, 2016 •

edited

fpletz Jul 2, 2016
Member

joachifm Jul 2, 2016
Contributor

ixmatus Jul 5, 2016 •

edited

Author Contributor

fpletz Jul 2, 2016
Member

ixmatus commented Jul 5, 2016 •

edited

domenkozar commented Aug 28, 2016 •

edited

ixmatus commented Sep 1, 2016 •

edited

ixmatus commented Feb 15, 2017 •

edited

rycee commented Feb 15, 2017 •

edited