Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

url-bot-rs: init at 0.3.1; nixos/url-bot-rs: init #105315

wants to merge 2 commits into
base: master
Choose a base branch


Copy link

@mweinelt mweinelt commented Nov 29, 2020

Motivation for this change

Add some sorely needed metadata to IRC conversations.

<urlmetadata> ⤷ url-bot-rs: init at 0.3.1; nixos/url-bot-rs: init by mweinelt · Pull Request #105315 · NixOS/nixpkgs · GitHub
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits
  NAME                                                        DESCRIPTION                                                                    EXPOSURE
✗ PrivateNetwork=                                             Service has access to the host's network                                            0.5
✓ User=/DynamicUser=                                          Service runs under a transient non-root user identity                                  
✓ CapabilityBoundingSet=~CAP_SET(UID|GID|PCAP)                Service cannot change UID/GID identities/capabilities                                  
✓ CapabilityBoundingSet=~CAP_SYS_ADMIN                        Service has no administrator privileges                                                
✓ CapabilityBoundingSet=~CAP_SYS_PTRACE                       Service has no ptrace() debugging abilities                                            
✗ RestrictAddressFamilies=~AF_(INET|INET6)                    Service may allocate Internet sockets                                               0.3
✓ RestrictNamespaces=~CLONE_NEWUSER                           Service cannot create user namespaces                                                  
✓ RestrictAddressFamilies=~…                                  Service cannot allocate exotic sockets                                                 
✓ CapabilityBoundingSet=~CAP_(CHOWN|FSETID|SETFCAP)           Service cannot change file ownership/access mode/capabilities                          
✓ CapabilityBoundingSet=~CAP_(DAC_*|FOWNER|IPC_OWNER)         Service cannot override UNIX file/IPC permission checks                                
✓ CapabilityBoundingSet=~CAP_NET_ADMIN                        Service has no network configuration privileges                                        
✓ CapabilityBoundingSet=~CAP_SYS_MODULE                       Service cannot load kernel modules                                                     
✓ CapabilityBoundingSet=~CAP_SYS_RAWIO                        Service has no raw I/O access                                                          
✓ CapabilityBoundingSet=~CAP_SYS_TIME                         Service processes cannot change the system clock                                       
✗ DeviceAllow=                                                Service has a device ACL with some special devices                                  0.1
✗ IPAddressDeny=                                              Service does not define an IP address allow list                                    0.2
✓ KeyringMode=                                                Service doesn't share key material with other services                                 
✓ NoNewPrivileges=                                            Service processes cannot acquire new privileges                                        
✓ NotifyAccess=                                               Service child processes cannot alter service state                                     
✓ PrivateDevices=                                             Service has no access to hardware devices                                              
✓ PrivateMounts=                                              Service cannot install system mounts                                                   
✓ PrivateTmp=                                                 Service has no access to other software's temporary files                              
✓ PrivateUsers=                                               Service does not have access to other users                                            
✓ ProtectClock=                                               Service cannot write to the hardware clock or system clock                             
✓ ProtectControlGroups=                                       Service cannot modify the control group file system                                    
✓ ProtectHome=                                                Service has no access to home directories                                              
✓ ProtectKernelLogs=                                          Service cannot read from or write to the kernel log ring buffer                        
✓ ProtectKernelModules=                                       Service cannot load or read kernel modules                                             
✓ ProtectKernelTunables=                                      Service cannot alter kernel tunables (/proc/sys, …)                                    
✓ ProtectSystem=                                              Service has strict read-only access to the OS file hierarchy                           
✓ RestrictAddressFamilies=~AF_PACKET                          Service cannot allocate packet sockets                                                 
✓ RestrictSUIDSGID=                                           SUID/SGID file creation by service is restricted                                       
✓ SystemCallArchitectures=                                    Service may execute system calls only with native ABI                                  
✓ SystemCallFilter=~@clock                                    System call allow list defined for service, and @clock is not included                 
✓ SystemCallFilter=~@debug                                    System call allow list defined for service, and @debug is not included                 
✓ SystemCallFilter=~@module                                   System call allow list defined for service, and @module is not included                
✓ SystemCallFilter=~@mount                                    System call allow list defined for service, and @mount is not included                 
✓ SystemCallFilter=~@raw-io                                   System call allow list defined for service, and @raw-io is not included                
✓ SystemCallFilter=~@reboot                                   System call allow list defined for service, and @reboot is not included                
✓ SystemCallFilter=~@swap                                     System call allow list defined for service, and @swap is not included                  
✗ SystemCallFilter=~@privileged                               System call allow list defined for service, and @privileged is included             0.2
✗ SystemCallFilter=~@resources                                System call allow list defined for service, and @resources is included              0.2
✓ AmbientCapabilities=                                        Service process does not receive ambient capabilities                                  
✓ CapabilityBoundingSet=~CAP_AUDIT_*                          Service has no audit subsystem access                                                  
✓ CapabilityBoundingSet=~CAP_KILL                             Service cannot send UNIX signals to arbitrary processes                                
✓ CapabilityBoundingSet=~CAP_MKNOD                            Service cannot create device nodes                                                     
✓ CapabilityBoundingSet=~CAP_NET_(BIND_SERVICE|BROADCAST|RAW) Service has no elevated networking privileges                                          
✓ CapabilityBoundingSet=~CAP_SYSLOG                           Service has no access to kernel logging                                                
✓ CapabilityBoundingSet=~CAP_SYS_(NICE|RESOURCE)              Service has no privileges to change resource use parameters                            
✓ RestrictNamespaces=~CLONE_NEWCGROUP                         Service cannot create cgroup namespaces                                                
✓ RestrictNamespaces=~CLONE_NEWIPC                            Service cannot create IPC namespaces                                                   
✓ RestrictNamespaces=~CLONE_NEWNET                            Service cannot create network namespaces                                               
✓ RestrictNamespaces=~CLONE_NEWNS                             Service cannot create file system namespaces                                           
✓ RestrictNamespaces=~CLONE_NEWPID                            Service cannot create process namespaces                                               
✓ RestrictRealtime=                                           Service realtime scheduling access is restricted                                       
✓ SystemCallFilter=~@cpu-emulation                            System call allow list defined for service, and @cpu-emulation is not included         
✓ SystemCallFilter=~@obsolete                                 System call allow list defined for service, and @obsolete is not included              
✓ RestrictAddressFamilies=~AF_NETLINK                         Service cannot allocate netlink sockets                                                
✗ RootDirectory=/RootImage=                                   Service runs within the host's root directory                                       0.1
✓ SupplementaryGroups=                                        Service has no supplementary groups                                                    
✓ CapabilityBoundingSet=~CAP_MAC_*                            Service cannot adjust SMACK MAC                                                        
✓ CapabilityBoundingSet=~CAP_SYS_BOOT                         Service cannot issue reboot()                                                          
✓ Delegate=                                                   Service does not maintain its own delegated control group subtree                      
✓ LockPersonality=                                            Service cannot change ABI personality                                                  
✓ MemoryDenyWriteExecute=                                     Service cannot create writable executable memory mappings                              
✓ RemoveIPC=                                                  Service user cannot leave SysV IPC objects around                                      
✓ RestrictNamespaces=~CLONE_NEWUTS                            Service cannot create hostname namespaces                                              
✓ UMask=                                                      Files created by service are accessible only by service's own user by default          
✓ CapabilityBoundingSet=~CAP_LINUX_IMMUTABLE                  Service cannot mark files immutable                                                    
✓ CapabilityBoundingSet=~CAP_IPC_LOCK                         Service cannot lock memory into RAM                                                    
✓ CapabilityBoundingSet=~CAP_SYS_CHROOT                       Service cannot issue chroot()                                                          
✓ ProtectHostname=                                            Service cannot change system host/domainname                                           
✓ CapabilityBoundingSet=~CAP_BLOCK_SUSPEND                    Service cannot establish wake locks                                                    
✓ CapabilityBoundingSet=~CAP_LEASE                            Service cannot create file leases                                                      
✓ CapabilityBoundingSet=~CAP_SYS_PACCT                        Service cannot use acct()                                                              
✓ CapabilityBoundingSet=~CAP_SYS_TTY_CONFIG                   Service cannot issue vhangup()                                                         
✓ CapabilityBoundingSet=~CAP_WAKE_ALARM                       Service cannot program timers that wake up the system                                  
✓ RestrictAddressFamilies=~AF_UNIX                            Service cannot allocate local sockets                                                  

→ Overall exposure level for url-bot-rs.service: 1.3 OK 🙂


Copy link
Member Author

@mweinelt mweinelt commented Nov 29, 2020

I'd totally be down for writing a test, but we'd need to spin up some IRCd. If someone can recommend an IRCd that I can spin up with just a few lines of config, that'd be great.

Copy link

@SuperSandro2000 SuperSandro2000 commented Nov 29, 2020

Patch to make it build on darwin:

diff --git a/pkgs/applications/networking/irc/url-bot-rs/default.nix b/pkgs/applications/networking/irc/url-bot-rs/default.nix
index 97445029dce5..5fe7f631bc2b 100644
--- a/pkgs/applications/networking/irc/url-bot-rs/default.nix
+++ b/pkgs/applications/networking/irc/url-bot-rs/default.nix
@@ -1,9 +1,11 @@
-{ lib
+{ stdenv
 , rustPlatform
 , fetchFromGitHub
 , pkg-config
 , openssl
 , sqlite
+, libiconv
+, Security
 rustPlatform.buildRustPackage rec {
@@ -21,13 +23,13 @@ rustPlatform.buildRustPackage rec {
   nativeBuildInputs = [ pkg-config ];
-  buildInputs = [ openssl sqlite ];
+  buildInputs = [ openssl sqlite ] ++ stdenv.lib.optionals stdenv.isDarwin [ libiconv Security ];
   preBuild = ''
     export HOME=$TMPDIR
-  meta = with lib; {
+  meta = with stdenv.lib; {
     description = "Minimal IRC URL bot in Rust";
     homepage = "";
     license = licenses.isc;
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 4df81a9efa63..0a8666e369a7 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -7901,7 +7901,9 @@ in
   uriparser = callPackage ../development/libraries/uriparser {};
-  url-bot-rs = callPackage ../applications/networking/irc/url-bot-rs {};
+  url-bot-rs = callPackage ../applications/networking/irc/url-bot-rs {
+    inherit (darwin.apple_sdk.frameworks) Security;
+  };
   urlscan = callPackage ../applications/misc/urlscan { };

Copy link

@nuxeh nuxeh commented Nov 29, 2020

This is awesome, thanks for taking the time to package url-bot-rs!

@mweinelt, totally agree about testing, this is something I've been thinking about for a long time, but have no personal experience with nix tests, so wasn't sure about the correct way to do it.

For my own testing I tend to use inspircd, since it can simply be run on the command line to give you a working ircd without having to play around with a configuration too much, if at all, and out of all the ircds i tried it seemed by far the easiest in this regard. I had guessed it could be easy enough to run it in a nix-shell for testing, but hadn't thought much further than that. However, it's not in nixpkgs, although I do have a derivation written for it that hasn't been submitted anywhere yet. Maybe this could be a good reason to get inspircd submitted too.

Copy link

@aanderse aanderse left a comment

@mweinelt exceptional work as always! I left a few optional suggestions that are worth bonus points, if you're interested.

}; = {
Copy link

@aanderse aanderse Nov 29, 2020

Can you please make a PR upstream contributing these changes? I'm sure they would greatly appreciate it.

We should consider using the upstream systemd unit via systemd.packages simplify this module.

Copy link
Member Author

@mweinelt mweinelt Nov 29, 2020

nixos/modules/services/networking/url-bot-rs.nix Outdated Show resolved Hide resolved
url-bot-rs is a minimal IRC bot that looks up URLs posted on IRC and
extracts title and other metadata and relays it back to the IRC channel
it was originally mentioned.
Copy link
Member Author

@mweinelt mweinelt commented Nov 29, 2020

Added StateDirectory, so the optional sqlite database backend has a place to store its state.

Copy link

@nuxeh nuxeh commented Nov 30, 2020

@mweinelt I made a PR mweinelt#4 against this branch to update the module and the config generation to make it feature complete, allowing a number of things to be configured, but primarily for connecting to multiple networks and enabling other additional features on a per-network basis.

Also, tested thoroughly using nixos-shell.

Copy link

@SuperSandro2000 SuperSandro2000 commented Dec 27, 2020

Result of nixpkgs-review pr 105315 run on x86_64-linux 1

1 package blacklisted:
  • tests.nixos-functions.nixos-test
1 package built:
  • url-bot-rs

Copy link

@stale stale bot commented Jun 26, 2021

I marked this as stale due to inactivity. → More info

Copy link

@nuxeh nuxeh commented Jun 26, 2021

Would like to get this merged. I see the benefit of automatic toml generation from expressions - seems to be the modern way, while also feel an ergonomic version as I proposed in the PR i made, is also nice to have, even if the implementation is more complex, it's a conundrum for me.

However, one essential feature as I see it is being able to specify multiple networks/connections in the nix. If I can get it to work I might suggest updates to this PR to do this, using the toml generation.

@stale stale bot removed the 2.status: stale label Jun 26, 2021
Copy link

@nuxeh nuxeh commented Jun 26, 2021

Btw... it's currently not possible to connect to multiple networks, since the expression will only generate a single configuration, but with the way url-bot-rs is designed, you currently need multiple configuration files to do this.

Copy link
Member Author

@mweinelt mweinelt commented Jun 26, 2021

Sorry, I was a bit overwhelmed at the time when you put up that huge pull request to my fork and was thinking about how to reconcile that with RFC42 ( and that is still an open question to me.

Copy link

@nuxeh nuxeh commented Jun 26, 2021

Makes total sense, i felt conflicted about the PR myself. Have never seen this RFC, but glad there are some guidelines in this area, otherwise it does seem to be a bit open ended how you implement modules.

I think it certainly seems that the expression to toml thing seems like the way to go, I'll see if I can knock up something to allow multiple configs using it.

Something like services.url-bot-rs.configs."name" = {};. Since I guess you can have an attribute set of the to-toml type i guess that should be straight forward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants