nixos/seafile: init service #119719

greizgh · 2021-04-17T15:53:47Z

Motivation for this change

This PR adds a seafile service.
It's partly based on previous work in #15900.

Seafile's official documentation suggest to download pre-built binaries or use a docker-compose deployment based on a sub-optimal (hardcoded path, multiple services) image.
Neither options are thrilling, that's why I went the "build from source" route and diverged from official documentation.

One component which does not appear here, but is the entry-point when deploying the official seafile "distribution", is the seafile-controller binary.
It turns out it is mostly a supervisor for the different components (seafile-server, seahub, seafdav) and can be replaced by proper systemd units.
Since "pro" components are out of scope, there are not many moving parts: only seafile-server and seahub (the web UI).

Things I checked:

nixos tests are passing
seahub (the webui) is working:
- file upload
- avatar upload
- encrypted library creation
- user creation
- creating and accessing a shared link
synchronization using seafile-client is working fine too

Most of the tests were done with the following container definition

{ pkgs, ... }: {
  services.seafile = {
    enable = true;
    ccnetSettings.General.SERVICE_URL = "http://10.233.1.2";
    adminEmail = "test@local.com";
    initialAdminPassword = "password";
  };
  services.nginx = {
    enable = true;
    virtualHosts."10.233.1.2" = {
      locations."/".proxyPass = "http://unix:/run/seahub/gunicorn.sock";
      locations."/seafhttp" = {
        proxyPass = "http://127.0.0.1:8082";
        extraConfig = ''
          rewrite ^/seafhttp(.*)$ $1 break;
          client_max_body_size 0;
          proxy_connect_timeout  36000s;
          proxy_read_timeout  36000s;
          proxy_send_timeout  36000s;
          send_timeout  36000s;
          proxy_http_version 1.1;
        '';
      };
    };
  };
  networking.firewall = { allowedTCPPorts = [ 80 ]; };
}

Feedback is highly welcome!

Things done

pkgs/top-level/all-packages.nix

pkgs/tools/networking/ccnet/default.nix

pkgs/servers/seafile/seafile-server.nix

pkgs/development/python-modules/seahub/default.nix

pkgs/development/python-modules/django-statici18n/default.nix

pkgs/development/python-modules/django-formtools/default.nix

pkgs/development/python-modules/django-statici18n/default.nix

pkgs/development/python-modules/django-formtools/default.nix

pkgs/top-level/all-packages.nix

pkgs/tools/networking/ccnet/default.nix

pkgs/servers/seafile-server/default.nix

pkgs/development/python-modules/seahub/default.nix

pkgs/development/python-modules/django-statici18n/default.nix

pkgs/development/python-modules/django-formtools/default.nix

nixos/modules/services/networking/seafile.nix

SuperSandro2000 · 2021-04-19T03:15:20Z

Also please create the commits according to the contributing guide.

aanderse

Thanks for contributing this. I have left a few comments.

I'd also like to point out that, as you mention, upstream highly suggests not packaging this application and just running their docker image or whatever. Are you going to be able to properly maintain this in nixpkgs?

nixos/modules/services/networking/seafile.nix

aanderse · 2021-04-19T12:24:32Z

pkgs/servers/seafile-server/default.nix

+    inherit stdenv lib fetchFromGitHub cmake libevent;
+  };
+  oniguruma = import ./oniguruma.nix { inherit stdenv lib fetchFromGitHub cmake; };
+in stdenv.mkDerivation rec {


I'm getting errors when trying some of these programs out:

~/nixpkgs ((792b8f64…))> /nix/store/kmhchjnd2ixm8n7bq23p6map8550qwig-seafile-server-8.0.4/bin/seafile-admin --help File "/nix/store/kmhchjnd2ixm8n7bq23p6map8550qwig-seafile-server-8.0.4/bin/seafile-admin", line 12 print 'Python 3 not supported yet. Quit now' ^ SyntaxError: Missing parentheses in call to 'print'. Did you mean print('Python 3 not supported yet. Quit now')?

And this one says nothing:

~/nixpkgs ((792b8f64…))> /nix/store/kmhchjnd2ixm8n7bq23p6map8550qwig-seafile-server-8.0.4/bin/seaf-server --help

Unfortunately that reflects the state of upstream 😒
seaf-server does not seem to respond to --help but requires several options to run. (see ExecStart of seaf-server for a working example)

As for seafile-admin, it's part of the official distribution but also fails on python3 so I guess this is a leftover 🤷.
It's not mentioned in the documentation either. It looks like something used before setup-seafile.sh.

greizgh · 2021-04-20T19:19:17Z

I'd also like to point out that, as you mention, upstream highly suggests not packaging this application and just running their docker image or whatever. Are you going to be able to properly maintain this in nixpkgs?

This is a very valid concern regarding quality and long-term support of packages in NixOS.
Regarding the suggestion from upstream, I guess it's because they prefer to focus their effort on their bundle.
Both docker image and bundled binaries are non starters for me: I'd rather not run an untrusted binary, especially when it is handling your data.

As for my ability to maintain the package: any answer would be an empty promise 😄
My main motivation to properly package seafile on NixOS is to migrate and existing deployment (on Debian) to a new NixOS machine.
It has been running for years and I'd like it to be that way on NixOS too, I guess that's a good enough incentive to keep the package updated.

Now, to be honest, I'm not a nix expert (learnt a lot thanks to your great feedback here already) and I hope this service can get enough community interest to get some help on the long-term.

aanderse · 2021-04-20T20:35:27Z

Good answers 😄

pkgs/tools/networking/ccnet/default.nix

pkgs/servers/seafile-server/oniguruma.nix

pkgs/servers/seafile-server/libevhtp.nix

schmittlauch · 2021-10-22T12:06:53Z

Or if you give me push access to your fork I guess I could fix this myself @greizgh

greizgh · 2021-10-23T09:40:50Z

Thank you @schmittlauch I've just rebased the branch 👍. (Meanwhile I'll give you access to my fork to ease future maintenance).

It succeeds for me. Apparently this line was forgotten in PR NixOS#119719.

Pacman99 · 2021-11-17T02:59:29Z

Glad to see this got merged!
I just updated my server to the module in the master branch now and to the latest seafile and seahub versions. Seafile itself works just fine still, I can sync with my client. But seahub seems to have stopped working. I get a cryptic python error in the seahub log and the webui says the page is unavailable after I login.

This might just be something I'm doing wrong, but any ideas as to what could have caused this?

Seahub.log error

2021-11-17 00:44:25,983 [WARNING] django.request:222 log_response Not Found: /favicon.ico
2021-11-17 02:34:43,023 [ERROR] django.request:222 log_response Internal Server Error: /
Traceback (most recent call last):
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/nix/store/wknwv8s3dj7xc7v8jlfmyxazqjwbzpdg-python3.9-seahub-8.0.7/seahub/auth/decorators.py", line 27, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/nix/store/wknwv8s3dj7xc7v8jlfmyxazqjwbzpdg-python3.9-seahub-8.0.7/seahub/views/__init__.py", line 1173, in react_fake_view
    return render(request, "react_app.html", {
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/shortcuts.py", line 36, in render
    content = loader.render_to_string(template_name, context, request, using=using)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/template/loader.py", line 62, in render_to_string
    return template.render(context, request)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/template/base.py", line 171, in render
    return self._render(context)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/template/loader_tags.py", line 150, in render
    return compiled_parent._render(context)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/django/template/library.py", line 192, in render
    output = self.func(*resolved_args, **resolved_kwargs)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/webpack_loader/templatetags/webpack_loader.py", line 20, in render_bundle
    tags = utils.get_as_tags(
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/webpack_loader/utils.py", line 61, in get_as_tags
    bundle = _get_bundle(bundle_name, extension, config)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/webpack_loader/utils.py", line 39, in _get_bundle
    bundle = get_loader(config).get_bundle(bundle_name)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/webpack_loader/loader.py", line 97, in get_bundle
    filtered_chunks = self.filter_chunks(chunks)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/webpack_loader/loader.py", line 45, in filter_chunks
    ignore = any(regex.match(chunk)
  File "/nix/store/dsb4rxn2vh17d9z1hvp554rhd3ffx5vp-python3-3.9.6-env/lib/python3.9/site-packages/webpack_loader/loader.py", line 45, in <genexpr>
    ignore = any(regex.match(chunk)
TypeError: expected string or bytes-like object

qbit · 2021-12-02T20:20:21Z

I just hit the same issue that @Pacman99 has hit. I am fronting everything with nginx and the following virtualhost entry:

        "seafile.example.com" = {                                                                              
          forceSSL = true;                                                                     
          enableACME = true;                                                                               
                                                                                                                                                                                                
          locations."/" = {
            proxyPass =
              "http://unix:/var/run/seahub/gunicorn.sock";
          };  
                                   
          locations."/seafhttp" = {
            proxyPass =                                             
              "http://127.0.0.1:${toString config.services.seafile.seafileSettings.fileserver.port}";
          };        
        }; 
        ```

greizgh · 2021-12-03T17:23:08Z

@Pacman99 @qbit the issue is related to an incompatible dependency version. PR with a fix.

dotlambda · 2021-12-30T13:42:18Z

pkgs/top-level/python-packages.nix

@@ -6950,6 +6954,8 @@ in {

  pysdl2 = callPackage ../development/python-modules/pysdl2 { };

+  pysearpc = toPythonModule pkgs.libsearpc;


This is garbage. Remove immediately!
EDIT: Use the correct version of Python instead.

dotlambda · 2021-12-30T13:45:21Z

pkgs/top-level/python-packages.nix

@@ -8311,6 +8317,10 @@ in {

  seabreeze = callPackage ../development/python-modules/seabreeze { };

+  seahub = callPackage ../development/python-modules/seahub { };
+
+  seaserv = toPythonModule pkgs.seafile-server;


This is garbage. Remove immediately!
EDIT: Use the correct version of Python instead.

dotlambda · 2021-12-30T13:47:46Z

pkgs/development/python-modules/seahub/default.nix

@@ -0,0 +1,56 @@
+{ stdenv, lib, fetchFromGitHub, python3Packages, makeWrapper }:
+
+python3Packages.buildPythonPackage rec {


Is this supposed to be used as import seahub? If not, this should not be in pythonPackages and should use buildPythonApplication.

dotlambda · 2021-12-30T13:49:18Z

Not even the commit messages had the proper format I will open a PR reverting this.
EDIT: GitHub can't revert it automatically anymore. We will remove the service and the appropriate packages if this is not fixed promptly.

dotlambda · 2021-12-30T17:00:11Z

I did not intend to criticize the author of this PR who probably had no idea that many things went terribly wrong here but I find it completely unacceptable that the PR was merged in this state.

qbit · 2021-12-30T17:05:28Z

@dotlambda I am a noob bystander and have no idea why the things you point out are issues. Would you mind going into a bit more detail on why this approach isn't good?

dotlambda · 2021-12-30T17:16:39Z

@dotlambda I am a noob bystander and have no idea why the things you point out are issues. Would you mind going into a bit more detail on why this approach isn't good?

pythonPackages.seahub can't use python3Packages cause that would mean it always uses whatever version of Python python3 currently refers to. Try instantiating python38Packages.seahub as well as python39Packages.seahub and you'll get the same result.
In fact seahub doesn't provide anything that could be used like import seahub so it doesn't belong in the Python package set.

pysearpc and seaserv apparently can be imported as Python modules, but they need to be passed the correct version of Python if they are supposed to be included in pythonPackages.

Commit messages for Python packages start with e.g. python3Packages.seahub.

See https://nixos.org/manual/nixpkgs/unstable/#python if you want to know more.

dotlambda · 2021-12-30T17:36:57Z

Here's an example of what needs to be done: dotlambda@14ded1f

greizgh · 2022-01-03T07:45:04Z

Thank you @dotlambda for the fixes and explanation 👍

pinpox · 2022-01-03T09:01:50Z

I was thinking about migrating from nextcloud to seafile using this service. Is this in a state where it can be used productively? I'm a bit concerned it might run into bugs or even be removed after I migrate my data and workflow to it.

github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: python 8.has: module (update) This PR changes an existing module in `nixos/` labels Apr 17, 2021

SuperSandro2000 reviewed Apr 17, 2021

View reviewed changes

pkgs/top-level/all-packages.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Apr 17, 2021

View reviewed changes

SuperSandro2000 requested changes Apr 17, 2021

View reviewed changes

pkgs/top-level/all-packages.nix Outdated Show resolved Hide resolved

ofborg bot added 8.has: package (new) This PR adds a new package 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 1-10 10.rebuild-linux: 11-100 labels Apr 17, 2021

greizgh marked this pull request as ready for review April 18, 2021 14:02

greizgh requested review from FRidh and jonringer as code owners April 18, 2021 14:02

SuperSandro2000 reviewed Apr 19, 2021

View reviewed changes

pkgs/tools/networking/ccnet/default.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Apr 19, 2021

View reviewed changes

pkgs/servers/seafile-server/default.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Apr 19, 2021

View reviewed changes

pkgs/development/python-modules/seahub/default.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Apr 19, 2021

View reviewed changes

SuperSandro2000 requested a review from aanderse April 19, 2021 03:15

aanderse reviewed Apr 19, 2021

View reviewed changes

greizgh force-pushed the seafile branch 2 times, most recently from ff7f70c to 3d3a35e Compare April 20, 2021 18:42

greizgh force-pushed the seafile branch from 3d3a35e to 0e7d6c3 Compare April 20, 2021 20:26

greizgh force-pushed the seafile branch from 0e7d6c3 to 40e15de Compare April 21, 2021 20:36

SuperSandro2000 reviewed Apr 22, 2021

View reviewed changes

pkgs/tools/networking/ccnet/default.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Apr 22, 2021

View reviewed changes

pkgs/servers/seafile-server/oniguruma.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Apr 22, 2021

View reviewed changes

pkgs/servers/seafile-server/libevhtp.nix Outdated Show resolved Hide resolved

greizgh force-pushed the seafile branch from 40e15de to 03c2646 Compare April 23, 2021 06:40

greizgh added 9 commits October 23, 2021 11:34

django-statici18n: init at 2.0.1

48934ca

django-formtools: init at 2.2

87eaa62

pysearpc: init at 3.2.0

793a6d8

oniguruma: enable posix API

abe6b7e

seafile-shared: prefer commit hash over tag

67979f2

seafile-shared: build CLI client

3860ea7

seafile-server: init at 8.0.7

32e6d67

seahub: init at 8.0.7

22ecb14

nixos/seafile: init service

7b7f3df

greizgh force-pushed the seafile branch from 274f081 to 7b7f3df Compare October 23, 2021 09:39

ofborg bot requested a review from schmittlauch October 23, 2021 09:53

schmittlauch approved these changes Oct 24, 2021

View reviewed changes

SuperSandro2000 merged commit 89a27a2 into NixOS:master Oct 26, 2021

vcunat added a commit to vcunat/nixpkgs that referenced this pull request Nov 7, 2021

nixosTests.seafile: link the committed test

c504b01

It succeeds for me. Apparently this line was forgotten in PR NixOS#119719.

vcunat mentioned this pull request Nov 7, 2021

nixosTests.seafile: link the committed test #144968

Merged

12 tasks

dotlambda reviewed Dec 30, 2021

View reviewed changes

dotlambda mentioned this pull request Dec 31, 2021

python3Packages.seahub: remove #152912

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos/seafile: init service #119719

nixos/seafile: init service #119719

greizgh commented Apr 17, 2021 •

edited

Loading

SuperSandro2000 commented Apr 19, 2021

aanderse left a comment

aanderse Apr 19, 2021

greizgh Apr 20, 2021

greizgh commented Apr 20, 2021

aanderse commented Apr 20, 2021

schmittlauch commented Oct 22, 2021

greizgh commented Oct 23, 2021

Pacman99 commented Nov 17, 2021 •

edited

Loading

qbit commented Dec 2, 2021

greizgh commented Dec 3, 2021 •

edited

Loading

dotlambda Dec 30, 2021 •

edited

Loading

dotlambda Dec 30, 2021 •

edited

Loading

dotlambda Dec 30, 2021

dotlambda commented Dec 30, 2021 •

edited

Loading

dotlambda commented Dec 30, 2021

qbit commented Dec 30, 2021

dotlambda commented Dec 30, 2021

dotlambda commented Dec 30, 2021 •

edited

Loading

greizgh commented Jan 3, 2022

pinpox commented Jan 3, 2022

		@@ -6950,6 +6954,8 @@ in {

		pysdl2 = callPackage ../development/python-modules/pysdl2 { };

		pysearpc = toPythonModule pkgs.libsearpc;

		@@ -0,0 +1,56 @@
		{ stdenv, lib, fetchFromGitHub, python3Packages, makeWrapper }:

		python3Packages.buildPythonPackage rec {

nixos/seafile: init service #119719

nixos/seafile: init service #119719

Conversation

greizgh commented Apr 17, 2021 • edited Loading

Motivation for this change

Things done

SuperSandro2000 commented Apr 19, 2021

aanderse left a comment

Choose a reason for hiding this comment

aanderse Apr 19, 2021

Choose a reason for hiding this comment

greizgh Apr 20, 2021

Choose a reason for hiding this comment

greizgh commented Apr 20, 2021

aanderse commented Apr 20, 2021

schmittlauch commented Oct 22, 2021

greizgh commented Oct 23, 2021

Pacman99 commented Nov 17, 2021 • edited Loading

qbit commented Dec 2, 2021

greizgh commented Dec 3, 2021 • edited Loading

dotlambda Dec 30, 2021 • edited Loading

Choose a reason for hiding this comment

dotlambda Dec 30, 2021 • edited Loading

Choose a reason for hiding this comment

dotlambda Dec 30, 2021

Choose a reason for hiding this comment

dotlambda commented Dec 30, 2021 • edited Loading

dotlambda commented Dec 30, 2021

qbit commented Dec 30, 2021

dotlambda commented Dec 30, 2021

dotlambda commented Dec 30, 2021 • edited Loading

greizgh commented Jan 3, 2022

pinpox commented Jan 3, 2022

greizgh commented Apr 17, 2021 •

edited

Loading

Pacman99 commented Nov 17, 2021 •

edited

Loading

greizgh commented Dec 3, 2021 •

edited

Loading

dotlambda Dec 30, 2021 •

edited

Loading

dotlambda Dec 30, 2021 •

edited

Loading

dotlambda commented Dec 30, 2021 •

edited

Loading

dotlambda commented Dec 30, 2021 •

edited

Loading