Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Syncthing NixOS module jq based config script doesn't merge folders correctly #230146

Closed
doronbehar opened this issue May 5, 2023 · 0 comments · Fixed by #230196
Closed

Syncthing NixOS module jq based config script doesn't merge folders correctly #230146

doronbehar opened this issue May 5, 2023 · 0 comments · Fixed by #230196
Labels
0.kind: bug

Comments

@doronbehar
Copy link
Contributor

Describe the bug

I'm experiencing the issue for a long time now, and it was really hard to spot and debug this issue. I'm talking about this line:

nixpkgs/nixos/modules/services/networking/syncthing.nix

Line 59 in 3fbdad0

"folders": (${builtins.toJSON folders}${optionalString (cfg.folders == {} || ! cfg.overrideFolders) " + .folders"})

Steps To Reproduce

It's kind of hard to give instructions how to reproduce the issue. In my case, whenever the merge-syncthing-config dash script runs, every folder I already have configured, fails to update. The whole configuration thus fails to be uploaded to the server. I can inspect the new_cfg and the old_cfg by editing that dash script, and I can see that I get folders twice in the json arrays. Hence I'm not surprised I see the following error in syncthing's web interface:

image

Expected behavior

I expect the merge script to not POST a JSON array with no duplicate folders IDs.

Proposed solution

Transition most of the logic from jq to bash, for easier maintenance, and use /rest/config/folders and /rest/config/devices API, and use -X PATCH when overrideFolders == false.

I'm working on a solution at the moment but I'd like to here your comments first.

Notify maintainers

cc @Lassulus who wrote the original dash script, other people have touched that line ever since, @raindev was the latest.

Metadata

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.26, NixOS, 23.05 (Stoat), 23.05.20230502.1a411f2`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.3`
 - nixpkgs: `/nix/store/ycwxxfi2yv4nd8y66bxdc8pww9mhjwgj-source`
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue May 5, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
eaac1e0 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support, and support for setting
settings per individual folders and devices via ID in the URL.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue May 5, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
b83d9a4 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support, and support for setting
settings per individual folders and devices via ID in the URL.

Hence this commit fixes NixOS#230146.
@doronbehar doronbehar mentioned this issue May 5, 2023
Closed
12 tasks
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue May 5, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
fcf7549 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
@doronbehar doronbehar mentioned this issue May 5, 2023
Merged
12 tasks
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue May 16, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
c562b96 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue May 16, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
05f9a73 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue May 17, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
44036d4 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue May 18, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
44c831d 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue Jun 8, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
d9d6dfd 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue Jun 18, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
5f5fa9e 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue Jun 29, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
badf9cf 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue Jun 30, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
0425aa9 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue Jul 17, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
e7a3403 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue Jul 22, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
7e143bd 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue Jul 22, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
a5ff056 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue Jul 23, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
9b5f283 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue Jul 23, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
33a8f1c 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue Jul 23, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
794be08 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue Jul 23, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
047fa8d 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via Nix / Bash loop the devices and folders IDs and merges the keys
using upstream's `curl -X POST` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue Jul 23, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
4b3cb79 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
doronbehar added a commit to doronbehar/nixpkgs that referenced this issue Jul 23, 2023
@doronbehar
nixos/syncthing: Use API to merge / override configurations
97137d0 
If one sets either of `override{Device,folder}s` to false, the jq `*`
operator doesn't merge well the devices and folders, creating duplicate
IDs for folders as observed in NixOS#230146. This PR makes the script iterate
via a Bash for loop the devices and folders IDs and merges the keys
using upstream's `curl -X PATCH` support for single objects.

Hence this commit fixes NixOS#230146.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
0.kind: bug
Projects
None yet
Milestone
No milestone
1 participant
@doronbehar