Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TypeError: Converting circular structure to JSON #327

Closed
3 tasks done
mzaferyahsi opened this issue Sep 11, 2023 · 16 comments
Closed
3 tasks done

TypeError: Converting circular structure to JSON #327

mzaferyahsi opened this issue Sep 11, 2023 · 16 comments
Labels
status(previewed) This issue or pull request should be fixed in a released beta version

Comments

@mzaferyahsi
Copy link

Checklist before opening an issue

Describe the bug

App crashes right after the sign in.

Error code

No error code is generated.

Welcome to icloud-photos-sync, v.1.2.0-nightly.25!
Made with <3 by steilerDev
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Authenticating user...
Device trusted
Sign in successful!
file:///root/.nvm/versions/node/v18.17.1/lib/node_modules/icloud-photos-sync/bin/app/event/error-handler.js:80
                    return Object.assign(data, JSON.parse(ErrorHandler.maskConfidentialData(JSON.stringify(data))));
                                                                                                 ^

TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'ClientRequest'
    |     property 'socket' -> object with constructor 'TLSSocket'
    --- property '_httpMessage' closes the circle
    at JSON.stringify (<anonymous>)
    at Object.beforeSend (file:///root/.nvm/versions/node/v18.17.1/lib/node_modules/icloud-photos-sync/bin/app/event/error-handler.js:80:98)
    at BacktraceClient.generateSubmissionData (/root/.nvm/versions/node/v18.17.1/lib/node_modules/icloud-photos-sync/node_modules/@backtrace-labs/sdk-core/lib/BacktraceCoreClient.js:142:29)
    at BacktraceClient.send (/root/.nvm/versions/node/v18.17.1/lib/node_modules/icloud-photos-sync/node_modules/@backtrace-labs/sdk-core/lib/BacktraceCoreClient.js:110:36)
    at ErrorHandler.reportError (file:///root/.nvm/versions/node/v18.17.1/lib/node_modules/icloud-photos-sync/bin/app/event/error-handler.js:298:29)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async ErrorHandler.handleError (file:///root/.nvm/versions/node/v18.17.1/lib/node_modules/icloud-photos-sync/bin/app/event/error-handler.js:108:27)
    at async file:///root/.nvm/versions/node/v18.17.1/lib/node_modules/icloud-photos-sync/bin/main.js:24:5

Node.js v18.17.1

Logs

[2023-09-11T09:25:45.103Z] DEBUG EventManager: Registering listener for event log-info from source LogInterface
[2023-09-11T09:25:45.103Z] DEBUG EventManager: Registering listener for event log-warn from source LogInterface
[2023-09-11T09:25:45.103Z] DEBUG EventManager: Registering listener for event warn-count_mismatch from source LogInterface
[2023-09-11T09:25:45.103Z] DEBUG EventManager: Registering listener for event warn-filetype_error from source LogInterface
[2023-09-11T09:25:45.103Z] DEBUG EventManager: Registering listener for event warn-library_load_error from source LogInterface
[2023-09-11T09:25:45.104Z] DEBUG EventManager: Registering listener for event warn-extraneous_file from source LogInterface
[2023-09-11T09:25:45.104Z] DEBUG EventManager: Registering listener for event warn-icloud_load_error from source LogInterface
[2023-09-11T09:25:45.104Z] DEBUG EventManager: Registering listener for event warn-write_asset_error from source LogInterface
[2023-09-11T09:25:45.104Z] DEBUG EventManager: Registering listener for event warn-write_album_error from source LogInterface
[2023-09-11T09:25:45.104Z] DEBUG EventManager: Registering listener for event warn-link_error from source LogInterface
[2023-09-11T09:25:45.104Z] DEBUG EventManager: Registering listener for event warn-mfa_error from source LogInterface
[2023-09-11T09:25:45.104Z] DEBUG EventManager: Registering listener for event warn-resource_file_error from source LogInterface
[2023-09-11T09:25:45.104Z] DEBUG EventManager: Registering listener for event warn-archive_asset_error from source LogInterface
[2023-09-11T09:25:45.104Z] DEBUG EventManager: Registering listener for event sync-retry from source LogInterface
[2023-09-11T09:25:45.104Z] DEBUG EventManager: Registering listener for event log-error from source LogInterface
[2023-09-11T09:25:45.104Z] DEBUG EventManager: Registering listener for event error-handled from source LogInterface
[2023-09-11T09:25:45.107Z] DEBUG EventManager: Registering listener for event warn-mfa_error from source CLIInterface
[2023-09-11T09:25:45.107Z] DEBUG EventManager: Registering listener for event warn-filetype_error from source CLIInterface
[2023-09-11T09:25:45.107Z] DEBUG EventManager: Registering listener for event warn-resource_file_error from source CLIInterface
[2023-09-11T09:25:45.107Z] DEBUG EventManager: Registering listener for event error-handled from source CLIInterface
[2023-09-11T09:25:45.107Z] DEBUG EventManager: Registering listener for event icloud-auth_started from source CLIInterface
[2023-09-11T09:25:45.107Z] DEBUG EventManager: Registering listener for event icloud-auth_done from source CLIInterface
[2023-09-11T09:25:45.107Z] DEBUG EventManager: Registering listener for event icloud-mfa_req from source CLIInterface
[2023-09-11T09:25:45.107Z] DEBUG EventManager: Registering listener for event icloud-trusted from source CLIInterface
[2023-09-11T09:25:45.107Z] DEBUG EventManager: Registering listener for event icloud-account_ready from source CLIInterface
[2023-09-11T09:25:45.107Z] DEBUG EventManager: Registering listener for event icloud-session_expired from source CLIInterface
[2023-09-11T09:25:45.107Z] DEBUG EventManager: Registering listener for event mfa-started from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event mfa-resend from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event mfa-received from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event mfa-not_provided from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event photos-setup-complete from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event photos-ready from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event token from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event scheduled from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event scheduled-done from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event scheduled-retry from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event sync-start from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event sync-fetch_n_load from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event sync-fetch_n_load_completed from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event sync-diff from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event sync-diff_completed from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event sync-write from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event sync-write_assets from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event sync-write_asset_completed from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event warn-write_asset_error from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event sync-write_assets_completed from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event sync-write_albums from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event sync-write_albums_completed from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event sync-write_completed from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event sync-done from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event sync-retry from source CLIInterface
[2023-09-11T09:25:45.108Z] DEBUG EventManager: Registering listener for event archive-start from source CLIInterface
[2023-09-11T09:25:45.109Z] DEBUG EventManager: Registering listener for event archive-persist_start from source CLIInterface
[2023-09-11T09:25:45.109Z] DEBUG EventManager: Registering listener for event archive-remote_delete from source CLIInterface
[2023-09-11T09:25:45.109Z] DEBUG EventManager: Registering listener for event archive-done from source CLIInterface
[2023-09-11T09:25:45.121Z] DEBUG EventManager: Registering listener for event photos-ready from source iCloud
[2023-09-11T09:25:45.121Z] DEBUG EventManager: Registering listener for event mfa-not_provided from source iCloud
[2023-09-11T09:25:45.121Z] DEBUG EventManager: Registering listener for event icloud-error from source iCloud
[2023-09-11T09:25:45.121Z] DEBUG EventManager: Registering listener for event mfa-error from source iCloud
[2023-09-11T09:25:45.122Z] INFO iCloud: Authenticating user
[2023-09-11T09:25:45.123Z] DEBUG ResourceManager: Reading resource file from /nfs/yahsi/mzaferyahsi/.icloud-photos-sync
[2023-09-11T09:25:46.061Z] DEBUG NetworkManager: Setting scnt header to 
[2023-09-11T09:25:46.061Z] DEBUG NetworkManager: Setting session secret to 
[2023-09-11T09:25:46.062Z] DEBUG iCloud: Acquired signin secrets
[2023-09-11T09:25:46.062Z] DEBUG iCloud: Response status is 200, authentication successful - device trusted
[2023-09-11T09:25:46.062Z] DEBUG ResourceManager: Reading resource file from /nfs/yahsi/mzaferyahsi/.icloud-photos-sync
[2023-09-11T09:25:46.063Z] INFO iCloud: Setting up iCloud connection
[2023-09-11T09:25:46.063Z] DEBUG ResourceManager: Reading resource file from /nfs/yahsi/mzaferyahsi/.icloud-photos-sync
[2023-09-11T09:25:48.465Z] DEBUG NetworkManager: Setting photosUrl to https://p122-ckdatabasews.icloud.com:443
[2023-09-11T09:25:48.466Z] DEBUG iCloud: Account ready
[2023-09-11T09:25:48.466Z] INFO iCloud: Getting iCloud Photos Service ready
[2023-09-11T09:25:48.466Z] DEBUG iCloudPhotos: Getting iCloud Photos account information
[2023-09-11T09:25:48.833Z] DEBUG NetworkManager: Settling rate limiter queue...
[2023-09-11T09:25:48.833Z] DEBUG NetworkManager: Queue has settled!
[2023-09-11T09:25:48.834Z] DEBUG NetworkManager: Settling CCY limiter queue...
[2023-09-11T09:25:48.834Z] DEBUG NetworkManager: Queue has settled!
[2023-09-11T09:25:48.834Z] DEBUG EventManager: Removed 3 listeners for source iCloudPhotos
[2023-09-11T09:25:48.834Z] DEBUG EventManager: No more listeners for source iCloudPhotos registered
[2023-09-11T09:25:48.834Z] DEBUG EventManager: Removed 11 listeners for source iCloud
[2023-09-11T09:25:48.834Z] DEBUG EventManager: No more listeners for source iCloud registered
[2023-09-11T09:25:48.838Z] INFO ErrorHandler: Handling error APP_SYNC caused by VALIDATOR_PHOTOS_SETUP_RESPONSE

Operating environment

  • OS: Ubuntu 22.04.3 LTS (GNU/Linux 5.15.0-79-generic x86_64)
  • Version: v.1.2.0-nightly.25
  • Execution environment: Node.js v18.17.1
@mzaferyahsi mzaferyahsi mentioned this issue Sep 11, 2023
Closed
3 tasks
@steilerDev
Copy link
Owner

Try turning error reporting off for this - need to make sure to handle masking confidential data differently it seems.

@mzaferyahsi
Copy link
Author

After doing some analysing, I noticed that I have three zones;

[
  {
    zoneID: {
      zoneName: 'PrimarySync',
      ownerRecordName: '_5d8a4591d58a4fe577090ceb96ec9233',
      zoneType: 'REGULAR_CUSTOM_ZONE'
    }
  },
  {
    zoneID: {
      zoneName: 'SharedSync-1F27FFE9-AB6C-4F4C-B970-EB46CBA18B16',
      ownerRecordName: '_5d8a4591d58a4fe577090ceb96ec9233',
      zoneType: 'REGULAR_CUSTOM_ZONE'
    }
  },
  {
    zoneID: {
      zoneName: 'CMM-6A3D1806-A29C-44C0-A710-5BDDE8681D61',
      ownerRecordName: '_5d8a4591d58a4fe577090ceb96ec9233',
      zoneType: 'REGULAR_CUSTOM_ZONE'
    }
  }
]

@steilerDev
Copy link
Owner

Very interesting...do you have any special setup in iCloud that could explain the third zone?

@mzaferyahsi
Copy link
Author

I have a family account with my wife, nothing special. But I do have some shared albums with other icloud accounts (like my sister, and other friends). That's the only thing I can think of.

@mzaferyahsi
Copy link
Author

Is there a way to check what's inside in that third zone?

@steilerDev
Copy link
Owner

You could try the postman collection within the repo - there are basically all functions in there, you'd need to swap the zone name and owner to retrieve the data from the zone

@mzaferyahsi
Copy link
Author

I can only run the Get Photos by Start Rank and it returns two records.

One with "recordType": "CPLMaster", which is a video that is actually in my shared library when I look via the Photos app on my Mac and the other is "recordType": "CPLAsset", linked to the CPLMaster with the following master reference value.

"masterRef": {
                    "value": {
                        "recordName": "AfbwGCXra8EQRu0L6bgxm9R+vTLH",
                        "action": "DELETE_SELF",
                        "zoneID": {
                            "zoneName": "CMM-6A3D1806-A29C-44C0-A710-5BDDE8681D61",
                            "ownerRecordName": "_5d8a4591d58a4fe577090ceb96ec9233",
                            "zoneType": "REGULAR_CUSTOM_ZONE"
                        }
                    },
                    "type": "REFERENCE"
                },
...
"parent": {
                "recordName": "cmm-root"
            }

One special thing I can think of is that my iCloud space is full. That's why the rush on using the tool to backup the images so I can open up some space.

@steilerDev
Copy link
Owner

I see - so it feels like Dave to ignore?

You should be able to alter the photos response Schema (bin
/lib/resources/schemas/photos-setup-response.json) to temporarily allow the array to be bigger - the code should be able to only pick up the primary and shared zone.

Currently don't have access to a laptop, so it will be only next week when I can implement the changes.

@mzaferyahsi
Copy link
Author

I did download the repo and modified the code to confirm, indeed it works when I increase the limit. The problem is that that video is still a valid video that I want to backup. I downloaded it and backed it up and deleted from iCloud manually for now. Let's see if the zone is deleted afterwards. It might be good to log additional zones are created at least to be able to track if there are any valuable assets that needs to be backed up.

@steilerDev
Copy link
Owner

The question would be if the video is included in any of the other zones - and just a duplicate indicating full storage - but yeah, a warning might be a good idea.

@mzaferyahsi
Copy link
Author

So, it seems that I've shared this video with a friend via iMessage but instead of directly sending it, I've shared it as a link as it seems. On the iCloud Photos Web Interface I see iCloud Links and it only contains this video. We can treat it as a duplicate, though deleting the one from my library didn't delete this one and I had to manually search where I shared this video.

@vguerci
Copy link

vguerci commented Sep 15, 2023 •
edited

If that can help, I had a similar error, having two SharedSync, but with a different reason.

The second one is marked as deleted this way:

        {
            "zoneID": {
                "zoneName": "SharedSync-ABC123",
                "ownerRecordName": "_abc123",
                "zoneType": "REGULAR_CUSTOM_ZONE"
            },
            "deleted": true
        }

Calling any API with this zoneName returns ZONE_NOT_FOUND errors.

Postman collection can be adapted with if(zones[i].zoneID.zoneName.startsWith(library) && !zones[i].deleted) {

I did look into code, but sorry I'm a bit puzzled by the validation logic to be confident at submitting a PR. For now just naively increased the maxItems to 3, as suggested in the thread (thanks!), which I guess works as the deleted is the last.

@steilerDev
Copy link
Owner

@vguerci are you sure about the structure? I'd expect

 {
    "zoneID": {
        "zoneName": "SharedSync-ABC123",
        "ownerRecordName": "_abc123",
        "zoneType": "REGULAR_CUSTOM_ZONE"
        "deleted": true
    },
}

If not could you please provide the full response object to https://setup.icloud.com/setup/ws/1/accountLogin?

P.S.: Validation is based on the schema that is generated from src/lib/resources/network-types.ts - extraction happens in src/lib/icloud/icloud.ts#setupAccount()

@steilerDev
Copy link
Owner

Nevermind - I missed something here, your example makes sense.

steilerDev added a commit that referenced this issue Sep 17, 2023
@steilerDev
fix: Allow additional and deleted zones
e58bfe9 
Fixes #327
@github-actions
Copy link

This issue should be resolved with version v1.2.0-beta.5, please confirm.

@github-actions github-actions bot added the status(previewed) This issue or pull request should be fixed in a released beta version label Sep 17, 2023
@vguerci
Copy link

vguerci commented Sep 18, 2023

Works for me, will report if I find any other issues, thanks :)

And yes I confirm my example, that was just a copy-paste excerpt of response.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
status(previewed) This issue or pull request should be fixed in a released beta version
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@vguerci @mzaferyahsi @steilerDev