Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multiple execution of checksum calculation in Specification #7854

Closed
Whirlwind opened this issue Jun 20, 2018 · 20 comments · Fixed by CocoaPods/Core#454
Closed

Multiple execution of checksum calculation in Specification #7854

Whirlwind opened this issue Jun 20, 2018 · 20 comments · Fixed by CocoaPods/Core#454
Labels
d1:easy An easy ticket that is a good start for first-time contributors help wanted Help from new or existing contributors would be greatly appreciated!

Comments

@Whirlwind
Copy link
Contributor

When I check the code from Specification, I found the checksum will be called many times. I try to add some code to debug it:

    def checksum
      unless defined_in_file.nil?
        require 'digest'
        checksum = Digest::SHA1.hexdigest(File.read(defined_in_file))
        checksum = checksum.encode('UTF-8') if checksum.respond_to?(:encode)
        puts "object: #{object_id} name: #{name} spec: #{defined_in_file} checksum: #{checksum}" if self.class.root_name(name) == 'AFNetworking'
        checksum
      end
    end

get the output:

Comparing resolved specification to the sandbox manifest
object: 70162289831340 name: AFNetworking spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
......
  - Writing Xcode project file to `Pods/Pods.xcodeproj`
object: 70162289831340 name: AFNetworking spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830040 name: AFNetworking/NSURLConnection spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289829600 name: AFNetworking/NSURLSession spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830360 name: AFNetworking/Reachability spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830680 name: AFNetworking/Security spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289831080 name: AFNetworking/Serialization spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
  - Writing Lockfile in `Podfile.lock`
  - Writing Manifest in `Pods/Manifest.lock`
  - Writing independent Lockfile in   - Writing independent Lockfile in /Users/whirlwind/Documents/Developer/TestApp/Podfile.lock
object: 70162289831340 name: AFNetworking spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830040 name: AFNetworking/NSURLConnection spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289829600 name: AFNetworking/NSURLSession spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830360 name: AFNetworking/Reachability spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830680 name: AFNetworking/Security spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289831080 name: AFNetworking/Serialization spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289831340 name: AFNetworking spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830040 name: AFNetworking/NSURLConnection spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289829600 name: AFNetworking/NSURLSession spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830360 name: AFNetworking/Reachability spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830680 name: AFNetworking/Security spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289831080 name: AFNetworking/Serialization spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289831340 name: AFNetworking spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830040 name: AFNetworking/NSURLConnection spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289829600 name: AFNetworking/NSURLSession spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830360 name: AFNetworking/Reachability spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830680 name: AFNetworking/Security spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289831080 name: AFNetworking/Serialization spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289831340 name: AFNetworking spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830040 name: AFNetworking/NSURLConnection spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289829600 name: AFNetworking/NSURLSession spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830360 name: AFNetworking/Reachability spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830680 name: AFNetworking/Security spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289831080 name: AFNetworking/Serialization spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289831340 name: AFNetworking spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830040 name: AFNetworking/NSURLConnection spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289829600 name: AFNetworking/NSURLSession spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830360 name: AFNetworking/Reachability spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830680 name: AFNetworking/Security spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289831080 name: AFNetworking/Serialization spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289831340 name: AFNetworking spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830040 name: AFNetworking/NSURLConnection spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289829600 name: AFNetworking/NSURLSession spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830360 name: AFNetworking/Reachability spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830680 name: AFNetworking/Security spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289831080 name: AFNetworking/Serialization spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289831340 name: AFNetworking spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830040 name: AFNetworking/NSURLConnection spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289829600 name: AFNetworking/NSURLSession spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830360 name: AFNetworking/Reachability spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289830680 name: AFNetworking/Security spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0
object: 70162289831080 name: AFNetworking/Serialization spec: /Users/whirlwind/.cocoapods/repos/publicthird_binary_repo/AFNetworking/2.5.4.1-binary/AFNetworking.podspec.json checksum: 89a70bcf75991409d3fa476fa3bfaf4e283aa1b0

I think we could cache the checksum by the path.
@dnkoutso
Copy link
Contributor

what is the stacktrace? which methods calls this many times?

@dnkoutso
Copy link
Contributor

you can put puts caller to print out the stacktrace. It will probably print many times

@Whirlwind
Copy link
Contributor Author

I am very sorry.... I found that there is a plugin to compare the checksum in my project, and it will cause the checksum be called more times.

It is not a bug, but I still hope CP could use attr_accessor or others way to cache the checksum result:

    attr_accessor :checksum
    def checksum
      @checksum ||= begin
        if root?
          unless defined_in_file.nil?
            require 'digest'
            checksum = Digest::SHA1.hexdigest(File.read(defined_in_file))
            checksum = checksum.encode('UTF-8') if checksum.respond_to?(:encode)
            checksum
          end
        else
          root.checksum
        end
      end
    end

It does not perform many computations, include the subspecs.

@dnkoutso
Copy link
Contributor

We want to move away from attr_accessor attributes to avoid having global state leak across the entire installation process. This could be an OK change, especially if it gets called multiple times outside of your plugin.

You can cache the result without an attr_accessor

@dnkoutso
Copy link
Contributor

Alternatively you can improve the plugin so it doesn't invoke this many times.

@Whirlwind
Copy link
Contributor Author

I can cache the checksum in my plugin. But I could not void to skip the first caller. It meant that I must get the checksum at first then cache it. It is still called once.

On the another hand, the subspec should get the checksum from the root spec, right?

@dnkoutso
Copy link
Contributor

dnkoutso commented Jun 20, 2018 •
edited

it does not matter. It seems its reading the file for the checksum see

checksum = Digest::SHA1.hexdigest(File.read(defined_in_file)) # see `defined_in_file`

So whether its called from subspec or root spec its the same file.

@dnkoutso
Copy link
Contributor

@Whirlwind you cannot avoid the first caller right? Even if its cached the first time will have to do the expensive part once.

Do you still want this issue open?

@Whirlwind
Copy link
Contributor Author

Right，it is the same file. But it will cause many times read the file and get the checksum.

@Whirlwind
Copy link
Contributor Author

We want to move away from attr_accessor attributes to avoid having global state leak across the entire installation process

Could you tell the reason?

@dnkoutso
Copy link
Contributor

you are right that we could optimize it for subspec to delegate to the root value and cache that (basically your change).

I think its best to make a PR for this.

@Whirlwind
Copy link
Contributor Author

If you agree with me, I will make a PR based my change.

@Whirlwind
Copy link
Contributor Author

Maybe you have a better way to cache it.

@dnkoutso
Copy link
Contributor

dnkoutso commented Jun 20, 2018 •
edited

You can do your change without an attr_accessor. It should just work.

@dnkoutso dnkoutso added d1:easy An easy ticket that is a good start for first-time contributors help wanted Help from new or existing contributors would be greatly appreciated! labels Jun 20, 2018
@Whirlwind
Copy link
Contributor Author

Whirlwind commented Jun 20, 2018 •
edited by dnkoutso 

    def checksum
      @checksum ||= begin
        if root?
          unless defined_in_file.nil?
            require 'digest'
            checksum = Digest::SHA1.hexdigest(File.read(defined_in_file))
            checksum = checksum.encode('UTF-8') if checksum.respond_to?(:encode)
            checksum
          end
        else
          root.checksum
        end
      end
    end

Like this?

@dnkoutso
Copy link
Contributor

yes! I think that should do it.

@Whirlwind
Copy link
Contributor Author

OK, I will make a PR later.

@Whirlwind
Copy link
Contributor Author

Should the checksum in Podfile be changed like the Specification?

@dnkoutso dnkoutso mentioned this issue Jun 20, 2018
Merged
@Whirlwind
Copy link
Contributor Author

Thanks for the PR.

@dnkoutso
Copy link
Contributor

It only gets invoked twice...but sure I dont think its harmful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
d1:easy An easy ticket that is a good start for first-time contributors help wanted Help from new or existing contributors would be greatly appreciated!
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Cache result of Specification#checksum and Podfile#checksum methods dnkoutso/Core
2 participants
@dnkoutso @Whirlwind