Add a validator for git / http sources by orta · Pull Request #137 · CocoaPods/trunk.cocoapods.org

orta · 2015-06-04T21:38:56Z

This is some of the initial work around #136

I'm struggling with figuring out how to test this and whether it is safe. Assigning to @alloy to give a one over.

segiddins · 2015-06-04T21:59:10Z

please, no subshells

I couldn't think of a way to do the git lint without a subtle

system(‘curl’, ‘-IL’, @specification.source[:http], ‘-f’)

On Jun 4, 2015, at 5:44 PM, Orta notifications@github.com wrote:

In app/models/specification_wrapper.rb #137 (comment):

@@ -48,6 +49,20 @@ def validation_errors
results
end

def validate_public_access

return validate_http if @specification.source[:http]

return validate_git if @specification.source[:git]

true

end

def validate_http

system("curl -IL #{ Shellwords.escape @specification.source[:http] } -f")
I couldn't think of a way to do the git lint without a subtle

—
Reply to this email directly or view it on GitHub https://github.com/CocoaPods/trunk.cocoapods.org/pull/137/files#r31780251.

also, will want to redirect out & err to /dev/null

Why not just use validate_url from Core? https://github.com/CocoaPods/Core/blob/816eda9f90dd8fc54901b0fd4b768691fc1295bf/lib/cocoapods-core/http.rb#L35-39

segiddins · 2015-06-05T00:48:26Z

I can write up specs tomorrow, if you'd like.
I also want to know how @floere feels about doing even more stuff in a synchronous request.

floere · 2015-06-05T16:04:56Z

@segiddins I can search my feelings ;) Not sure what problem it solves though. Help?

segiddins · 2015-06-05T16:07:24Z

It makes sure pods in trunk are actually public ally accessible before allowing them to be pushed.

-Samuel E. Giddins

On Jun 5, 2015, at 9:04 AM, Florian R. Hanke notifications@github.com wrote:

@segiddins I can search my feelings ;) Not sure what problem it solves though. Help?

—
Reply to this email directly or view it on GitHub.

floere · 2015-06-05T16:22:13Z

Ah thanks. Seems weird we haven't thought of this right when writing trunk.

The current code probably needs more error handling (or rather, status codes checking) and better - and more precise - info for the user so they can act appropriately.

Is it time yet for asynchronous processing of trunk pushes and reporting the running jobs on the command line?
Something like:

Checking if publicly available? ... Yes! (<repo url>)

segiddins · 2015-06-05T16:27:25Z

The only reason I see to do that is the heroku timeout... It's ok to me if the actual push takes a bit from the users perspective.

-Samuel E. Giddins

On Jun 5, 2015, at 9:22 AM, Florian R. Hanke notifications@github.com wrote:

Ah thanks. Seems weird we haven't thought of this right when writing trunk.

The current code probably needs more error handling (or rather, status codes checking) and better - and more precise - info for the user so they can act appropriately.

Is it time yet for asynchronous processing of trunk pushes and reporting the running jobs on the command line?
Something like:

Checking if publicly available? ... Yes! ()
—
Reply to this email directly or view it on GitHub.

floere · 2015-06-05T16:30:48Z

@segiddins We can set the Heroku timeout, should that be a problem.

segiddins · 2015-06-05T16:38:50Z

@floere pretty sure it's bounded at 30 seconds

floere · 2015-06-05T16:58:33Z

@segiddins True: https://devcenter.heroku.com/articles/http-routing#timeouts

alloy · 2015-06-08T10:32:39Z

Seems weird we haven't thought of this right when writing trunk.

Trunk was really only interested in validating data it needed to be able to do its work, ensuring the rest of the podspec data is valid was upto the client.

floere · 2015-06-08T10:34:33Z

@alloy Not saying it needs to do it – saying it's weird we haven't considered the case (at least I can't remember).

floere · 2015-06-08T10:36:56Z

@segiddins @orta @alloy If the long-runny-ness of this becomes an issue, I see two solutions: background jobs/multi-request (the client periodically gets the latest state – good if we have many requests for trunk), or response streaming (updating the info the client has as we go – good if we don't have many requests for trunk). Putting this here so I don't forget.

floere · 2015-06-08T10:37:44Z

@alloy Would you then prefer this verification be in the client?

alloy · 2015-06-08T10:46:46Z

Yeah I did consider it then, but left it to the client. Clearly that didn't turn out to be enough, probably because people push pods from private repos that their machines have access to at the time of validating :)

floere · 2015-06-08T10:58:37Z

@alloy Ah yes… good point. I do remember that we discussed that trunk act as a guard for ok pods – and that the client only does checks to provide a responsive/comfortable UI. According to that line of thought, this code would need to be in trunk.

alloy · 2015-06-08T11:04:01Z

Yeah I think it’s fine to do this now in Trunk. I think it would just be good to keep these things to a minimum and not require an extra queue :)

floere · 2015-06-08T11:35:20Z

@alloy We're both from the "do the minimum that works" school :)

alloy · 2015-06-08T11:48:09Z

@floere Yeah I know, it was a general statement ;)

orta · 2015-06-11T19:59:54Z

Got a sense of what I think it would just be good to keep these things to a minimum and not require an extra queue means here?

What do I need to do to get this wrapped up?

alloy · 2015-06-11T20:40:14Z

I think it would be a good idea to add a timeout block around the call and log how long it took to perform, then you an fine tune it after it's in use.

I'd also urge you to not use a shell at all, but instead call open/spawn with an array of arguments. That way there can never be any illicit command execution.

orta · 2015-07-01T08:46:41Z

OK, this PR now has everything that people have asked for 👍 - I'm still unsure how to test this, any guesses?

alloy · 2015-07-01T12:46:10Z

I’d just mock the call to Pod::HTTP.validate_url, expect the right argument, and return true/false, it’s implementation is already tested in the CP gem, and add an expectation for the call to system as well?

orta · 2015-07-01T17:06:44Z

Alright, this is 👍

segiddins · 2015-07-01T17:07:50Z

Why not take an &blk arg so you don't have to create explicit procs?

not sure how to do that, and the proc worked for QIs earlier /shrug

This would be a bit nicer yeah, not a show stopper, though. @segiddins can you add it, if you feel strongly about it?

segiddins · 2015-07-01T17:08:43Z

shouldn't that be @specification.source[:tag] || @specification.source[:commit] || @specification.source[:branch] || 'HEAD' ?

Given that it's trunk, it can only be :tag, and I'm validating it exists pod spec lint will have checked that the tag exists by this point.

Given that it's trunk, it can only be :tag, and I'm validating the repo exists - pod spec lint will have checked that the tag exists by this point.

segiddins · 2015-07-05T20:13:28Z

@alloy can you give this a final look?

alloy · 2015-07-05T22:20:18Z

LGTM 👍

segiddins · 2015-07-06T00:43:18Z

I'll make my few style changes, then merge tonight. Thanks @orta :)

Add a validator for git / http sources

alloy · 2015-07-06T10:54:00Z

Nice work, fellas 👍

orta · 2015-07-06T10:58:21Z

Thanks sir

orta assigned alloy Jun 4, 2015

orta force-pushed the validate_spec_source branch from 04dcc4e to 3f27ae9 Compare June 4, 2015 21:39

segiddins reviewed Jun 4, 2015
View reviewed changes

orta force-pushed the validate_spec_source branch 3 times, most recently from acd14ee to 8ab7a4e Compare July 1, 2015 16:28

segiddins reviewed Jul 1, 2015
View reviewed changes

orta and others added 8 commits July 5, 2015 18:20

Initial work on adding a validator for git / http sources

25ee1d6

Use CocoaPods HTTP and ensure git verification system can't be hijacked

bf0cbc2

[SpecificationWrapper] rename validation method

edb6fdd

Add support for timing out the linter calls

5e56017

[rubocop] Appease the rubocop

da41f58

Add tests

622319b

Addresses @segiddins' feedback

0204472

[SpecificationWrapper] Use block argument to wrap_timeout

95914cc

segiddins force-pushed the validate_spec_source branch from 98deb0a to 95914cc Compare July 6, 2015 01:23

segiddins added a commit that referenced this pull request Jul 6, 2015

Merge pull request #137 from CocoaPods/validate_spec_source

e065632

Add a validator for git / http sources

segiddins merged commit e065632 into master Jul 6, 2015

segiddins deleted the validate_spec_source branch July 6, 2015 01:28

orta mentioned this pull request Jul 21, 2015

No error when http source is not found CocoaPods/CocoaPods#3856

Closed

Conversation

orta commented Jun 4, 2015

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

segiddins commented Jun 5, 2015

Uh oh!

floere commented Jun 5, 2015

Uh oh!

segiddins commented Jun 5, 2015

Uh oh!

floere commented Jun 5, 2015

Uh oh!

segiddins commented Jun 5, 2015

Uh oh!

floere commented Jun 5, 2015

Uh oh!

segiddins commented Jun 5, 2015

Uh oh!

floere commented Jun 5, 2015

Uh oh!

alloy commented Jun 8, 2015

Uh oh!

floere commented Jun 8, 2015

Uh oh!

floere commented Jun 8, 2015

Uh oh!

floere commented Jun 8, 2015

Uh oh!

alloy commented Jun 8, 2015

Uh oh!

floere commented Jun 8, 2015

Uh oh!

alloy commented Jun 8, 2015

Uh oh!

floere commented Jun 8, 2015

Uh oh!

alloy commented Jun 8, 2015

Uh oh!

orta commented Jun 11, 2015

Uh oh!

alloy commented Jun 11, 2015

Uh oh!

orta commented Jul 1, 2015

Uh oh!

alloy commented Jul 1, 2015

Uh oh!

orta commented Jul 1, 2015

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

segiddins commented Jul 5, 2015

Uh oh!

alloy commented Jul 5, 2015

Uh oh!

segiddins commented Jul 6, 2015

Uh oh!

alloy commented Jul 6, 2015

Uh oh!

orta commented Jul 6, 2015