Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Git clone issue. #1519

Closed
7Two1 opened this issue Mar 26, 2013 · 17 comments
Closed

Git clone issue. #1519

7Two1 opened this issue Mar 26, 2013 · 17 comments

Comments

@7Two1
Copy link

7Two1 commented Mar 26, 2013

Initial clone under windows...

git.exe clone --progress -v "git://github.com/ginatrapani/ThinkUp.git" "S:_incoming\Internet\Twitter\ThinkUp2"

Cloning into 'S:_incoming\Internet\Twitter\ThinkUp2'...
Receiving objects: 100% (28807/28807), 17.53 MiB | 1.00 MiB/s
Resolving deltas: 100% (21685/21685)
remote: Compressing objects: 100% (9075/9075)
remote: Compressing objects: 100% (9075/9075), done.
remote: Total 28807 (delta 21685), reused 26026 (delta 19341)
error: unable to create file webapp/plugins/twitter/tests/testdata/testofcrawlertwitterapiaccessoroauth/testinitializeendpointratelimits/application_rate_limit_status.json-resources=account,statuses,users,followers,lists,friends,favorites,friendships,application (No such file or directory)
Success (53774 ms @ 26/03/2013 18:54:39)

Even though "success", a random subsequent pull will fall over.

@ginatrapani
Copy link
Member

What about that filename is incompatible with Windows? Too long? Invalid character?

@petdance
Copy link
Contributor

I can't look into it now, but my first guess would be a conflict with another filename where the filenames are only distinguished by case. Windows and Mac OS X HFS+ are not case-sensitive.

@gavinr
Copy link

gavinr commented Mar 27, 2013

Sounds like a possible same issue to what I was seeing:
https://groups.google.com/a/expertlabs.org/forum/?fromgroups=#!topic/thinkup-dev/XoWpUfjaKxA
#1406
?

@7Two1
Copy link
Author

7Two1 commented May 9, 2013

hmm, surprised that this issue is still present, i had to create a 3rd folder as i CANNOT PULL

git.exe clone --progress -v "git://github.com/ginatrapani/ThinkUp.git" "S:_incoming\Internet\Twitter\ThinkUp3\ThinkUp"

Cloning into 'S:_incoming\Internet\Twitter\ThinkUp3\ThinkUp'...
remote: Counting objects: 29903, done.
Receiving objects: 100% (29903/29903), 19.81 MiB | 1.48 MiB/s, done.
Resolving deltas: 100% (22530/22530), done.
remote: Compressing objects: 100% (9455/9455), done.
remote: Total 29903 (delta 22530), reused 26976 (delta 20051)
error: unable to create file webapp/plugins/twitter/tests/testdata/testofcrawlertwitterapiaccessoroauth/testinitializeendpointratelimits/application_rate_limit_status.json-resources=account,statuses,users,followers,lists,friends,favorites,friendships,application (No such file or directory)
error: unable to create file webapp/plugins/twitter/tests/testdata/testofcrawlertwitterapiaccessoroauth/testinitializeendpointratelimits/application_rate_limit_status.json-resources=account,statuses,users,followers,lists,friends,favorites,friendships,application,search (No such file or directory)

Success (17222 ms @ 09/05/2013 14:07:23)

@ginatrapani
Copy link
Member

What do we have to change about those files to make them compatible? Once we know for sure, we can fix.

@randomecho
Copy link
Contributor

Replicated this on a fresh git clone under Windows Vista (we are out there) 32-bit.

The commas (,) and equal signs (=) are valid filename characters and the string lengths are under the 255 character limit. Normally when you grab files that contain invalid characters Windows will just automatically delete it for you and leave you with a need to commit something.

Looks like the filename and folder lengths are too long. By the time you get to the file you stop here:

application_rate_limit_status.json-resources=account,statuses,users,followers,lists,friends,favorites,friendships,application,sear

That's "ch" left off. Too long and/or too deep.

@ginatrapani
Copy link
Member

This is 240 characters long:

webapp/plugins/twitter/tests/testdata/testofcrawlertwitterapiaccessoroauth/testinitializeendpointratelimits/application_rate_limit_status.json-resources=account,statuses,users,followers,lists,friends,favorites,friendships,application,search

What is Windows Vista's limit? If you store the files in a very short base foldername, like c:\tu, does that make a difference?

@jacktandy, which version of Windows specifically are you using? What is the folder/filename length limit?

@randomecho
Copy link
Contributor

On Vista, tested with this filename of 254 characters under a path of w:\z (single letter):

01234567890123456789012345678901234567890123456789012345678901234567890123456789
01234567890123456789012345678901234567890123456789012345678901234567890123456789
01234567890123456789012345678901234567890123456789012345678901234567890123456789
01234567890123

The above only formatted here with breaklines to prevent scrolling.

@7Two1
Copy link
Author

7Two1 commented May 15, 2013

@ginatrapani Windows 7, 64 bit.

To recreate, simply create the same directory path i.e.
drive_letter:_incoming\Internet\Twitter\ThinkUp2
then try cloning into it.

not sure what length limits are these days, sure you can find that out, but the filenames in question could be considered questionably long anyway ;-)

@ginatrapani
Copy link
Member

I can't personally repro because I don't have Win7 but if someone can
identify what exactly needs to change about those files to make them
compatible I can help. Let me know.
On May 14, 2013 6:15 PM, "JackTandy" notifications@github.com wrote:

Windows 7, 64 bit.

To recreate, simply create the same directory path i.e.
drive_letter:_incoming\Internet\Twitter\ThinkUp2
then try cloning into it.


Reply to this email directly or view it on GitHubhttps://github.com//issues/1519#issuecomment-17915285
.

@gavinr
Copy link

gavinr commented May 17, 2013

Gina,

It looks like two offending files (at least for me) are in this directory:
https://github.com/ginatrapani/ThinkUp/tree/master/webapp/plugins/twitter/tests/testdata/testofcrawlertwitterapiaccessoroauth/testinitializeendpointratelimits

That directory plus filename is approx 240 characters, and it appears that Windows 7 (even 64-bit) has a 256 character limit (http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx), so if someone has his or her git repo in a location that has over 16 characters, there will be problems.

My repo, for example, is at:
C:\GitHub\ThinkUp\webapp\plugins\twitter\tests\testdata\testofcrawlertwitterapiaccessoroauth\testinitializeendpointratelimits\application_rate_limit_status.json-resources=account,statuses,users,followers,lists,friends,favorites,friendships,application,search

... which is causing problems because that path is 258 characters. I tested shortening the path to C:\g\ThinkUp\webapp..... and the problems went away for me, so I do believe the 256-character limit is the issue.

tl;dr - we need to shorten the path lengths for the twitter test data

Thanks!!

@ginatrapani
Copy link
Member

How about we advise Windows-based developers to use folder names under 16 characters in our documentation?

randomecho added a commit to randomecho/ThinkUp that referenced this issue May 17, 2013
Adds note in docs under "Set Up Your Environment" that long
filepaths will be troublesome if working on Windows platform.

Also fixes up some formatting of other points and italicises paths
for menu actions to help visually stand out from rest of text.

Updates a couple of URLs that have since 301 to new places.

Related to ThinkUpLLC#1519
@7Two1
Copy link
Author

7Two1 commented May 18, 2013

Hmm, it's your project to do as you see fit, however, in my opinion this is a terrible work-around, albeit better than nothing, and definitely the only one of many projects from svn/git or bazaar repos that has this filename issue. To pass the responsibility to the installer instead of addressing the issue properly by reducing the filename length is may i say poor. Also, I am not even talking about the install stage, simply cloning the repo to a folder greater than 16 chars is all that is required to raise this issue.

Does...
"application_rate_limit_status.json-resources=account,statuses,users,followers,lists,friends,favorites,friendships,application"

and...
"application_rate_limit_status.json-resources=account,statuses,users,followers,lists,friends,favorites,friendships,application,search"

really add value as file names on any OS?

The longest name here is 133 chars, leaving 16 to win, how about renaming it to 16 so win has 133 chars?

Perhaps, i'm missing something, is the resistance to rename due to the file name (and this would be shocking) being used as parameters to a function.

Please reconsider, much preferable are shorter meaningful file names that can be neatly organized in sub folders rather than the ancient 1990s practice of things requiring to be placed at drive root (as per the docs suggestion) yuk!

Thanks

@ginatrapani
Copy link
Member

@jacktandy each of those files represents the payload of an API call which developers use to write automated tests for code which make those calls to actual URLs when the app is running. The filenames look odd because they map simply and clearly to URLs with specific parameters defined by third-party APIs, in this case Twitter's. Shortening them would add yet another level of mapping that developers would have to decode while they write tests. My job is to make writing tests easier, not more difficult, hence the more verbose/clear filenames.

The files lengths are well within Windows' 256-character limit. We're not asking devs to store the files at the root, just within a 16-character folder name. I realize that is short.

What do you suggest ThinkUp's max file length should be? 200? 225?

@7Two1
Copy link
Author

7Two1 commented May 18, 2013

Tricky to say without looking at relevant use code, perhaps create stubs...
application_rate_limit_status.json-resources=account
application_rate_limit_status.json-resources=statuses
...
application_rate_limit_status.json-resources=search

and if this doesn't fit, and in all honesty, the mapping should be refactored. File names should never be used as function parameters for all manner of reasons e.g. scalability.

@ginatrapani
Copy link
Member

Digging into this a bit more, I recall that we do (attempt) to enforce a 200-character file length for mock test data. See https://github.com/ginatrapani/ThinkUp/blob/master/tests/classes/class.ThinkUpBasicUnitTestCase.php#L234 for the code which is supposed to do that.

Turns out the command is only checking the filename, not the path AND the filename, so these files got in past it. I'm working on a fix for both the command and shortening the folder names so that Windows devs will have to stay under a 56-character limit pathwise, which seems more reasonable than 16.

@ginatrapani ginatrapani reopened this May 26, 2013
ginatrapani added a commit that referenced this issue May 26, 2013
Shorten existing filenames over 200 characters for Windows users
Closes #1519
ginatrapani added a commit that referenced this issue May 26, 2013
On Windows your path must be less than 56 characters long, not 16 (see update to #1519). Not short enough to warrant extra note.
@7Two1
Copy link
Author

7Two1 commented May 28, 2013

@ginatrapani Yup, a 56 char limit is a far better proposition than 16.

Thanks for sorting this. I can confirm thinkapp now clones without issue into a "reasonable" path on windows where in my use case, the base path cloned into is... X:_incoming\Internet\Twitter\ThinkUp\

A virtual chocolate muffin and a high five to Gina.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants