Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Double archive creation v3.0 #285

Open
n00dles opened this issue Nov 2, 2012 · 32 comments
Open

Double archive creation v3.0 #285

n00dles opened this issue Nov 2, 2012 · 32 comments
Labels
Bug PRIORITY
Milestone
Eventually

Comments

@n00dles
Copy link
Contributor

n00dles commented Nov 2, 2012

Original author: borislav...@gmail.com (January 20, 2012 23:32:55)

What steps will reproduce the problem?
Doing website archive on RackSpace cluster

What is the expected output? What do you see instead?
One file, and instead two archive files with almost the same (time differs a bit) name appear.

What version of the product are you using? On what operating system?
3.0
RackSpace cluster
There are some plugins, I've tested with and without them and no change

Please provide any additional information below.
The site is very large 140MB+

Original issue: http://code.google.com/p/get-simple-cms/issues/detail?id=285
@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From c...@3pcmedia.com on January 21, 2012 01:48:21
This happens when you click the "archive now" button?

Can you attempt this on 3.1 to see if you can replicate this problem? I cannot.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 21, 2012 14:48:18
I will try to replicate the site on 3.1 and will report back.
Will test on two server instances, one is managed debian cluster (if I am not mistaken) and the other WAMP on 7 64.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 21, 2012 15:43:30
Bare installation works, I am uploading and testing with:

  1. content
  2. plugins

Will report back.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 21, 2012 16:23:42
Done test nr. 1 and the same happened:

Backup action created two archives with a bit different timestamp.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 21, 2012 16:24:26
Here is the screenshot.
I don't have a clue what this might be, any ideas?

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 21, 2012 16:49:54
The title should be changed to: Double archive creation on 3.0 and 3.1b

I will contact tech support, but I will need some input from you, as I wouldn't know what to tell them. From the testing I've done (turning stuff on and off), I couldn't come to idea what might cause this. Maybe it is an internal problem on their side.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 21, 2012 17:03:11
I have done some further testing.

Deleting all stuff from uploads resulted in one archive created. Could it be that the archive creation is capped in size?

I've noted 600M limit in the code.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 21, 2012 18:35:05
New details and SUM + sizes in the end

  1. default installation creates single website archives / 128K
  2. problem are not plugins, but contents of uploads folder
  3. uploads folder, containing 45 smaller (20-30k) files in root, two subdirs with larger files, and one subdir with 400 files (small gifs) renders creation of two archives / 25.1MB, 25.1MB
  4. if all is removed from uploads, stuff works again / 10.7MB
  5. if only large files are added in their respective subdirs, archive is one / 22MB
  6. if 45 small files are returned to the root, the archives are two / 23.3MB
  7. large files removed, but smaller 45 in root left, archive is one /12MB
  8. 45 files in root + 400 in subdir, archive is one / 13.8MB
  9. three large mp3 files (~8mb each) added to root, archive is one / 36.5MB
  10. returned one of two dirs with larger files (PDF), archive is one / 43.3MB
  11. returned second larger dir (RZF files), archive is one, 47.8MB

attach

Than, I return to the old install, try backup, and again two archives. Somewhere in the middle, FTP gives two files in creation, in paralel. Attached.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From m...@digimute.com on January 21, 2012 20:57:14
duplicates will be created if you double/treble click the create archive button...they will be created in parallel with different timestamps.

the 2 other file above are the temp files created when zipping up the folders , why they failed who knows...

we should disable the create archive button until the zip file is created or fails.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 21, 2012 23:22:33
Double/triple click was not an issue.

The strange.png is just a representation of a process when process created zips, both archives were correctly created. The what now.png show the situation with new release.

I have checked the time stamps, and they are created ~30 sec apart, in both cases (old version and new version).

Please mind that I am testing on a quality cluster, using quality equipment, and don't have Parkinson's. :-)

Any idea beyond double click?

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 21, 2012 23:34:36
Further testing, on iPad Safari three archives were created. It seems the browser tries to reconnect the server while it is busy creating the first archive, triggering more archive events. Clicking the button, and than stopping the browser solves the issue.

Any way to solve this?

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From ccagle8 on January 23, 2012 01:42:41
how long does it take for one (or all) of the archives to complete?

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 23, 2012 09:06:36
There are two states:

  1. When the cluster is not under strain (US sleeps) about 15 sec for single 150MB archive. Than the script works.
  2. When it is day in US, the cluster can be under strain, single archive is created from bout 35 up to 105 sec.

The archive job(s) are always completed.

According to tech support, ZIP is resource intensive, and can even be turned off sometimes (than an error pops up). They suggested using tar gz instead zip, as it always available.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 23, 2012 23:22:57
Just to note; I am testing on a production website, on a production 100% SLA hosting.

Please do ask me to do any tests you think out to get this problem sorted out, I will be happy to oblige!

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From ccagle8 on January 26, 2012 01:58:08
borislav, can you make this change, and see if it fixes the issue?

http://code.google.com/p/get-simple-cms/source/detail?r=643

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 26, 2012 06:46:35
I have tested it, but the result is the same.

Tried set_time_limit(0) myself, but it didn't work. I've also tried doing that in .htaccess (php engine is controlled from it on the cluster), but it also didn't work.

I am setting up a three environments for testing:

cluster

dedicated server

convetional hosting

Will report back with findings.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 26, 2012 14:49:20
Hello,

I have tested the archive creation on

cluster - the result is as before

dedicated server - while ZipArchive was installed, and GS reported it works, when the create archive was clicked, it gave ZipArchive error

convetional hosting (cpanel) - the archive creation worked nicely

//

What should I do now?

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From ccagle8 on January 28, 2012 00:41:46
is it the same issue on 3.1 beta?

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 28, 2012 01:11:30
Yes.
I've also tested if it was caused by plugins, different types of documents, etc., but none changed what happened.
I have decided to pass this hosting for this and subsequent projects on GS. It would be good to solve this issue as RackSpace cloud is premium hosting service.
The site copy is still on the RS, I have left it there so I can conduct testing to further GS.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on January 28, 2012 01:12:35
If you wish, you can contact me directly, and I will provide account and all resources needed for testing on the cluster.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From tablatronics on May 11, 2012 19:26:19
Cannot reproduce.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on May 11, 2012 19:53:02
It happens occasionally when the server is under load, or very big archive is created. Happened regularly during rush hours on RackSpace cloud, and happens occasionally during high traffic on shared hosting.

Archive size is 200+ MB.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From tablatronics on May 12, 2012 04:18:48
Sounds like script timeout.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on May 12, 2012 06:57:43
Can you fix it?

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From tablatronics on May 20, 2012 01:04:47
Not without reproducing it first.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on May 20, 2012 20:34:56
Put a lot of large files on your GS installation, and it will eventual break. You need to play with it a bit.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From tablatronics on May 22, 2012 14:30:37
Have you done this with debugging on, or checked the error logs ?

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on May 22, 2012 15:55:41
Don't remember anymore.

Give me instructions what to do, and I will.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From tablatronics on August 20, 2012 15:54:34
I am seeing this on my server now, so i can reproduce.
I am getting about 6 archives per operation.

Will debug.

@n00dles
Copy link
Contributor Author

n00dles commented Nov 2, 2012

From borislav...@gmail.com on August 20, 2012 17:19:01
Thanks! :-)

@tablatronix
Copy link
Member

This is hard to nail down, but I am pretty certain it has to do with our synchronous browser request waiting for the response, or a php timeout or both.

This is a bad pattern, browsers will timeout, which might be recalling the page.

We might have to have have an ajax request that checks periodically instead of the synchronous browser request for long waits.

This is something we should build in for other operations as well.

@tablatronix
Copy link
Member

No progress on this yet, problem not persisting.

@tablatronix tablatronix added this to the Eventually milestone Oct 2, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug PRIORITY
Projects
None yet
Milestone
Eventually
Development

No branches or pull requests
2 participants
@n00dles @tablatronix