Deploying to remote server hangs

[seongbae-ml]

Hi,

I used to be able to deploy to a remote server fine... but starting at some point, it stopped working. When I run "./ml dev deploy modules" command, it just hangs and doesn't do anything. When I break out of it using Ctrl+C, it outputs following message:

MacPro-2352:mlwonline sbae$ ./ml dev deploy modules
^CERROR: 
ERROR: ["/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/protocol.rb:155:in `select'", "/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/protocol.rb:155:in `rescue in rbuf_fill'", "/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/protocol.rb:152:in `rbuf_fill'", "/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/protocol.rb:134:in `readuntil'", "/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/protocol.rb:144:in `readline'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/xcc.rb:53:in `read_status_line'", "/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/http/response.rb:28:in `read_new'", "/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/http.rb:1406:in `block in transport_request'", "/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/http.rb:1403:in `catch'", "/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/http.rb:1403:in `transport_request'", "/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/net/http.rb:1376:in `request'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/RoxyHttp.rb:376:in `block in request'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/RoxyHttp.rb:343:in `loop'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/RoxyHttp.rb:343:in `request'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/MLClient.rb:101:in `go'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/xcc.rb:165:in `go'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/xcc.rb:132:in `block in load_files'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/xcc.rb:125:in `each'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/xcc.rb:125:in `each_with_index'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/xcc.rb:125:in `load_files'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/server_config.rb:757:in `load_data'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/server_config.rb:1329:in `block in deploy_src'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/server_config.rb:1304:in `each'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/server_config.rb:1304:in `deploy_src'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/server_config.rb:1289:in `deploy_modules'", "/Users/sbae/Documents/Projects/mlwonline/deploy/lib/server_config.rb:717:in `deploy'", "deploy/lib/ml.rb:155:in `<main>'"]

The only change we had that could potentially break the deployment is using a lot of front-end libraries and dependencies. When I checked number of files in my /xquery folder, there are over 3500 files in it. Would that potentially break the deployment?

Thank you.

Seong

[joemfb]

You could try running deploy in verbose mode:

./ml dev deploy modules -v

That would give you more insight into what's happening (for instance, is the process slow overall, or is it hanging on a particular file).

[grtjn]

I think the batch approach for remote deployment is getting in the way here, try adding --batch=no:

$ ./ml local deploy modules -h
Usage: ml {env} deploy WHAT [options]

General options:
  -v, [--verbose]  # Verbose output
  --batch=(yes|no) # enable or disable batch commit. By default
                     batch is disabled for the local environment
                     and enabled for all others.

...

That will cause modules to be sent across one by one.

[dmcassel]

@seongbae-ml, did grtjn's batch suggestion help?

[seongbae-ml]

Hey @dmcassel,

Thank you for following up. It works.... but whenever we make a deployment, it looks like it tries to deploy everything in the application folder. We have over 3500 files. Batch runs one file at a time so I always Ctrl+C out of it in the middle.

We opted to log onto the server and use git to download the latest changes. Not the most optimal way.. but it seems to work.

Actually, when I use -v flag, it shows that it does deploy, but one file at a time. There are over 3500 files in the application folder. Maybe it takes a while to deploy all the files?

Are there any options to exclude certain folders from being deployed? Most of 3500 files are front-end libraries that don't update often during development.

Seong

[grtjn]

Maybe the batch option doesn't make too much difference. Uploading 3500 files simply takes time. Maybe use the batch=no option only if you are receiving timeouts, a batch=no approach could otherwise mean it simply takes even longer.. :-/

I need to take a closer look at this batching part. Might be useful to be able to influence batch size..

[seongbae-ml]

@grtjn Is there a way to place libraries outside the application folder? I think either that or being able to exclude certain subfolders during deployment would solve the issue.

[rlouapre]

@seongbae-ml if you can identified the files that has been modified since the last deployment you could use ml {env} load.

There is also the property ignore-folders to exclude folders #331.

[seongbae-ml]

@rlouapre Sweet! Thank you. I will give those a try.

[grtjn]

I think it should also be possible to literally split src/ into for instance src/ and ui/, or src/ and static/. Then you add a method in app_specific called deploy_ui or deploy_static, that overrides @properties["ml.xquery.dir"], and then calls deploy_modules..

[dmcassel]

@seongbae-ml, there's a wiki that shows how to add Custom Build Steps. It has examples on that page.

[grtjn]

I think #551 provides the right solution for this, just append --incremental=yes. It was implemented initially to work only for deploying content, but a small fix was merged (see #777) to allow it to be used with deploy modules too.

[grtjn]

I think we can safely close this ticket now.