| title | layout | section | breadcrumbs | ||||||
|---|---|---|---|---|---|---|---|---|---|
IronWorker Environment |
default |
worker |
|
The operating system and version information is provided for completeness and transparency. We recommend, however, you avoid binding your workers to specifics of the OS as much as possible.
IronWorker contains several popular Linux packages as part of the standard worker environment.
| Package | Full Name | Purpose |
|---|---|---|
| ImageMagick | ImageMagick Image Processing | Image processing |
| FreeImage | The FreeImage Project | Image processing |
| SoX | Sound eXchange Library | Sound processing |
| cURL | Client URL Request Library | URL file processing |
These are included because they are common binary libraries. Other binary libraries and files can be included as part of your worker code package, though you'll first need to compile them to target Linux x64 architectures.
If you don't see what you need here, please contact us and tell us what you're looking for. If it's a common/popular package, we can certainly look to include it.
The following is the maximum data payload that can be passed to IronWorker. A data payload that exceeds this size will generate an error response from the API.
Tip: We recommend that you avoid sending large payloads with your workers. Instead use a data store to hold the data and then pass an ID or reference to the worker. The worker can grab the data and then do its processing. It's more efficient on the API as well as better in terms of creating atomic/stateless processing.
The standard worker sandbox environment contains a certain amount of accessible memory. This amount should be sufficient for almost all workloads. We are working on a super worker environment that would allow greater memory allocations. Please contact us if you have specific needs here.
Tip: We recommend distributing workloads over multiple workers—not only for better resource management, but also to take advantage of the massive concurrency enabled by a cloud worker system.
Each worker task has local disk space available to it for use on a temporary basis while the worker is running. You have full read/write privileges to create directories and files inside this space, and can perform most ordinary file operations. This directory is used as the current working directory (".") when executing your workers.
There is a system-wide limit for the maximum length a task may run. Tasks that exceed this limit will be terminated and will have timeout as their status.
Tip: You should design your tasks to be moderate in terms of the length of time they take to run. If operations are small in nature (seconds or milliseconds) then you'll want to group them together so as to amortize the worker setup costs. Likewise, if they are long-running operations, you should break them up into a number of workers. Note that you can chain together workers as well as use IronMQ, scheduled jobs, and datastores to orchestrate a complex series or sequence of tasks.
Each priority (p0, p1, p2) has a targeted maximum time limit for tasks sitting in the queue. Average queue times will typically be less than those listed on the pricing page. High numbers of tasks, however, could raise those average queue times for all users. To keep the processing time for high priority jobs down, per user capacities are in place for high priority queues. Limits are on per-queue basis and are reset hourly. High priority tasks that exceed the limit, are queued at the next highest priority. Only under high overall system load should queue times for tasks exceeding the capacity extend beyond the initial targeted time limits. Usage rates will be based on the actual priority tasks run on, not the priority initially queued.
| Priority | Capacity Per Hour Per User |
|---|---|
| p2 | 100 |
| p1 | 250 |
The following is the default number of scheduled tasks. It should be sufficient for even the largest projects. If you would like this number increased, however, please feel free to contact us.
Tip: A common mistake is to create scheduled jobs on a per user or per item basis. Instead, use scheduled jobs as master tasks that orchestrate activities around sets of users or items. When scheduled tasks run, they can access databases to get a list of actions to perform and then queue up one or more workers to handle the set. View the page on scheduling for more information on scheduling patterns and best practices.
Tasks can only be scheduled to run every 60 seconds or more using the run_every parameter.
IronWorker provides an AWS security group and IP ranges in the event users want to isolate AWS EC2, RDS, or other services to these groups/ranges.
| EC2 Security Group | Account ID |
|---|---|
| simple_worker_sg | 7227-1646-5567 |