Home
Are you a programmer or hardware hacker actively avoiding the problems of distributed systems? Yes, yes you are. No, no, shut up, you are. The sooner you can admit that, the sooner we can fix you.
Okay, so maybe you know how to code, or have some electronics chops, maybe you're new to this and would like to learn... Now what's a cluster? How do you fight one? How do you learn this stuff? Is this even legal? Here's a breakdown of what ClusterFights is and what it can offer.
ClusterFights is an organization where members design, build, and deploy inexpensive, off-the-shelf, distributed supercomputers. We then use these supercomputers to solve unique programming challenges in a competitive environment, often with drinking involved. While our challenges are somewhat contrived and arbitrary, the software and hardware, and the skills used to make them, are applicable to many real-world applications.
"Cluster" is a very loose term here. It encompasses aspects of distributed systems, high performance computing, parallel computing like FPGA's & GPU's, and computer architecture. It's an opportunity to learn about strange new ways to solve problems, get your hands dirty making real rigs writing your own software, and party with / talk trash to like-minded nerds like yourself.
You've probably heard of Moore's Law and you've also probably heard that it's probably going to end soon (or already has). Semiconductor processes are reaching the limits of classical physics and in order to continue the development of ever-more-powerful computers, we believe that experimenting with other kinds of distributed architectures are the solution (at least until some breakthroughs in material science make their way into production). There have been many different attempts at interesting, non-standard, distributed architectures (the Connection Machine, for instance) and the current price of modern compute hardware makes it worthwhile to revisit them.
- Weight Classes
- Clusters are separated into different "weight classes", which set the maximum price of a cluster.
- Bill of Materials Cost
- BOM cost is determined by the total price of all components that compute, store, or actively move data.
- This means that items like SD cards, development boards, network switches, and single board computers count toward the cost.
- Items like cables, power supplies, pieces of flair, LEDs, bells, whistles, and standoffs do not.
- Availability and Cost
- The price for these items must be based on the publicly available cost of the components.
- It doesn't count if your aunt works at Nvidia and gets you a bunch of free graphics cards, or you steal your little brother's high-end gaming PC... those items get added to your bill of materials at retail price.
- However, it is completely legal to find an old laptop or something on a public website like Ebay within the weight class budget and compete with that.
- DIY Hardware Incentive
- There is a strong incentive to make your own electronics. For example, if a dev board cost's $100 but the processor on it only costs $10, getting a board made with that chip on it means that only $10 goes toward your budget. Copper and fiberglass don't compute or store data, do they? ;)
- There's a big caveat on this, however: YOU MUST ASSEMBLE THE BOARDS YOURSELF! This could either mean soldering by hand or using a modified toaster reflow oven or something similar. Using a service to assemble your boards means the PCB fabrication cost, assembly cost, and parts cost factor into your bill of materials. You basically just ordered a dev board off the internet with extra steps
- Size, Weight, and Locality of a Cluster
- All the hardware used to compete must be moved by the competitor through the door of the competition area, by themselves. This puts practical limits on size, weight, and locality of the cluster. There are ways to violate the spirit of this rule and most of them are encouraged because they make the competitor look ridiculous.
- Oh, your 50ft, $250 cluster has to be assembled with a crane once through the door? Go for it.
- Your $75 cluster somehow weighs 300 pounds and requires simple machines to move it through the threshold? Totally legal.
- This does mean, however, that you can't use cloud services like Google Bigtable or Amazon EC2 unless you somehow physically have access to those servers, you bring them with you, and they conform to all the the rules above.
- All the hardware used to compete must be moved by the competitor through the door of the competition area, by themselves. This puts practical limits on size, weight, and locality of the cluster. There are ways to violate the spirit of this rule and most of them are encouraged because they make the competitor look ridiculous.
- Golden Rule
- Don't be a dick.
Each challenge page has (or should have) a script to generate a sample dataset, a script to generate a sample challenge, and a trivial non-optimized single-core solution.
- Clustemón
- Massive Maze
- Gigantic Graph
- Jigsaw
- Taylor Series
- Chestful Grits
- My Little Internet
Clusters that qualify for the ThreeBangers competitions cost <$250 BOM.
- Wrathberry PII
- Cluster Buster
- Mister Clustoffelees
- Star Crunch Cereal®
- MunchMan
- Fruit Salad
SuperMicro clusters are <$75. They're most likely microcontroller-based.
PISS - python, interactive, ssh and scp
Actor Model (Akka)
- Fighters will (hopefully) update this list every fight with their latest BOM-contributing component hardware manufacturer, model, purchase cost, purchase link
- Official Hardware List
There are several ways to get in touch and get involved in ClusterFights and build your skills.
Our main online hangout is the #clusterfights-general channel on Matrix.org.
Another easy way to get in contact is to shoot us an email at info@clusterfights.com or, if you're local to the SF Bay Area, check out our Meetup page and come to one of our events.
It doesn't matter how you do it, but we want fresh meat to fight! Download the challenges, build a cluster, and compete!