@pacman: my home quad-core skylake @4.4 GHz builds an allmodconfig kernel in about 13mn, and when coupled with the 6-board build farm, it goes down to about 4mn45. It compiles up to 120 files per second, which is truly impressive. At work, the kernels I build for our products normally take 9-11 mn (depending on the options) and go down to 2mn50-3mn10 with the 4-board farm. In fact building all the kernels now take 29mn instead of 1h30.
The home PSU suffers a bit at 2 GHz, it gets very hot. It's sold as a 60W one, I'm seeing 67W consumed on the input on average during a build and a few peaks to 72-75 (which is fine when taking into account the PSU's efficiency). I've seen one board reboot so I'll have to try another PSU just to be sure. But anyway even at 1.8 GHz the performance is really cool.
Using a dumb switch requires to let distcc perform the load balancing itself and it's really not that good at this task, some boards having 4 builds in parallel and others none (which is suboptimal). haproxy instead ensures that all boards have the smallest possible number of jobs needed to sustain the load, and queue the extra jobs to deliver them immediately. I've run haproxy on the build machine instead, the performance used to be slightly better with the initial setup which was not as fast as what I'm getting now. But the extra CPU cycles spent in haproxy are not available for cpp/gcc. Surprizingly, compressing with lzo significantly improves the performance by wasting less time pushing data (even over gbit).
BTW regarding switches, only marketing counts twice the number of ports for switch capacity, because any traffic that flows out of one port had to enter via another one, so you must count either the sum of all Rx or the sum of all Tx but not both