Re: make -jn not necessarily helpful

["Freddie Cash" <fcash-ml@xxxxxxxxxx>]

On Tue, March 14, 2006 4:42 pm, Justin C. Sherrill wrote:
> On Tue, March 14, 2006 12:03 pm, Freddie Cash wrote:
>> Try moving the make clean to the end, and rebooting between runs.
>> That will eliminate any caching or disk buffering that may be
>> happening.

> That step I pasted repeated with different -j levels, so a 'make
> clean' happened after and before every step.

> I can't imagine something as large as a 'make buildworld' would be
> significantly cached, and even if it was, then I'd see a sudden
> performance improvement after the first run.

>> I haven't timed the buildworld runs on my systems in a long time,
>> but the last time I did, -j2 was faster than -j1 on a UP system.
>> All higher -j settings were slower.  And on a dual-Opteron sytem,
>> -j4 and -j6 alternated between being the fastest.

> If you have a system on which you can try it, I'd like to know the
> results.  I suspect there may be a cutoff where this does not help,
> and I want to define this better than the folklore it is now.

I don't have any DragonFlyBSD systems anymore, but I ran a buildworld
loop on a FreeBSD 6.0p4 system at work.  It's a dual-AthlonMP 2200+
with 4 GB RAM and a single 80 GB ATA100 HD.  I ran a for loop that
did:
  make cleanworld
  make clean
  make cleandir
  make clean
  make -jX buildworld
with X values of 1 through 20.  Attached are my results.

-j1 is the slowest by over 40 minutes.  -j3 is the fastest by a measly
minute and a bit.  -j2 through -j20 are all within 3 minutes of
eachother.

On this system, I'd have to guess that three make processes are able
to access the disk at one time, and the rest just spend their time
waiting to access the disk.

We have a spare rackmount here (dual-Opteron 244, 4 GB RAM, 4x 150 GB
SATA drives connected to a 3Ware Escalade 9550SX RAID controller). 
I'll see if I can get FreeBSD installed and run through the same
tests.  The I/O should be much less of a bottleneck in this system,
and it should get a better gain from the higher -jX values.  That's my
thoughts, anywho, we'll see what it actually does.  :)

----
Freddie Cash, LPCI-1 CCNT CCLP        Helpdesk / Network Support Tech.
School District 73                    (250) 377-HELP [377-4357]
fcash-ml@xxxxxxxxxx

Wed Mar 15 08:15:18 PST 2006 Testing make -j1 ...
Wed Mar 15 10:30:02 PST 2006 done
----------
2 hours  15 mins

Wed Mar 15 10:31:59 PST 2006 Testing make -j2 ...
Wed Mar 15 11:51:14 PST 2006 done
----------
1 hours  20 mins

Wed Mar 15 11:53:12 PST 2006 Testing make -j3 ...
Wed Mar 15 13:11:55 PST 2006 done
----------
1 hours  18 mins

Wed Mar 15 13:13:52 PST 2006 Testing make -j4 ...
Wed Mar 15 14:33:08 PST 2006 done
----------
1 hours  20 mins

Wed Mar 15 14:35:04 PST 2006 Testing make -j5 ...
Wed Mar 15 15:54:07 PST 2006 done
----------
1 hours  21 mins

Wed Mar 15 15:56:04 PST 2006 Testing make -j6 ...
Wed Mar 15 17:14:55 PST 2006 done
----------
1 hours  18 mins

Wed Mar 15 17:16:50 PST 2006 Testing make -j7 ...
Wed Mar 15 18:35:47 PST 2006 done
----------
1 hours  19 mins

Wed Mar 15 18:37:42 PST 2006 Testing make -j8 ...
Wed Mar 15 19:56:48 PST 2006 done
----------
1 hours  19 mins

Wed Mar 15 19:58:43 PST 2006 Testing make -j9 ...
Wed Mar 15 21:17:43 PST 2006 done
----------
1 hours  19 mins

Wed Mar 15 21:19:38 PST 2006 Testing make -j10 ...
Wed Mar 15 22:38:48 PST 2006 done
----------
1 hours  19 mins

Wed Mar 15 22:40:42 PST 2006 Testing make -j11 ...
Thu Mar 16 00:00:02 PST 2006 done
----------
1 hours  20 mins

Thu Mar 16 00:01:58 PST 2006 Testing make -j12 ...
Thu Mar 16 01:21:27 PST 2006 done
----------
1 hours  20 mins

Thu Mar 16 01:23:22 PST 2006 Testing make -j13 ...
Thu Mar 16 02:42:44 PST 2006 done
----------
1 hours  19 mins

Thu Mar 16 02:44:39 PST 2006 Testing make -j14 ...
Thu Mar 16 04:05:17 PST 2006 done
----------
1 hours  21 mins

Thu Mar 16 04:07:14 PST 2006 Testing make -j15 ...
Thu Mar 16 05:26:49 PST 2006 done
----------
1 hours  19 mins

Thu Mar 16 05:28:45 PST 2006 Testing make -j16 ...
Thu Mar 16 06:48:18 PST 2006 done
----------
1 hours  10 mins

Thu Mar 16 06:50:13 PST 2006 Testing make -j17 ...
Thu Mar 16 08:09:53 PST 2006 done
----------
1 hours  19 mins

Thu Mar 16 08:11:48 PST 2006 Testing make -j18 ...
Thu Mar 16 09:31:25 PST 2006 done
----------
1 hours  10 mins

Thu Mar 16 09:33:20 PST 2006 Testing make -j19 ...
Thu Mar 16 10:53:11 PST 2006 done
----------
1 hours  20 mins

Thu Mar 16 10:55:07 PST 2006 Testing make -j20 ...
Thu Mar 16 12:15:05 PST 2006 done
----------
1 hours  20 mins