[Kittyhawk] 25,000 node scaling

Eric Van Hensbergen ericvh at gmail.com
Thu Mar 22 13:26:53 EDT 2012


Okay, in order to meet the INCITE 100k key/value evaluation milestone,
we are going to need a 25k node kittyhawk run, which is (5x? 6x?) what
we've done previously.  Here's what I think we need:

a) resolution on IP address assignment (have we asked Bill Scullin on
this yet?) -- although technically we may be able to do this with
completely private network, but we may need additional nodes to drive
the load.
b) we need to eliminate dependence on the khctl node or allow for
distributed khctl.  Once upon a time Jonathan mentioned being able to
just boot Linux kernels+ramdisk at startup versus coordinating boot
through khctl.  This may be quickest path to getting to where we need
to be
c) libfox (w/integrated memcached servers) running on khfoxdev so we
can build a fox cpio for the test
d) some test which will stress the nodes
e) all of this running on intrepid

What did I miss?  Let's start putting names and dates on all of these
since we already missed the INCITE milestone deliverable (which I
think is understandable given all the trouble we had getting up and
running on the new allocation).

       -eric


More information about the Kittyhawk mailing list