SeaMicro has a pretty sweet looking product with their SM10000-64.

A while back I spoke to some SeaMicro sales guys and engineers and was pretty impressed.  These guys know their stuff and many of them were movers and shakers at Force10, Brocade, AMD, Sun, Cisco and Juniper.

With that pedigree it seems a foregone conclusion they’d be able to come up with the new hotness in systems design.

So, what problem are they REALLY trying to solve?  In their own words:

“Historically, servers were designed to quickly solve a relatively small number of very hard problems. The Internet, however, changed this. In the Internet data center, the challenge is to handle millions of relatively small, independent tasks like those needed for searching, social networking, viewing web pages, and checking email. Volume servers failed to adapt to this fundamental change.”

What did they do?

With some magic ASIC design they crammed 256 netbooks into 10U to save power and bet that the simpler Atom CPUs will be enough to handle the workloads we see in the serving layer.

Well, really its more complicated and impressive than that.  It is a 10U chassis that holds:

• 64 daughter cards each with:
  • 4 Intel N570 Atom CPUs, 64-Bit / 1.66Ghz
  • 4GB of RAM per CPU.
  • Essentially four instances. per card.
• A rules-based system for bringing instances up and down to save power.  Auto-scaling along the lines of Amazon’s Elastic Beanstalk.
• Storage and networking, through the marvel of modern ASIC design, are abstracted from the instances.
  • 64 disks that are divied up across the nodes.
  • A built in Load balancer that works against L3, L4 and L7.
  • A big ‘ol networking back plane to handle all the inter-node traffic.

SM10000 Compute Card

One sweet unit!  However, they have been marketing it heavily (at least from what I’ve seen) to Hadoop users and I just don’t think it is an appropriate choice for that role.

These are the problems I see for it as a hardware platform for Hadoop:

Problem 1: 64 disks for 256 instances.

The backplane apportions 1/4 of a disk per instance if you want all nodes to have a drive.  You’d have multiple datanodes hitting the same physical disk.  It seems to me that that would be an I/O headache.

Problem 2: Memory is limited to 2G per core and is not ECC.

With Hadoop’s sometimes sketchy memory issues, this is not an ideal config.

Problem 3: One big single chassis.

In my opinion a Hadoop cluster should be distributed across several racks/PDUs/circuits.  If your default replication factor is 3, then to be truly safe and available you’d need to have three of these beasts on separate racks.

So, I am highly skeptical on using Hadoop here due to these limitations for I/O-bound and RAM-intensive workloads.

… but for application nodes I can see it as a perfect fit.

A pretty cool SeaMicro setup might break down like:

• 64 Instances mapped 1:1 with the 64 drives for a distributed database like Cassandra, Redis. MongoDB.
• 64 Instances booted from PXE as memcached nodes (2-3GB caches).
• 128 Instances booted from PXE running your app server.
• Logs stream using Flume or Scribe to a Hadoop cluster (perhaps in C6100s) elsewhere.

If you can fit your app server (Tomcat, Tornado, Nginx, etc..) in that memory footprint then you’re set.

Our Tomcat nodes have between 8-24G of RAM depending on the application so it would take a bit of re-engineering, however that application would be spread across several more smaller instances.

The parallelization may actually do some good too.  Using the in-built load balancer you can think of all the small instances as workers in a larger process.

I’ve seen the idea described before as RAIN (redundant array of inexpensive nodes) or FAWN (fast array of wimpy nodes).

Most co-location facilities I currently work with allow us to pay for power by usage. With SeaMicro’s ability to scale up and down we could save quite a bit by dynamically bringing nodes up and down as needed.

There is lots of money to be saved as your hardware and power footprint is reduced.  Also, when I last spoke with SeaMicro, New York State was offering to subsidize a large potion of purchase cost of the SM10000 series to encourage power savings in datacenters.

There is a lot of room for improvement though. Perhaps ARM might be a more appropriate architecture since they could at least use ECC and I’d like the freedom to use any disks instead of being locked into SeaMicro.

The SeaMicro reps mentioned that they have several companies using it with Hadoop and at an upcoming NYC Hadoop Meetup someone, from somewhere will present on the subject.  It will be interesting to see how it all works.