Bringing HPC Sexy Back

Since Seymour Cray, considered the father of supercomputing, invented the first supercomputer (the CDC 6600) in 1964, HPC has been growing exponentially—in speed. But, the fact remains, that high performance computing as an industry is decades old. In comparison to young, sexy technologies like mobile development and cloud computing, HPC is a dinosaur.

The YouTube video above illustrates the attempt and desire to bring high performance computing back into the limelight, perhaps to make it sexy again (dare I say, has it ever been “sexy?”). They do an excellent job incorporating the comical Penguins of Madagascar, who were themselves the beneficiaries of HPC systems. As the video illustrates—even for being decades old—HPC is still a thriving industry. In a recent IDC press release, they state that HPC revenues grew by 7.7% overall last year, which exceeded their own predictions, and the high-end supercomputing segment saw growth over 29%.

Why are these stats relevant?

HPC growth is modest compared to some technology markets, even during a recession. However, it’s showing solid and consistent increases—especially considering the barrier to entry with HPC is steep, given the average price of even the smallest HPC systems.

Why bring HPC sexy back?

The thought is that HPC is growing, and there’s no shortage of good admins and programmers to work on HPC problems. Or is there? According to some industry experts’ opinions (including our own founder, David Jackson), the latter of those statements might not be true. Not only are good HPC admins and developers tougher to come by, but also the increasing size of “big” problems being mounted by HPC—e.g. “big” data—are upping the ante for HPC admins and developers alike. The race towards exascale brings with it a laundry list of tough design and development challenges—let alone the challenges of administering and programming to such a system.

Perhaps the mobile wave—smaller, simpler, cleaner, and emphasis on usability, will help to make the industry better. More attractive UIs, more usable software, and a higher emphasis on quality are all things that younger programmers and admins are used to in their daily lives. Management and application providers would be wise to consider having usability be at the forefront of all product development considerations.

While attending Cray Users Group in Napa Valley this past month, I had to ask a few of the younger attendees what drew them to HPC. It seems that their stories had a common thread—they kind of fell into it in one way or another. One young admin commented that he happened across a job buried in pages and pages of research on his University’s website. Another application programmer heard about it from a fellow student doing an internship. Now, these young HPC Padawans are very talented and impressive, no doubt. The problem lies in the fact that we will need more like them.

I think the “why” of working with HPC is the key. So many great tech companies of our day—Facebook, Google, Twitter, and others—are concerned with triviality in the grand scheme of things. Facebook research scientist, Jeff Hammerbacher, said, “The best minds of my generation are thinking about how to make people click ads. That sucks.”

Those working in HPC won’t be concerned with ad clicks. The cure for cancer, when found, will likely pass through the hands of these brilliant programmers. Indeed, they don’t do the research directly, but they make obtaining answers to the research possible. They will help prove the origins of the universe, combat disease, forecast inclement weather, increase energy efficiency, combat terrorism, invent new devices and technology, save numerous lives, make manufacturing more efficient, and many more countless research endeavors. Why work in HPC? For starters, you can enable life-changing research to happen. I can’t think of a better reason.

Do you think HPC is “sexy,” and why? If you work in the field, what drew you to HPC and continues to keep you there?

Facebook Twitter Email
  • Adam Diaz

    I think HPC will ultimately be reinvented in Hadoop via Yarn but not immediately. Grand scheme most IT concepts (like most movies) are rehashes of some old ideas with a very slight new twists. I saw Hadoop as a big cluster the minute I laid eyes on it. It just needs some updated workload management. I wonder who will do that first..

  • http://twitter.com/iannate Ian Nate

    Interesting point, Adam – I read an article about Hadoop around the MapReduce methodology, which also mentions YARN, and I found it very interesting. I think it’s definitely viable that Hadoop could be the future of HPC, although some talk around HPC Cloud could also displace some typical workload models. Here’s a link to the article: http://www.admin-magazine.com/HPC/Articles/Is-Hadoop-the-New-HPC