tech

One of the perquisites of being on an NSF fellowship is that I have access to supercomputing facilities that ordinarily you need to be a professor to use. Wednesday, I ventured to Cambridge to attend a seminar about what I can do, and I'm pretty excited, especially given that I've been wanting supercomputer capabilities for a while.

Even nicer is that one of the supercomputers is optimized for serial processing like Monte Carlo simulation, which is a large part of how Bayesian inference is done. My biggest worry about supercomputing was figuring out how to parallelize Monte Carlo simulation to take advantage of the multiple processors. This seems to mean that I don't have to worry about that.

I saw this press release at a Sun site and found it rather interesting. There's not a lot of content other than that Sun has hosted an emerging markets summit and intends to focus more on citiesin developing worlds because (1) the markets there are rather large and expanding, (2) they're not inhibited by legacy technlogies, and (3) they are friendly to open source technology.

I think (2) is a big point, which is why I've been so interested in who's going to lay fibre optic line in Uganda. There are a number of technologies like fibre optic Internet that are cheaper and better than we have in the States. The problem is that the States has companies like Comcast that own vast amounts of copper wire and don't want to upgrade. Developing countries could perhaps have nicer infrastructures than ours, which I find very exciting.

Sun is probably my favorite computer company lately. Their open sourcing of
java was pretty spectacular, as was their open sourcing of the Solaris operating system and the Niagara chip. All of these have been tremendous gains for the free software community. They've been making noises about also releasing Solaris under the latest GPL license (currently it's on a modified mozilla license). Since the Linux kernel is avoiding the new GPL currently, a Solaris switch could cause Solaris to be the open source Unix of choice for most geeks.

That said I don't have enough information to know what Sun intends or if it will follow up. I did, however, see a lot of signs for Sun Microsystems in Uganda, many more than I've ever seen in America. This to me suggests they actually are taking overseas markets seriously.

EDIT: While I'm on a Sun lovefest, I thought I'd link to some of their work on energy efficient datacenters. They claim up to 60% reduction in energy bills. Pretty cool.

I've been saying for a while that I want a supercomputer. The data analysis I've been doing has so far been okay on my main work computer, but it could be seriously improved. For my pre-dissertation work, some hierarchical data analysis took as much as 30 hours to complete. With that kind of time, it becomes difficult to put a lot of effort into making improving the model.

GPUs

It turns out there are several ways to get a 'supercomputer' for not that much money. One interesting way is to use the floating point capabilities of graphics cards for scientific floating point operations. This is the idea behind NVIDIA Tesla device that advertises itself as a supercomputer that goes in a PCI express slot. I believe the card retails at about $1,500, but that will likely come down soon. I'm not sure if it's possible to use a regular graphics card for a similar purpose. But NVIDIA sells graphics cards that cost up to $2,400. This implies to me that the cost of the Tesla is just due to the fact that it's a relatively high-end video card, not that its use as a supercomputer has increased the cost.

Playstation 3

Another possibility is to use a cluster of Playstation 3s. I remember when the Playstation II was introduced, several countries were banned from buying them because they could be connected to control military weaponry. I thought this was a joke at the time, but it turns out that game consoles have a ridiculous amount of power. Because of heavy competition and economies of scale, the power comes at a price that's quite difficult to beat. The result is an powerful off-the-shelf computer for under $700.

In the case of the Playstation 3, the secret to its usefulness is the new cell chip jointly developed by Sony, Toshiba, and IBM. The cell (a relative of the PowerPC chips from older macs) is a vector processing beast making it ideal for media (hence the PS3) and also potentially for scientific computation.There's a cool paper on using Playstation 3 consoles for supercomputing [in pdf form]. The paper indicates that the PS3 cell chips are crippled in some ways and that the hardware speeds single-precision floating point operations but not double-precision ones. Double precision operations are the more important for science. You can get much more performance by combining precisions according to the paper, but I'm not sure how to do that. At any rate, the next line of cell chips will have hardware support for double-precision operations, which suggests that I shouldn't invest in a cluster of PS3s right now.

Simple Cluster

Another possibility is simply to add another motherboard or two to my existing computer. An 8-core cluster can be built for under $1,300 (see microwulf). If I add my current motherboard to that, I'd have a 10-core cluster. The performance seems much less than that of either the GPU approach or the PS3 approach, but there is probably less need to convert everything to single-precision. The microwulf gets performance of about 26 gigaflops (billion floating point operations per second), whereas a PS3 gets 14 gigaflops of double-precision performance. With the mixed precision, one PS3 can get up to 155 gigaflops for a double-precision matrix. Since one PS3 is less than half the cost of a microwulf, it seems like a better idea just to get two PS3s and add them to my current computer.

Conclusion

I'm still not sure what I'm going to end up doing. The GPU idea is neat, but from my understanding you can only do single-precision calculations on them. If I need to do mixed precision to take advantage of either the GPU or the Playstation, then the GPU seems slightly better. The Tesla claims it can do up to 500 gigaflops, which beats the PS3's 155. There are a lot of resources out there and I only just now found out about all this, so it's going to take me some time to decide on exactly what kind of supercomputer I want.

This paper details how to write an application thant can monopolize CPU time without being detected. The program works on virtually all operating systems except OS X.

Basically, the CPU in most operating systems keeps track of programs by ticking like a clock. For each tick, the CPU determines which program was using the CPU at that time and bills the whole inter-tick interval to that program (typically 10-15 milliseconds).

The cheat program works by starting just after a tick and then stopping before the next tick, running let's say for 8 milliseconds. It then hands the CPU over to another program that runs for 2 milliseconds but gets billed for 10. When you looks your system monitor, it look like your normal programs are running.

It outlines how you would make a program so that you could run any arbitrary program this way, by typing something like

cheat 99% myprogram.exe

where 99% is the amount of CPU time you'd like to take up.

This would be really useful if, say, you had a lot of math computations to do and you wanted to take over a supercomputer or two. You could use the program to run the computer at 25% of the CPU and let other people be billed for it. Sadly, I'm too scared of getting kicked out of grad school to try this.