According to the source of all knowledge, Wikipedia, Intel co-founder Moore’s law goes as follows:
the number of transistors that can be inexpensively placed on an integrated circuit is increasing exponentially, doubling approximately every two years.
This is generally interpreted as meaning that processing speed and memory will improve at exponential rates as well.
Now we’ve come to the limit of how small we can get those transistors. Transistor gates are 5 atoms wide, and this is too small to avoid leakage. Intel seems to have found some way to decrease that leakage (high-k materials), but consensus is that the transistors won’t get much smaller.
Now hardware vendor’s response to this has been: don’t worry, show must go on. If we can’t make them smaller, we’ll just add more of them. Hence the bi-core, quad-core desktop machines, and the multiple CPU servers.
The only thing is that strictly speaking Moore’s law doesn’t apply any more: we’re not speaking about the number of transistors on one circuit, but on several circuit.
And Moore’s law corollary is no longer true either. Several CPU’s means that parallel processing will speed up. But there are sequential computation (i admit i’m struggling to find a real-world example here), which won’t get faster by this multiplication of CPUs. We’re not talking about pure increases in processing power like the last 20 years – the nature of the increase is different.
This multiplication of CPU’s has consequences for our software. Two ways to deal with this:
- implicit
modern versions of compilers (and VMs) now deal with this parallelization. I’ve read a fascinating article in one of the IEEE magazines (unfortunately i forgot which one) which describe how the compiler works through the code to recognize operations which can be split in multiple parallel bits.
This leaves the software developer free to continue working as if nothing happened – although it would help if they try to make their operations as parallelizable as possible (no simple matter, i know). - explicit
several languages have been gaining momentum by the fact that they allow the developer to explicitly decide how it’s going to go (disclaimer: i haven’t studied any of these languages myself, though i might).
Erlang works with message passing between threads, which apparently makes programming distributed/parallel systems simpler. Haskell allows you to annotate your program for parallelization.
I’m curious – it should change the way we work, if only a little bit.
There was a debate about the future of computing at Javapolis this year. I think it was James Gosling who speculated that if Moore’s law remains applicable, then in a few years we’d have a machine with 4000 CPU’s, and we’d have to deal with that.
I had to grin: I thought the point was that we can’t make them any smaller ? So in fact i’d advise to start a physical training program, so we can carry round the 4000 CPU laptop.
Very intresting article. Do you know the Blinn’s law (from the name of the bump mapping inventor) often used in CG industry : It states that regardless of hardware, all renderings take about the same amount of time. Because more power you have, more effects you add.
And in these applications, parallelization is not a real issue (rendering farm were already built on a multi CPU basis before multi cores)
So, things are not going to change, rendering won’t take less time in the future
I believe one very important aspect needs to be added to the equation and that is power (electrical power). Both for the limit of how much heat can be extracted from the chip in high-end computing and how much battery power can be consumed for low-power/portable computing, the real challenge is MIPS/Watt (or reverse Joule/instruction). The Moore law was about Si (silicon) area being the limiting factor. Let’s say a 20*20 mm2 chip is the maximum practical size for a chip, the Moore’s law dictates how much transistors can go on a chip and at how much GHz they can run. But the real challenge now is MIPS/Watt (and not longer MIPS/mm2). And the core trick is that lower frequencies are more efficient in MIPS/Watt (because it allows you to further lower the voltage of the chip, which to the 2nd power reduces the energy consumed by each flip of a bit (E = C.V2/2)). The most extreme example I remember up to now is the cochlear hearing implant of Cochlear.be (developed in Mechelen), that uses 16 parallel dedicated processors at 5 MHz (yes, 0.005 GHz) to do all audio processing for a power of a few microWatts (so the entire processor can run for a day on a very small charge that is applied with an electromagnetic coil to the electronics implanted inside the body of the patient). I wrote something (in Dutch) earlier about the questions of languages optimal for parallel processing.
@pixel: yes, graphical application seem very parallelizable to me.
@peter : very interesting ! you’re much more of an expert than i am. A new, more complex law is indeed called for
http://newsroom.intel.com/community/intel_newsroom/blog/2011/05/04/intel-reinvents-transistors-using-new-3-d-structure