There are many ways to dramatically lower power usage if you're willing to accept slower speeds. Intel has recently demoed some extremely low-voltage transistors that used power in the microwatts. You can make up for this dramatically lower speed by adding far more circuitry using 3D chip-stacking.

Another possible solution is to use 3D chip stacking to add dedicated hardware for a wide variety of tasks, most of which is turned off when not being used.