The numbers in the SAE rating (e.g. 20W-40) refer to the
viscosity of the oil at low and high temperatures. Think of the
numbers as a measure of the oil's thickness, and the first number
is the thickness at low temperatures, while the second number is
the thickness at high temperatures.
When the engine is cold, you want the oil to be thinner so that it
doesn't provide too much resistance as the engine is working its
way up to operating temperature. However, when the temperature is
high, you want it to be thicker to reduce wear on the engine parts.
Viscosity index improvers are added to oils to give them these
properties at differing temperatures.
While thinner oils reduce resistance to the pistons as they move in
the engine block, the wear also increases. Some estimates are that
a 1% improvement in fuel efficiency can result in 30% more wear on
the parts.
Also, note that the oil in part helps to seal in the fuel-air
mixture as it is compressed and ignited - if it is too thin,
compression can be lost. This applies particularly to older and
more worn engines, when the space between the piston and cylinder
lining is increased, thereby giving the oil a larger gap to
fill.
Before choosing what oil to use, it would be best to first check
the manufacturer's specifications, because pistons and cylinders
are constructed differently in different engines, and therefore
require different oils. Also, it is best to perform a compression
test on your engine, especially if it is an older engine, to assess
its state of wear.
If it's a new engine and in good condition, you might be able to
use a 10W-40 oil and get away with it. On the other hand, chucking
10W-40 into an engine that's done 200,000 km will result in you not
being able to use your rear-view mirrors because all you'll see is
smoke. 
Over and above SAE ratings, you will also need to choose between
synthetic and mineral oils. Synthetic oils have the advantage of
longer life, so you can go further befoore needing to change oil,
but it may not be suitable for all engines. Again, check the
manufacturer's specs.