CP stands for Climax Points, which are used to feel-up the Moetans.

You will gain 4 CP after beating the bosses on every 10th stage.

The chest dropped by the flying Fairy (see Fairy Chest FAQ) can also drop CP.

The Wall Spirit (Stage 250) and Kraken Spirit (Stage 700) will help improve CP gain (Wall Spirit will only be viable at extreme levels).