Arrange the gingercake in an even layer in the base of a 1.2 litre (2 pint) glass serving dish. Spoon the pineapple, orange segments and kiwifruits on top and drizzle with the rum. Pour the custard over the fruit and spread into an even layer.
Whip the cream in a bowl until it forms soft swirls, then fold in half the orange rind and half the lime rind. Spoon the cream over the custard, then sprinkle with the remaining fruit rinds. Chill until ready to serve.