Mix together the orange rind and juice, lemon rind, raisins, dried fruit and ginger in a bowl then pour over the brandy. Stir well, cover and leave to soak overnight.
Grease and line an 18 cm (7 inch) square or a 20 cm (8 inch) round deep cake tin. Wrap and tie a double thickness of brown paper or newspaper around the outside of the tin to come 5 cm (2 inches) above the rim.
Beat together the butter and sugar in a bowl until light and fluffy. Sift together the flours and spice into a separate bowl and stir in the ground almonds. Gradually add the eggs to the creamed mixture, adding a little flour mixture if it begins to curdle. Fold in the remaining flour mixture and the soaked fruit and any juice.
Spoon the mixture into the prepared tin and place in a preheated oven, 160°C (325°F), Gas Mark 3, for 1½–2 hours or until a skewer inserted into the middle of the cake comes out clean. Remove from the oven, remove the paper and transfer to a wire rack to cool.
Decorate the cooled cake with the dried fruit or nuts and glaze with the apricot jam.