Line a 12-section bun tray with paper cake cases. Pull the lavender flowers from their stems and put in a bowl with the butter, caster sugar, orange rind, eggs, flour and baking powder. Beat with a hand-held electric whisk for about a minute until light and creamy. Divide the cake mixture between the paper cases.
Bake in a preheated oven, 180°C (350°F), Gas Mark 4, for 20 minutes or until risen and just firm to the touch. Transfer to a wire rack to cool.
Mix the icing sugar with enough orange juice in a bowl to make a thin glacé icing. Colour with the food colouring. Spread over the cakes and decorate with small sprigs of lavender flowers.