Grease a baking sheet.
Sift the flour and baking powder into a food processor. Pull the lavender flowers from the stems and add to the bowl with the butter. Blend until the mixture resembles breadcrumbs. Add the buttermilk and blend briefly to make a soft dough.
Turn out onto a lightly floured surface and roll out to 1.5 cm (¾ inch) thick. Cut out 24 rounds using a 3 cm (1¼ inch) round cutter, rerolling the trimmings as necessary. Transfer to the baking sheet, brush with milk or beaten egg and sprinkle with caster sugar.
Bake in a preheated oven, 220°C (425°F), Gas Mark 7, for 8-10 minutes until risen and pale golden. Transfer to a wire rack to cool.
Whip the cream until just holding its shape. Split the scones and sandwich with the jam and whipped cream.