To make the custard, place the milk and vanilla extract in a saucepan and heat until just below boiling point. Place the egg yolks, sugar and cornflour in a heatproof bowl and beat with a hand-held electric whisk until pale and thick, then gradually whisk in the warm milk. Return to the pan and cook over a medium heat for 2–3 minutes, whisking continuously, until the custard has thickened. Transfer to a bowl, cover with clingfilm to prevent a skin forming and chill.
Meanwhile, arrange the trifle sponges in the base of a 20 cm (8 inch) trifle bowl and pour over the sherry, if using. Mix together the jam and lemon juice in a bowl and spoon over the sponges. Add the raspberries and top with the cooled custard.
Lightly whip the cream in a bowl with a hand-held electric whisk until it forms soft peaks. Spoon on top of the custard and decorate with the almonds. Serve immediately or chill until ready to serve.