Melt the butter in a large heavy-based saucepan over a medium heat. Add the onion and cook for 5 minutes or until softened. Add the apples and cayenne and cook, stirring, for another 2 minutes.
Pour in the stock, then add the potatoes. Bring to the boil, then reduce the heat and simmer gently for 15–18 minutes until the apples and potatoes are very tender.
Blend the soup in batches in a blender or food processor until very smooth, then transfer to a clean saucepan. Reheat gently and stir in the hot milk. Taste and adjust the seasoning if necessary.
Make the apple garnish, meanwhile. Melt the butter in a small frying pan, add the diced apple and cook over a high heat until crisp.
Serve the soup in warm bowls, garnishing each portion with some diced apple and a sprinkling of cayenne.