Put the potatoes in a saucepan of lightly salted cold water, bring to the boil and cook for 15–20 minutes or until just cooked through. Drain the potatoes and let them cool slightly.
Meanwhile, chop the capers and combine them with the mayonnaise, lemon juice and horseradish. Season to taste with salt and pepper. Put the warm potatoes in a large salad bowl, add the mayonnaise dressing and stir to combine thoroughly.
Arrange the salmon on 4 plates, top with the potatoes and garnish with mustard and cress.