Preheat oven to 375. Halve any larger potatoes or quarter them if very large, keeping the small ones whole. Place on a roasting pan, or if you have one into a large cast iron skillet. Toss with the olive oil and the salt. Place in the oven for 35-45 minutes, stirring 3-4 times during baking, until the potatoes are cooked, browned and crispy. The exact amount of time will depend on the type of potato you use and how you cut them, so just check them occasionally toward the end of cooking time.
Meanwhile, make the dressing. If using whole anchovies, chop finely, smashing with the flat side of your knife, until a paste. Place in a bowl with the grated garlic, zest and juice from the ½ lemon, the vinegar and mustard. Stir well, then add the cheese, pepper and salt.
When the potatoes are done, remove from the oven and toss with the dressing in the still warm pan. Remove to a serving bowl and eat warm, or serve at room temperature.