Preheat oven to 325°-350°. Peel potatoes and cut into 6 or 8 chunks. Place in large saucepan with cold water. Water should just cover the potatoes when all are added. Cook over high heat until water boils, reduce heat slightly, and cook until potatoes are tender, about 15 minutes.
Drain well and return to saucepan over low heat, tossing well to cook away any remaining liquid. Using a masher or electric beater on low speed, mash potatoes, adding warm milk just until potatoes begin to take on creamy texture but before all lumps are gone.
Add cream cheese, butter and sour cream, mashing or beating until potatoes are smooth. Stir in salt and pepper and adjust seasoning, if needed. Add more milk, if smoother, thinner potatoes are desired.
Place in 2-quart baking dish and bake until heated through and peaks turn brown, about 30 to 35 minutes. May be refrigerated 2 to 3 days before baking.