Preheat the oven to 400°F.
Remove the florets from the head of cauliflower. Prepare the red bell pepper by stemming, seeding, and cutting into thin strips OR halve the cherry tomatoes. Trim and slice the cremini mushrooms if they are large.
In a large bowl, combine the cauliflower florets, red pepper strips or halved cherry tomatoes, and mushrooms. Add 1/2 teaspoon kosher or sea salt, freshly ground black pepper to taste, 1 tablespoon lemon juice (freshly squeezed), and 2 tablespoons extra virgin olive oil. Toss until everything is evenly coated.
Spread the vegetable mixture in a single layer on a rimmed baking sheet.
Roast in the preheated oven for 20 to 25 minutes, stirring once halfway through, until the cauliflower is slightly browned and tender.
Remove the baking sheet from the oven and stir in 1 cup green olives (use pitted olives if available) and 1 tablespoon capers until evenly distributed.
Stir in 1/4 cup chopped parsley, if using. Taste and adjust seasoning with more salt or pepper if needed. Serve warm.