Make the chili garlic oil: in a small saucepan combine 1/2 cup olive oil, 4 crushed garlic cloves and 2 tablespoons crushed red pepper flakes. Bring just to a low boil, then reduce heat to low and simmer gently for 20–30 minutes. Remove from heat and let cool.
Prepare the tomatoes: place sliced heirloom tomatoes and halved cherry tomatoes in a bowl or on a plate. Drizzle with the cooled chili garlic oil and season with salt and pepper to taste; set aside to infuse.
Heat 2 tablespoons of the reserved olive oil in a medium Dutch oven or large skillet over medium-high heat.
Cook the chicken: add the chicken pieces, 2–3 minced garlic cloves, 2 tablespoons balsamic vinegar, 1 tablespoon smoked paprika and 1/2 teaspoon salt plus pepper to the pan. Toss to coat and cook, stirring occasionally, until browned and cooked through, about 5 minutes. Stir in the chopped oregano and cook 1 more minute, then transfer the chicken to a plate.
Sear the peppers: add another tablespoon of olive oil to the same pan, add the sliced bell peppers and a pinch of salt and pepper, and cook until the edges begin to caramelize, about 3–4 minutes.
Add the orzo and 1 tablespoon olive oil to the peppers, stir to coat, then pour in 2 1/2 cups water. Cook, stirring frequently, until most of the water is absorbed and the orzo is creamy and tender; if needed add an extra 1/2 cup water and continue cooking until soft.
Finish the dish: once the orzo is cooked, return the chicken and add the olives to the pan. Remove from heat and let warm through for about 3 minutes.
Serve: top the orzo and chicken with crumbled feta and serve alongside the tomatoes in chili oil.