Preheat oven to 375°F (190°C). Pat chicken thighs dry and season both sides with oregano, smoked paprika, garlic powder, 1/2 teaspoon salt, and pepper.
Heat olive oil in a large oven-safe skillet over medium-high heat. Sear chicken skin-side down until golden brown, about 5 minutes per side. Transfer to a plate.
Reduce heat to medium. Add rice to the skillet and toast for 2 minutes, stirring. Add garlic and cook 30 seconds until fragrant.
Stir in chicken broth, lemon zest, lemon juice, and remaining 1/2 teaspoon salt. Bring to a simmer, scraping up any browned bits.
Nestle chicken thighs on top of rice, skin-side up. Scatter olives around. Transfer skillet to oven and bake uncovered for 25–30 minutes, until rice is tender and chicken reaches 175°F.
Remove from oven and sprinkle with feta and fresh dill. Let rest 5 minutes before serving.