Lay fish on a 13x9" casserole pan. Stuff each cavity with salt, pepper, thyme, lemon slices, and orange slices. Drizzle oil over top and sprinkle with more salt and pepper. Place in the oven and cook about 20 minutes until skin crisps up.
For the Fennel
Trim the outer leaves and remove and discard stalks. (reserve fronds if you like to toss with fennel at the end). Cut the bulbs in have lengthwise and cut out the core. Then slice thin. Place in a bowl with the rest of the ingredients and toss well. Spread on a big baking sheet and bake for 20 minutes until golden, tossing and stirring half way through.