Cube your brioche and leave it out overnight on the counter to dry out. I usually leave it on a baking tray.
The next day, grease a 9x13 baking pan.
Mix the ingredients for the sticky caramel together until combined. Add half of it to the bottom of the baking pan with a spatula. Set the rest aside.
Add brioche cubes to the baking pan on top of the sticky caramel.
Make the custard mixture. Crack eggs and lightly whisk. Add milk, vanilla, cinnamon, nutmeg, brown sugar, and salt and whisk until combined.
Slowly pour the custard over the brioche in the baking pan. Once I’m done, I like to use my hands to gently toss the brioche and ensure more pieces get soaked in the custard. Proceed with caution, as your brioche might start to disintegrate on you if you overwork it.
Add chopped pecans to the rest of the sticky caramel mixture and give it a good stir. Spoon over the entire casserole.
Now you have two options: you can refrigerate the casserole, covered, overnight and bake in the morning, or let it rest for 20-30 minutes in the refrigerator and then bake right away.
When you are ready to bake, preheat oven to 375 degrees F and bake for 35-40 minutes or until the top is golden brown and the custard is set. If, during the baking, you notice the top starting to get too brown, cover with tin foil. Remove and serve with whipped cream, maple syrup or both!