Prepare the Baking Dish: Butter a 9x13-inch baking dish and set aside.
Assemble the Bread: Add 1 14-ounce brioche loaf, cut into 1-inch cubes to the baking dish, spreading it evenly. Set aside.
Mix the Custard: In a large mixing bowl or 8-cup measuring cup, whisk 5 large eggs. Add 1 cup pumpkin puree and 1/2 cup packed light brown sugar. Whisk until well combined.
Season the Custard: Add 2 teaspoons pumpkin pie spice and 1 teaspoon vanilla extract. Whisk again until fully incorporated.
Add the Milk: Pour in the 2 cups whole milk and whisk until fully blended.
Soak the Bread: Pour the custard mixture over the bread. Use a fork to gently press the bread down so every piece is coated.
Make the Streusel: In a medium bowl, combine 1/2 cup all-purpose flour, 1/2 cup packed light brown sugar, 1/3 cup chopped walnuts, 1 teaspoon ground cinnamon, and 1/4 teaspoon ground nutmeg. Mix to combine, then stir in 4 tablespoons salted butter using a fork until crumbly.
Top the Casserole: Sprinkle the streusel evenly over the soaked bread. Do not mix.
Preheat the Oven: Cover the dish and refrigerate while the oven preheats to 350°F.
Bake: Remove the cover and bake for 45 to 50 minutes, or until golden brown and set in the center.
Cool & Serve: Let cool for 10 to 15 minutes before serving. Enjoy warm!
Notes
Make-Ahead Tips: You can prepare the casserole up to Step 6 the night before. Cover tightly with plastic wrap and refrigerate overnight. Bring to room temperature while the oven preheats the next morning.Storage: Store leftovers in an airtight container in the refrigerator for up to 3 days.Freezing Tips: Not recommended. The texture of the custard base may become watery after freezing and thawing.Reheating Tips: Reheat individual portions in the air fryer at 300°F for 5 minutes, or in the microwave for 30–60 seconds until warmed through.Customization Ideas: Add chopped pecans, chocolate chips, or dried cranberries before baking. You can also drizzle with maple syrup, caramel sauce, or top with whipped cream for extra flair.