Preheat the oven to 350˚F. Generously butter a 9- to 10-inch (12 cup) Bundt pan and set aside.
In a medium bowl, whisk the flour, baking soda and salt together. Set aside.
In the bowl of a stand mixer, beat the butter until creamy. Add the sugar and beat at medium speed until pale and fluffy. Beat in the vanilla, then add the eggs one at a time, beating for about 1 minute after each egg goes in. Reduce the mixer speed to low and mix in the bananas. Finally, mix in half the dry ingredients all the sour cream and then the rest of the flour mixture.
Carefully stir in Reese's peanut butter cup pieces. Scrape half of the batter into the prepared pan and smooth the top. Swirl in half of the the peanut butter, using a toothpick. Add the rest of the batter and smooth the top. Swirl in the remaining peanut butter, using a toothpick. Pound the pan on the counter so the cake batter will settle and even out.
Bake the cake for 65 to 75 minutes, or until a toothpick inserted into the center of the cake comes out clean. Check the cake after about 30 minutes - if it is browning too quickly, cover it loosely with a foil tent. Transfer the cake to a rack and cool for 10 minutes before unmolding onto the rack to cool to room temperature.
While the cake is cooling, make the peanut butter glaze. In a medium bowl, whisk together peanut butter, powdered sugar, milk, and vanilla extract. Whisk until smooth. When the cake is cool, drizzle glaze over the top of the cake. Cut into pieces and serve.