On a sheet pan, mix the broccoli, oil, salt, and pepper. Roast for 8 to 10 minutes, or until the broccoli is evenly roasted but not burnt. Remove from the oven and allow to cool.
Heat a skillet over medium-low heat.
Spread butter evenly on one side of each slice of bread. Dividing the ingredients evenly, layer cheese on the unbuttered side of half of the bread slices, then add chopped broccoli in the middle and more cheese on top so that the broccoli lies between two layers of cheese. This keeps the small pieces of broccoli from falling out as the cheese melts. Top each with another slice of bread, butter side out.
"Grill" in the skillet, turning sandwiches over once, until the cheese melts and the bread is evenly toasted.