Season 11 - BABF11 - Missionary: Impossible
Elmo (the red one) and Oscar the Grouch from Sesame Street shows up in this episode.
Big Bird is also in this episode.
Season 17 - GABF16 - The Girl Who Slept Too Little

Maggie is watching Count Von Count on Sesame Street while Marge worries about the cemetery being their new neighbour.

Marge: mmmrf, this is a little too spooky for me. I don't even like watching the Count on Sesame Street.
The Count: One coconut, two coconuts, three coconuts... mwahahahaha!
Marge: Go back to your own country!