Commonly the windows are made of a cellulose and can be recycled, but food items that directly contact the paper will transfer oils and grease making the paper and cardboard difficult or impossible to recycle.
The above answer is not completely correct. I did a little on-line research and found that the windows are generally not recyclable. However, these windows are so common that most recycling facilities automatically remove these pieces of plastic as a part of the recycling process.