I was wrong. A user uploaded a malicious file. I deleted it, banned the user, and asked Google to take another look at the site.
It took a while for the idea to get through my thick skull to look at some of the recent uploads, but it did, and now that I'm a little wiser, I'm going to look at new user uploads, and if they contain exe files, it's an immediate ban.