Since email is used to recover lost passwords, just checking/sending mail is equivalent to "the keys to every website I have an account on". By trusting Google to keep your email safe, you're trusting them to keep your entire online presence safe. In that light, letting someone run code on your computer seems relatively tame.