SSH by default will always try all keys known by the agent in addition to any identity files. To keep this from happening, you can specify IdentitiesOnly=yes in addition to specifying a particular key to use when authenticating.
ssh -i ~/.ssh/id_rsa me@myserver hostname
will give you Received disconnect from myserver: 2: Too many authentication failures for me
However, if you add IdentitiesOnly=yes like so…
ssh -o IdentitiesOnly=yes -i ~/.ssh/id_rsa me@myserver hostname
you will see…
Generally this is caused by having more than 5 keys loaded in ssh-agent.
You can remove keys by running
ssh-add -d ~/.ssh/[key]