I just wasted an hour on this, so I hope that explaining the solution here will help someone. If you’re using TortoiseSVN and PuTTY, perhaps using a private key created with PuTTYgen (like me), then this probably applies to you. I’m using Windows XP Pro, but Home is probably the same. I didn’t have this problem before because I just used cygwin’s ssh.exe which worked fine.
PuTTY Internal Error
Invalid Port Number
For TortoiseSVN’s SSH cilent, leave the box blank. Do not choose putty.exe, that doesn’t work. Blank is fine, TortoisePlink is fine. Otherwise you see the error above. When you enter the URL to the repository, use the PuTTY saved session name instead of the server name. For example, if your session is MyConnection, use MyConnection for the server instead of, for example, svn.osafoundation.org.

Thanks! Very helpul
Merci! (thanx)
Like everyone else here, I was banging my head against the wall for quite a while before finding your post — thanks for making it!
Thank you Elliot, you probably saved me a big chunk of time. I came across this problem while trying to relocate my wc to a non-standard port address.
Bless You (and google)
Thanks! saved me lots of troubleshooting.