What FPS are you recording at? YouTube will step down anything above 30 down to 30. This can cause some weird artifacting and distortions to appear. By recording at 30 FPS, you're likely to see better quality of uploaded videos.
Im away from my PC this weekend, so I can't compare your setup to mine, but I can tell you that I run at around 6000 bit rate and let my graphics card do all the encoding. A lot of the games I play tend to be CPU relent then GPU, which allows up my gtx970 to do most of the work for recording.