The forumula they have in the maual is (1000000/baud)-20, which results in -11.
As Al pointed out (but didn't really spell out) the 1000000 comes from Fosc/4 and since you're running at 20MHz the formula becomes (5000000/115200) - 20 = 23. I don't know if 115200 "should" work at 20MHz but if you can't get it going try DEBUG instead.

/Henrik.