Well, if the screenbuffer is 1kB it means that you're converting 512 bytes to "double bytes". Even at 4MHz that conversion can be done 30 times/second.
If the refreshing the screen takes 1.2s (!) this "bit doubling" is only ~3% of that time. The bottleneck is elsewhere.