The plot thickens...

Long stort short, flashed the device with the PICKit3, could not reproduce the error. Reflashed the bootloader into it and now I can no longer reproduce the error at will - even my ACTUAL code now works without that magic PAUSE 5 in there.

Before doing that I had reduced the test code to this:
Code:
Main:
    IF INTCON.2 = 1 THEN        ' TMR0 interrupt flag? 
        LD2 = ~LD2              ' Toggle LED so we know we're alive and kickin'
        INTCON.2 = 0
    ENDIF

    If BTN1 = 0 THEN
'        Buffer[107] = $00       ' Testcase with LONGs
        Buffer[117] = $00       ' Testcase without LONGs
        WHILE BTN1 = 0 : WEND
    ENDIF
    
    IF BTN2 = 0 THEN
'        Pause 5
'        Buffer[108] = $00       ' Testcase with LONGs
        Buffer[118] = $00       ' Testcase without LONGs       
        WHILE BTN2 = 0 :WEND
    ENDIF

Goto Main
As soon as the code for BTN2 was executed it crashed while the code for BTN1 worked fine. Enabling the PAUSE 5 made it work. And the value being written made no difference.

As for the question if bootloaders are worth it I'd say maby. In this particular case I'm juggling two boards but I only have one PICKit so swapping between boards would be less than ideal. The USB bootloader is super fast but makes up for that by forcing you to manually reset the device and press connect in the PC application before the bootloader times out.

Strange one this.