Thanks, your answer is very helpful and informative. Just to be sure though, when you said point 4 above (for the high value resistor) did you mean point 2?
Indeed I did, sorry about that.

With the switch that makes sense, before starting on this I was originally thinking that there was high low and open but as you say it is either 0 or 5V.
Actually, you can, instead of switching the output between high and low switch the pin between output (high or low) and input (high impedance). But in this case you'll end up with the same problem as with the LED. When the pin is an input current would flow thru whatever "feeds" the switch, into the PIC pin, up thru the clamp diode and into the 5V rail making the voltage at (4) around 5.5V which might fool the external circuit into thinking the button is pressed. Obviously all this depends on what is actually inside that black box.