Thanks Mack, MCLR is connected to VDD through a 10K resistor and also grounded with a bypass ceramic capacitor of 0.1uf. I also placed a big 1000 uf capacitor between ground and 5V line but oscillator still doesnt start at 5V supply but it does if the VDD is reduced to 3.4V. Its behaviour is same if fuse is set to Internal Oscillator with OSCCON and OSCTUNE registers set properly.
Right now I have loaded the following simplest program for testing:

DEVICE = 16F819

WARNINGS 0

Config HS_OSC, WDT_OFF, PWRTE_OFF, BODEN_OFF, LVP_OFF, CP_OFF, MCLRE_ON
WARNINGS 1
XTAL= 12

ALL_DIGITAL TRUE
OUTPUT PORTB
loop:
PORTB=255
DELAYMS 500
PORTB=0
DELAYMS 500
GOTO loop