Let's see all the code...
Are you resetting the interrupt flags when you go to process the interrupt itself? They don't reset themselves...