I think it might be easier to think of it this way...

You are an accelerometer. You are blind folded sitting in a car. It's completely quite. The car isn't moving.

How can you tell if the car starts moving? You get pushed back in the seat a little bit for little awhile.
How can you tell if the car is speeding up? You get pushed back in the seat for a bit for a little while.
How can you tell if the car is slowing down? You get 'pulled' forward out of the seat a little bit for a little while.
If you don't get pushed back in the seat or 'pulled' forward out of the seat, did the car change speed?