I would not mess with the weak pullups , just pretend they are not there for now.
Can you post all of your code and configs? Makes it a bit easier to see what you have.