I'm looking into it too.
Trying to build a board.

But first I have to ask ...
Do you have VDD (3.3V) tied to VUSB?
There's no internal regulator on those chips.