You need charger and step up with enable pin. But you will need step up with load disconnect during shutdown.
If there is no power on usb port, enable step up and check if slave device is connected. If no, turn it off. Repeat that every 1-2sec.
Or you can monitor current sensor to usb current, and if there is no current, then no device is attached.
If you have power on usb, just enable charger, and that is all you need.
Other option is to bust battery voltage to 4,6V. If that is enough for slave device. When Vusb is higher than 4,5V then disable step up, and enable charger.