You were probably not contagious during those first days (at least for everyone except your partner).The threshold for the rapid test is higher than what the PCR detects. The antigen test is highly reliable for catching contagiousness, not just any viral load. Michael Mina (Harvard Biologist) presents the data in a great way if you want to check.