The remaining contacts are likely in a “stopped” status. This has a few trigger reasons.
You can find out more about why your contacts are stopped at the following link: https://help.bisonsphere.com/en/articles/14-why-are-my-contacts-stopped-in-a-campaign