So as seen in the image, why do I lose the challenge when the opponent only has 3 fives, not 4? This has been bugging me for a while now.

According to the rules "...make a bid based on the quantity and face values of the dice they believe to be in play." This means that their hand and your hand are counted together. So on that hand, there are technically 6 - 5's...3 in their hand and 3 in yours. Hope this helps.