Exactly! Any site that's going to accept payments online should have what's called a valid "SSL Certificate." You'll know if they do by using the "https://..."
Just like it was stated above, try typing in the site's domain using "https" and see what warnings (if any) you get. Assuming you're using a secure site, online bill pay should be as safe as any.
The other thing to keep in mind is that most reputable credit card companies often go above and beyond in protecting their clients from online fraud. If you ever DID have an issue with someone hacking your credit card online, chances are you'd be protected by the card company.
