Pay all my bills on the internet...
I use my bank's "Bill Pay" functionality for this. It is really quite flexible and functional. When you pay a bill, it tells you about when the payment will be delivered. There's one section where you list all the companies (and people) whose bills that you pay, your account number, their address, etc. There's another section where you can list the status of all your recent payments, whether and when they were delivered and so on. You can make the payments automatic, or just make them yourself when the time is right. Personally, I make all my payments manually, because that's the way that I like to do it *smile*. This also reminds me of all the bills that I'm paying, so I don't get over my head in bills to pay.
The only problems that I've had were when the company that I was paying had changed their address, or changed my account number and I didn't notice. You do get notice of when a payment is not deliverable, at which point you can research what happened. This hardly ever happens, but it has happened a couple of times over the 5 years (or so) that I've been using this. And you can always call up whoever is getting paid to make sure that they got your payment. I did this a few times to reassure myself that it was working okay at the beginning , and I still do it for things like property taxes where it really matters that the payment arrives, and arrives on time.
As far as guarding your identity online, I like to centralize things so that as few internet companies as possible have my personal information. My bank already had my personal information, so just starting to pay bills from there wasn't very scary. You need to make sure that you have a really good password that isn't easily cracked, and make sure that you choose carefully when picking where you're going to pay your bills from...You can also get 3 free credit reports per year, which I generally use, since companies aren't always forthcoming in telling their customers that their database has been hacked. The good thing about credit reports is that stuff that you don't recognize stands out in them, and you can track them down. I've never had anyone do anything with my identity in all the years of buying stuff and banking and paying bills online. Maybe I've been lucky, but I'm also careful with who has my personal information, and like I said, I check fairly often to see if my credit has been compromised.
One last thing, the bank that I use is a brick-and-mortar bank, where I mostly do online banking (except for using the ATM's). They have something called "safe-pass", where while you're logging in they send you a code on your cell phone, and you have to put in that code as well to log in. What this means is that not only does the identity thief have to have your password, they have to have your cell phone as well, or they can't log in as you. I find that this gives me just a little more secure feeling, since I always know where my cell phone is *smile*. Nothing is fail safe, but this feels safer than just putting in my password.
So I guess that's about it. Just be careful, think about what you're doing online. Take the more secure path, even if it's more trouble. Use strong passwords, and change them occasionally. You can use one of those "password vault" applications where you have one super-strong password that unlocks the rest of your strong passwords, so you don't have to remember them all. That's another thing, don't use the same password for multiple money-related or personal information related internet sites. You don't want to be set up so that if one of your passwords gets cracked then that allows access to all your "critical" web sites.