I have a POA for my husband and his mother (who is in a nursing home); they both are not active until my husband is not able to make responsible decisions. He no long has any idea about money; he thinks he can just keep buying. I have taken all the charge cards and check books and his computer; he stills sees things on TV and gets email on his phone.
I have POA for my Dad, even though he was doing ok, his caregivers were finding bills thrown into the wastebasket. Oops. So I went over to Dad's house and dug through the recycling only to find more unpaid and over due bills. Not good. That in itself told me it is time to take over the bill paying... for me I had all of Dad's bills sent to my house so I could have control over them. For you, if possible you get to the mailbox before hubby does.
You did the right thing taking your hubby's credit cards, check books and computer. I would suggest giving him an allowance so he can have some dollar bills in his wallet. Your hubby is light years ahead of me by him getting his email on his phone... my phone doesn't even offer that option. Is there anyway you can turn off that option on his phone? How much email does he get? Can friends/relatives write to your email instead?