You shouldn't second guess yourself if you did the right thing. It's never easy taking this decision, but put yourself in his shoes. If you were in pain, tired, knew your end was near. You would have stopped it if you could.
You put his needs over your own, and that's a bravery worth admiring. I lost a precious friend too two days ago. I don't know if the pain will ever go away. But I know that I will see her again when my life on this earth has ended. Until then, I will keep holding on and share the love that she taught me. You should do the same, for there are things that an amazing furry friend can teach you, that other people cannot.
Keep his memories close, grief for him, speak to the empty walls, because I'm sure he can hear you. And you will be reunited, and then you can tell him about all the wonderful things that happened in your life once he was gone, and he will be grateful for that.