You can jigsaw it since the flanges will likely cover a multitude of sin. Put masking tape on the transom to mark on so it doesn't scuff up your gel. If you do it from outside the boat, be aware that the teeth and motion of a sabersaw with want to pull the material toward you, possibly taking off a chip of gel. Small teeth, more time, less risk.
If you want to use a hole saw, you won't have material for your pilot to keep it centered, so you'll have to attach a 1x6 block to the inside of the transom with another block mounted to that block to bring your pilot surface closer to the outside of the transom. If you screw the block to the inside, drill small pilot holes into the transom so you don't split the wood/glass. And make damn sure the screws won't get close to the gel. I've had to do similar even boring bigger holes in door slabs for new deadbolts. Kinda like what this fellow is about to do.
