I've drilled out a few rims with no problems, I don't know why the 321's would be different unless you're drilling with a hammer and chisel. How do you think the first hole got there?
It got there in a professional machine shop with excellent tools.
I drilled an extra hole in an aluminum windsurfing boom once. It worked with "no problems". After two years the boom cracked. Guess where!
When you drill a hole you put a stress concentration area into material. It will naturally be weaker at that point.
When you drill a bigger hole, you make a larger stress concentration.
When you drill a hole with jagged burrs - you bet - it gets even weaker
Don't do it.