Thats what i was going to suggest. Buffing it out. Two things come into play for that process: 1) buffer speed/motion 2) metal flake.
1) You can burn the clear/paint easy with a buffer if not careful.
2) if the clearcoat has worn down to where you can feel the flakes rubbing your hand across you are in for more work than just buffing. You will need to get it to a point where you can add more layers of clear back on top then buff out.
If its still smooth just discolored then it will buff out using the right compound.