You can change jobs after receiving the key item if you wish. You do not have to be a Bard to fight Tros.
Head to Ranguemont Pass and proceed past the granite door in the room with the San d'Orian guards.
Go to (D-5) in the northwest corner of the map. There are some level 90+ monsters on the way to (D-5) that aggro players of any level, so Sneak and Invisible are required. Circumspection from the Grounds Tome grants these effects for 5 tabs.