I submit a blending of some rules that I've experienced. Take the 'you must be trained' aspect from, say, WoW, and combine it with the rule from Dungeons & Dragons (pen-paper version, not computer) that states your character must choose a weapon or weapons to become 'proficient' with.

Before a character could use a weapon, the training on that weapon must first be sought out and paid / quested for. At this stage, a character could maybe acquire training on every weapon in the game. Of course, some usual class and racial limitations could be added. However, while the character would have a very base knowledge on how to use the weapon, becoming proficient with one or two weapons would have to be a choice, and would only happen over the course of time. A character using a weapon he is not proficient with would do so with a penalty, and vice-versa when using a weapon he was proficient with.

I might even throw in something like what's used in Ultima Online, where each character has a finite number of proficiency points to be used in becoming so with a weapon of choice. Every level gain awards a number of those points that the character can spend as he chooses, until the level cap is reached. For example, Dave the Barbarian (who's in the corner and needs another drink, quick!) reaches level 20, where he gets 6 weapon proficiency points. He has been trained on mace (wields with a penalty), hammer (wields with a penalty), and longsword (proficient, wields with a bonus). He can spend all 6 points on longsword, increasing his skill, or divide the points between two or all his weapons. At some point, he may achieve a proficiency with all his weapons, if he chooses.

I think this would work other equipment, like armor and shields, also. The more you use an item, the better you become with it. I don't like those systems that, as you said, allow me to suddenly "possess that wondrous knowledge I didn't have a second before". That is a trait that should be saved for those high quality uber-magical items. Eh, maybe I'll see it one day!