Keep in mind that Lore doesn’t max out at 5, it maxes out at 10. At least in pre 3e world. So you kind of need space for 5 more levels of “knows things not available to everyone.”
The way I work it is by orders of magnitude. A person with Lore 1 knows things only 1 in 10 people know. Lore 2 is one in a hundred. Lore 3 is one in a thousand. And so on. Specialties add to this as well.
So if you have Lore 10 Relevant specialty 3 you know things that only one in 10,000,000,000,000 people would be aware of.