These pilot burners can get crudded up with white crusty crap. They make a pilot burner cleaning wire type kit to clean the little beggers. Looks like the wires that you use to clean a cutting torch nozzle. Perhaps the pilot burner could use a good cleaning? Also, since the gas valve button won't pop back out, there could be additional issues with the gas valve. The gas valve does have pressure adjustment set screws on it for both the pilot burner and the main burner. If the furnace has worked in past years, these shouldn't need fussed with. They're only for initial setup. If, when the pilot burner is cleaned, you still have a weak pilot, the gas valve is the only suspect. Since you say that you have a weak pilot flame, the thermocouple is doing it's job if it won't let the furnace run. You can check the thermocouple output with your meter if you want. Just unscrew it from the gas valve and have a helper hold the pilot button in to put a flame on the thermocouple. Should have around 13mv min.
"Nothing is foolproof to a sufficiently talented fool"