I don't know how late they go (probably depends on the court), but Family Search has US naturalization records from various courts as part of their unindexed collections. I believe you need the certificate number (you can double check the naturalization index records) and the court name/district. I found one from 1951 issued in the Eastern District of New York.
Also, according to the Social Security website, noncitizens can have Social Security numbers.
Sent from my Samsung Galaxy Tab