I admit I'm not sure of the original question since the proposition that we can't trace our families further than our grandparents seems patently absurd. I have records for all of my great-grandparents, in several cases have reliable records for 2x great-grandparents, some 3rd and 4th great-grandparents, and in two cases can go back 9 generations.
On my Sztejsapir line I can go back to my 7th great-grandfather, Jankiel Roterozen 1720-1790 (Rajgrod, Poland). I have his death record through JRI-Poland and records for both my 6th great-grandparents. On a different branch of that line, I have records for another 7th great-grandfather Zelik Ludwinowski c.1705-? (Suwalki, Poland).
I have a few branches were I haven't managed go back past my great-grandparents. But that doesn't mean the records aren't there, and I couldn't document those branches to earlier generations; I just haven't connected the right dots yet (I hope).
Though I haven't found anyone born before 1700, last year thought I'd traced my Ludwinowski line back 20 generations to the 12th century. One evening Ancestry prompted me with "potential father" and "potential mother" links for my 3x great-grandmother Slawa Ludwinowska. I naively accepted the new records and was peppered with even more hints leading back to 12th c. English-Norman nobility, via a 15th c. Portuguese slave, two famous rabbis in exile in London, and several prominent Sephardi families in Amsterdam, with connections to Fez, Constantinople and Venice. The histories of the Sarfati and da Crasto families – among others – are well-documented – through extensive records in Amsterdam, combined with Inquisition records and other historical accounts. If you can link your tree to one of their members you will suddenly have a very, very old lineage indeed.
However, in my case, the link to my 3x great-grandmother was specious. After more careful investigation and consultations with other Ludwinowski descendants, I learned the link was based on an unsupported record in Geni, since corrected. (Given how pervasive Geni records are, it's disappointing how easily mistakes can be entered there and how unreliable the means of correcting them is.) The reputed 4x great-grandmother turned out to be a fictional character, the product of a conspiracy tract trying to prove that most of the leading Kabbalists of the early modern period were secret Catholics. A couple of the people in the account are famous, but some key characters never appear in any historical records and/or the key events (marriages and births) can't be documented. Examining available records, there are at least two significant gaps in the narrative which decisively uncouple my family from that "old lineage."
The history of the the earlier families with well-documented lineages is real. But the link to my Ludwinowski ancestors is extremely doubtful. That doesn't keep the majority of online Ludwinowski family trees I've surveyed from including those lineages. There will be Ludwinowski heirs who believe they are descended from a Norman baron whose daughter married an exiled Portuguese diplomat whose son fathered a child with his Portuguese Jewish slave who became the matriarch of a prominent Sephardi family in Amsterdam. After all, before the Internet and digitized records, what was genealogy but family lore supported, at best, with a handwritten family tree? But now we need DNA and contemporary reports before we can say the woman we called "mom" is really our mother.
My point (and I do have one) is that some very old Jewish lineages can be documented. Let's keep in mind we are all descend from old families: we just can't all document them past a certain point. It seems to me that the gap for many of us takes place in the Eastern European communities before 1800, sometimes earlier, where records for Jews weren't made or didn't survive or still haven't surfaced. But there are some Jewish communities with good records of very old lineages and some of them are our ancestors.
Lee Jaffe, Santa Cruz
Brodowicz > Ludwinowska > Braun > Stzejnsapir > Jaffe
Wizajny, Suwalki > Philadelphia, PA