Without knowing more about the reason you never got aid (wealthy parents refused to pay tuition?), there's not much I can say about the general applicability of your anecdote.
There are plenty of articles out there documenting the inaccessibility of higher education. In my experience, and many others, the "aid" system is a ploy to make people who can afford college feel good that they're there on merit (instead of ability to pay, even if that's the actual primary criteria), as well as to give colleges excuses for jacking up tuition even more ("well it -says- $50,000/year, but act now and we'll give you it for $29,999!"...inflates people's egos that they're "so smart" that the college is giving them a $20,001 discount, increases perceived "value" of an institution, and also helps to obscure the real price of attendance so that the only way to find out if you can afford to attend is to gamble away $100+ in application fees before they'll tell you or not).
That does not match with my experience, especially considering that the majority of scholarships are need based. Especially at top universities, where merit scholarships are vanishingly rare (except at top LACs - liberal arts colleges).
Unfortunately, your experience does not match reality, wther it "jives" with it or not. Blithely ignoring the actual situation is, of course, almost guaranteed to ensure that poor but smart people will continue to be priced out of college educations in the US.
"Over all, the report found, published college tuition and fees increased 439 percent from 1982 to 2007 while median family income rose 147 percent. Student borrowing has more than doubled in the last decade, and students from lower-income families, on average, get smaller grants from the colleges they attend than students from more affluent families."
"While 79% of students born into the top income quartile in the U.S. obtain bachelor’s degrees, only 11% of students from bottom-quartile families graduate from four-year universities, according to Postsecondary Education Opportunity."
Anyway, if 1 trillion dollars in student loan debt doesn't sound like a problem to you, there's probably not much to be gained from continuing the discussion.
People like you have no idea what it's like to live below the poverty line.