A lot of it comes down to sheer cost, IMHO. I was somewhat lucky when I built my RV-9 in that the $$ was much higher than it is now. I have probably $130,000 in my RV, but to built 'like for like' today would probably cost you close to $170,000 and that is an awful lot of $$ for someone who is trying to pay off a mortgage, or even simply save a deposit. Even over a 10-year timeframe, that's $17,000 every year, or a mortgage payment for a smallish house. Then there's the ongoing costs. I'm up for $6,000 a year in fixed costs before I spin the prop. Add in $200/hour just in landing fees for circuit training at Warnervale or some other airports, why on earth would you pursue something so expensive for so little reward? Those that have newish RV's, or those composite wonders aren't going to put them up for sale unless they have to, because they'll lose far too much replacing them.
As much as I love the flying, CAsA (with their million-and-one rules and tacking on "this is an offence of strict liability" onto every piece of legislation), RAAus (the Privacy Policy disclosure debacle and the direction the organisation is going) DOTRS (ASIC, need I say more) and the various airport owners (who mandate an ASIC, 24/48H PPR, have 12'high razor wire, guard dogs, and a 10-person armed response unit to guard against a wayward RAAus pilot or weekend warrior dropping in just to have a look-see, or refuel, all backed up by an ARO with an attitude*) have taken an awful lot of the enjoyment out of it. If I could have my own property and self-insure a cheaper 4-seater, then perhaps I'd reconsider, but there will shortly be one more plane advertised on the classifieds....mine...
*May be a slight exaggeration