Technically you only need refcounting of objects with a destructor. For everything else, destruction is a no-op and you can just let the GC re-use the object's memory. And if the destructor is used pretty much just for side effects (closing files & sockets etc.), these will typically be rare & pose little overhead.
Would suck in a dynamically-typed language, though, because you have no idea whether a reference points to an object with or without a destructor, and so always have to pay the overhead of checking and possibly decrementing the refcount. I don't really see this working without a static type system.
BTW, you're often better off with explicitly-scoped destruction (eg. the Ruby & Python way) than refcounting, for the same reason that boost::shared_ptr can be tricky and Java programs can leak memory. If you don't have clear object ownership & lifetimes, it's very easy to stash a reference to an object in some longer-lived object (a cache, for example) and accidentally hold onto it way longer than anticipated.
> Would suck in a dynamically-typed language, though, because you have no idea whether a reference points to an object with or without a destructor, and so always have to pay the overhead of checking and possibly decrementing the refcount. I don't really see this working without a static type system.
It's worse than that. Any object that contains references to other destructible objects (including transitive references), must have an implicit destructor to decrement the refcounts of the destructible objects it points to. In fact, any object that holds a reference, directly or indirectly, to a generic "Object" class or equivalent must have an implicit destructor. This makes a huge part of the object graph implicitly destructible.
This is exactly what Python does, and while Python isn't the fastest language around, it's not that slow either, when compared to other dynamically typed, interpreted languages.
OTOH, refcounting is one of the main reasons why it's virtually impossible to eliminate the GIL, which is a showstopper for many multi-core usages of Python.
