Even on the Apple side there doesn't seem to be that much demand, as demonstrated by the fact that the SEs are just iterations on old iPhone tooling (currently the iPhone 8) rather than brand new compact designs, and the size of the SE has crept upwards as they moved it to newer tooling. I think the motive behind the SE has more to do with cost optimization than size, it just happens that the cheapest way to make an iPhone is to refresh one of the older iPhones, which incidentally were smaller, as was the style at the time.