That thread led me to a project/product idea where you take an existing Postgres instance used for normal products or whatever, replicate it to various read only clusters with a custom search extension loaded and some orchestrator sitting on top (I’ve written most of one in rust that uses 0mq to communicate with it’s nodes) and create drop in search from existing databases with a nice guided web gui for automatic tuning suitable for most business use cases.
It fell off when my friend who wanted to help work on it went off to Google and you know working on a search engine for them became a bit of a no no. I still think it's a great idea with a lot of value, I should circle back.
> That thread led me to a project/product idea where you take an existing Postgres instance used for normal products or whatever, replicate it to various read only clusters with a custom search extension loaded and some orchestrator sitting on top (I’ve written most of one in rust that uses 0mq to communicate with it’s nodes) and create drop in search from existing databases with a nice guided web gui for automatic tuning suitable for most business use cases.
Very interesting idea -- just want to add one thing, write it in rust (with pgx?[0]) :)
With some tuning and memory page size adjustments in Postgres you get very compelling speeds.
I’ve actually previously written a full twitter thread on a somewhat inactive account about why we don't use existing dbs for search more.
https://twitter.com/kinglycrow/status/1533270619353231360?s=...
That thread led me to a project/product idea where you take an existing Postgres instance used for normal products or whatever, replicate it to various read only clusters with a custom search extension loaded and some orchestrator sitting on top (I’ve written most of one in rust that uses 0mq to communicate with it’s nodes) and create drop in search from existing databases with a nice guided web gui for automatic tuning suitable for most business use cases.
It fell off when my friend who wanted to help work on it went off to Google and you know working on a search engine for them became a bit of a no no. I still think it's a great idea with a lot of value, I should circle back.
I also write (at length) about how ease of use will win search here: https://twitter.com/KinglyCrow/status/1532402654218964993