Raf's answer about Perplexity Pro for the competitor column is the part I'd poke at, not the scraping. Store data is a fact you can re-fetch; "who competes now" is a model's answer on the day it was asked, and that's the exact field someone bets a month of rebuilding on. We run an aggregated job board, around 6,700 open listings re-pulled from dozens of sources every six hours, and the columns we generate rather than fetch are the ones that quietly go wrong, because nothing errors when they age, they just stop being true. A dead-by-Manifest-V3 row also reads as a very different bet from a dead-by-no-demand row, since only the first leaves the userbase genuinely orphaned. Does the competitor field get re-run on a row before someone buys, or is it the single snapshot from when the 9,656 were collected?
Thanks for the reply, even though it's a plug to your own job board. Store data cannot be refetched after extensions are delisted. Columns that require up-to-date information (like the "competitor snapshot" and "unique selling point") are updated quarterly. So these updates are free forever (not a subscription, just a one-time payment).
Then about the dead-by-Manifest-V3: the database only contains extensions that were profitable, including data on userbase, review count, review score, etc.
Hope this answers your questions, and good luck with the job board!
@R4FKEN Fair point, that mention wasn't needed. The delisting argument makes sense, a snapshot nobody can recreate is the real asset. Does each row show when its competitor column was last refreshed, so a buyer can tell two weeks from almost three months?
