{"id":1263,"kind":"story","title":"When AI-written tests all pass but are hard to trust, Stryker measures test strength by deliberately breaking the code","url":"https://qiita.com/Dingu-suke/items/b5318ef2040a9f59224a?utm_campaign=popular_items&utm_medium=feed&utm_source=popular_items","host":"qiita.com","by":"startupxo","score":0,"comments":1,"editorial_comments":1,"time":"2026-11-10T00:41:40.139Z","origin":"curated","language":{"source":"ko","reading":"en","translated":true},"summary":["Even if all tests pass, you cannot tell whether those tests actually catch mistakes. Mutation testing deliberately breaks the code in small ways and then checks whether the tests fail. If they fail, the mutant was caught; if they pass, it was missed. Stryker is a tool that automates this process.","The author's team applied it to a Prisma client (131 models) that automatically filters data per company. The definition table listing per-model conditions was 100% from the start, and the hand-written client tests became 100% after being supplemented at 89.74%.","Among the missed cases was a test where an attempt to delete another company's row failed because of a foreign key constraint, not the per-company filter. The bind call the AI added following a review comment turned out to be unnecessary code, since removing it left behavior unchanged.","In the author's environment, 407 mutants took 3 minutes 30 seconds, and when the scale grew to 1,326, it took 24 minutes. They had to specify only the real-DB tests so unrelated tests would not run, and the author is considering running it once before a PR."],"discussion":"https://startupxo.com/en/1263/","markdown":"https://startupxo.com/en/1263.md","api":"https://startupxo.com/api/v1/item.json?id=1263&lang=en","text":null,"edited":null,"sources":[{"site":"rss","name":"Qiita","url":"https://qiita.com/Dingu-suke/items/b5318ef2040a9f59224a?utm_campaign=popular_items&utm_medium=feed&utm_source=popular_items","score":null,"comments":null,"observed_at":"2026-11-10T00:40:32Z"}],"thread":[{"id":536,"by":"startupxo","score":0,"time":"2026-11-10T00:41:40.139Z","text":"O: In a multi-tenant SaaS, a backend developer who owns the per-company filter is encouraged to run Stryker before a PR, using --mutate on just the one file that must never be wrong.\nX: In the author's setup, 407 mutants took 3 min 30 sec, and when it grew to 1,326 mutants it took 24 min. Tests unrelated to the file also ran, so you should specify only the real-DB tests.","editorial":true,"replies":[]}]}