In my experience complex end-to-end tests cast a wide net that often results in finding a lot of issues and they provide enormous value. Their main negative is maintenance around robustness as the article discusses and hardening tests can take a lot of investment. That said, the alternative is worse (not having them) so I find your approach, and the author’s is what I’ve often done. I think there needs to be understanding (across the team and management) that automated tests are software and it will require a similar dev effort to maintaining any other software, especially so because there usually aren’t tests to test the tests!
I’m founder at Tesults (https://www.tesults.com) where we have a flaky test indicator that makes identifying these tests easier. It’s free to try and if you can’t get budget for a proper plan send me an email and I’ll do what I can.
In general the only way to never have flaky tests is to have simpler tests but I find those often don’t provide as much value - that’s just my personal belief after having spent years focused on automated tests, e2e tests do have robustness issues but the bugs they find make them totally worth it. Out of the issues mentioned in the article that affected my tests the most, it’s timing. They can be overcome though, I’ve run test suites with a couple of thousand e2e tests (browser) that have been highly robust and reliable after time was devoted to hardening them. You do have to focus on that and refuse to add new test cases until the existing ones are sorted out in some cases.
I’m founder at Tesults (https://www.tesults.com) where we have a flaky test indicator that makes identifying these tests easier. It’s free to try and if you can’t get budget for a proper plan send me an email and I’ll do what I can.
In general the only way to never have flaky tests is to have simpler tests but I find those often don’t provide as much value - that’s just my personal belief after having spent years focused on automated tests, e2e tests do have robustness issues but the bugs they find make them totally worth it. Out of the issues mentioned in the article that affected my tests the most, it’s timing. They can be overcome though, I’ve run test suites with a couple of thousand e2e tests (browser) that have been highly robust and reliable after time was devoted to hardening them. You do have to focus on that and refuse to add new test cases until the existing ones are sorted out in some cases.