One thing they don’t tell you at school is that your customers will have less memory, less diskspace and fewer cores than your machines. Their laptops will have 37 chrometabs open. They will run several other software alongside yours. Excel, Chrome, their custom accounting app when using the software you are testing.
Clean machine testing is all nice and good. But dirty machine testing is here all the weird things come out. So don’t be afraid to crank it up when it come to front-end client. They need to be able to operate in resource constrained environments.
Another thing they never tell you is that a DB with 134 users in it is no DB. A DB begins to act like a DB with half a million records in it. Anything smaller and you are testing wishful thinking, but not realistic performance and scenarios. So if you want to measure what your DB can take, put in a million users first, and then start cranking the levers.
The third thing they don’t tell you is that you should run your smoketests against a system that has active traffic on it. Get your toolsmiths to build a little automated traffic generator for you and start it up when you are doing manual tests. Keep an eye out for things that feel different. It’s like when your car begins to feel a little soft and woolly and you know you have a blown tyre. You may also run into unexpected race conditions they would never happen during testing but would always come out in production.
