Skip to main content

Many problems with one button



I admire the complexity which can emerge from elementary designs. Especially in the problem space.

One example speaks for itself, the 'On/off app':
- an webapp consisting only from one button which can either turn 'things' on or off (
'Picture 1' is an aproximate illustration of both visual states of the app)
- 'things' are marketing campaigns, some more complex processes to send particular marketing messages under particular conditions to particular customers (details of such don't matter in this context)



Picture 1
source: https://www.vecteezy.com/vector-art/88994-on-and-off-chrome-buttons




Even when my team was highly qualified in software development, under the fog of communication  and implicit expectations that we missed, we discovered the following problems with the implementation:
  •  Wrong template
    • The template which we used in the test environment was not meant for production, the impact was low - only a few business users saw the word 'test', but it taught us a lesson
  • Wrong business logic
    • This one was tricky, while the button should turn everything on or everything off, we didn't have the information about the production data - some marketing campaigns in production were off for long period of time. In other words the new requirement was "turn off everything what is on, but when I click you again, you turn on only those campaigns which you turned off"
    • This brought us to the realisation that we need the requirements explicit, so while writing them own, I discovered curious situations which can occur from such simple design, including pressing it from different sessions/users. As far I remember there were around 10 different scenarios which have been documented
  • Initial account sync
    • This was another blind spot, bound to the first launch of the app - the items which needed to be migrated to our DB. We found out there were several millions of them, which naturally caused our app to freeze. Althought this was fixed quite quickly, I will always remember to think about this aspect in future

Lesson learned

Don't be comforted with the straightforward simplicity of the system in front of you, the complexity can be much deeper than you realize. Like a famous clothing brand says: "Never stop exploring"

Comments

Popular posts from this blog

When to start automation?

If you are asking this as a tester, you probably asking too late.

Automation is something that can save you some portion of your work (understand resources for your client) and i rarely found cases of testing work that did not need at least some portion of automation.

I know that it is rarely understood that automation is something to be developed & maintained and if you cover enough of the application, you do not need any more regression - well i do not think that somebody has done an automation regression suite that if fully reliable (i am not speaking about maintaining this code - which is another topic). There can be always a bug (or quality issue) that slips through, even when you scripts go through the afflicted part.

I understand that many testers have no development background or skills, but i doubt the developers that could help you are far away. I am not assuming that they can do the scripts for you....
However if they understand what you need, they can say how easy is …

Thrown into automation

Situation & Problem I was thrown into an automation test project.

Concretely test automation of 3 different applications (different in purpose, look, structure, behavior) which regression testing was covered only by a automation test suite that was written in AutoIt and the size of the code was quite complex and huge for a new person in the automation.
Well, that was not the problem, it is a description of the situation.

The problems weren't initially visible. I was never automating before, so I needed to learn quite a bit of the code & got to know all the applications that were part of the project.

The problems were not so appealing at the start, but I would formulate then as: Maintenance of the scripts took too longBy new versions of the application, it took some time to adjust the scripts to the changesThis caused delay in information flow from testers to managers & developersThe changes in the application were not clearly communicated to testersTesting was purely t…

Mandelbug - bug, who didn't want to be found

Returning from holiday recently, I was expecting a calm day of catching up and doing some basic tasks. The opposite was true, this day I was introduced to a situation which puzzled us for two weeks.

Situation We have been reported that Android sometimes get the wrong reply to a particular GET requests. Ok, let us investigate, I got this, will be quick...

Reproducibility The bug is up till now non-deterministic to us. We were firstly not able to find the determining factor, it just occasionally occurred, persisted for some minutes (maybe up to half an hour) and then disappeared without a trace. This made the investigation and also any communication much harder. This happened for both iOS and Android apps.
We got ourselves here a Mandelbug:
A bug whose underlying causes are so complex and obscure as to make its behavior appear chaotic or even non-deterministic
First hypothesis We have decided to focus only on the android part. A debugging proxy was attached shortly for catching all tr…