My issue with UI tests
So here's the thing: I have a problem with UI tests. (Kind of.) I’ve tried them, but did not particularly like them. Actually, I really don’t like that I don’t like them, since I think the problem they are aiming to solve is extremely important. Testing how your system acts as a whole, rather than just testing its parts in isolation, is a brilliant idea. The problem I have is that I find UI tests in their current shape to be badly cut out for solving such an important problem.
My issue with UI tests basically boils down to two points:
1) They’re slow.
UI tests are slow, since they consist of actual interactions with your UI. Even though automated tapping of buttons can be pretty darned quick, you still need to wait for all those pretty view controller transitions, animations and (possibly) slow network requests. Also, since UI tests run in an actual instance of your app, you can’t just fire up any view controller and test it in isolation. This means that if you have a view controller that’s eleven button taps away from your user, it’s going to be eleven button taps (and who knows what else) away at the beginning of your test session.
2) They’re black-box tests.
When we use UI tests as opposed to unit tests, we give up on all the control we have over our system. We’re constrained by dependencies such as the backend and local databases, which we can mock to our heart’s content while we’re doing unit tests. One thing that’s often highlighted about UI tests is the fact that they’re testing a non-altered app session, free from possibly incorrect developer assumptions and test code bugs. This is definitively something worth highlighting, but I think it’s also worth highlighting that since they have all these dependencies that can’t be controlled, the app session they test often tends to become very restricted, and not very representative of what the end user would experience anyway.
Something in between
What I would like us to try out in this post is to come as close as possible to UI tests while still being able to tweak our code just as when we’re doing unit tests. To achieve this, we’re going to use logic-free view controllers together with view models set up with data bindings. (What I mean by logic-free is that the only thing our view controller is going to care about is sending its current state to the view model, receiving a new state from said view model, and rendering itself accordingly.)
For simplicity's sake, we’re not going to use RxSwift or any other 3rd party library for our bindings, but rather just the tools that are provided by Swift out of the box. However, there shouldn’t be any problem applying the same techniques using Rx, Bond or any other great framework out there.
The app we’re going to build to try these techniques is a very simple one: a login page with two fields (email and password), a login button with two different states (enabled/disabled), a loading indicator and a small label which will tell you what, if anything, went wrong when trying to log in.
Inputs & outputs
Starting off by setting up our view model, I’m going to straight off borrow a concept from the awesome, open-sourced Kickstarter app, where view models are defined as containing inputs and outputs, represented by two different protocols. In this case, we’re going to call them LoginViewModelInputs and LoginViewModelOutputs. Inputs is going to hold functions that will be triggered by events: in this example, we’re going to deal only with events that are fired whenever the user interacts with the app (text input, tapping buttons etc.), but it could’ve also been system events such as the view being loaded or the app going into background mode. Outputs will contain methods that the view controller listens to in order to re-render itself when the view model changes its state.
Next, we’re setting up our view controller. We’ll define a setupBindings method where we pass all our user input along to the view model, while also listening to its output to update our UI.
Then we’ll define some very simple business logic in our view model. User input is being constantly evaluated and considered valid enough for sending to our backend when the email field contains an @ and the password is four or more characters long. The observing view controller is continuously updated about what state the view model is in, and will toggle the enabled state of its login button as the user’s input changes. The login button’s tap event triggers a login request, and the view model notifies the view controller as soon as its loading state changes.
Finally, before we start writing tests, we’ll add a small extension to UITextField which we’ll use to update our fields programmatically.
Testing, testing…
So, we’ve set up our controller and model, and now it’s time to take it for a spin. We’ll start by initialising our view controller and running its viewDidLoad method, where we ran the setupBindings method. By running viewDidLoad rather than setupBindings directly, we're actually testing that this method will be triggered by the system when running the app.
Alright, so we have our VC set up, and now we want to start by testing its default state. Since no input has yet been given, we want to verify that the info label is not showing and the login button is disabled.
Ok, our assertions passed. Next up, we’re going to enter a valid email and password and verify that this enables the login button. Note how we’re testing both our email and password validation logic as well as our UI’s response, all in a few lines.
Just to be sure, let’s verify that our loginButton is again disabled after changing the email address to something invalid (like not containing an @).
So far so good, but we haven’t really done anything yet that we could not do with UI tests. In a previous article, I looked at how you can easily mock methods rather than entire objects, and we’re going to use that same approach here to simulate the user trying to log in with invalid credentials. Just like when we’re doing “regular” unit testing, we can have complete control over our dependencies, which means we can easily mock server responses to generate all kinds of different UI states.
Now, when “tapping” our button, we should immediately get an error, showing our infoLabel with the expected error message:
While we’re at it, we might want to test our app’s behavior when/if our given credentials were valid instead.
Our tests passed! Although this is a rather trivial example, I think it’s not hard to see how easily this could be extended. If our app had a profile page, for example, we should be able to mock a logged in user and verify that all the correct information was displayed. If we then wanted to change any field on our user, we should easily be able to do so by mocking some backend response and then verifying again that our UI is displaying the new information correctly. Should we want to test the UI correctly handling this same request failing, we should also be able to do so with very little extra ceremony.
Conclusion
We’ve seen how we could simulate user interaction and make this run as quickly as any suite of unit tests. We were able to do inline mocking of our backend, easily testing how our UI reacts to various weird states. One could say we’ve taken some powerful traits of unit tests (controlling dependencies, speed) and combined them with an equally powerful trait of UI tests (testing using interaction).
Testing business logic through the UI can be quite powerful. For example, we might write unit tests for an email validation method, but we might still forget to actually use that method in our app when when the user types in their value. By simulating input into a text field and observing the UI’s reaction to it, we test both the validation logic and that it’s actually used for text input. Of course, testing business logic only by observing the UI would result in a lot of boilerplate, so when we need some really thorough testing of our business logic, unit tests are probably the way to go.
One problem with this kind of testing is that we’re not only mocking our dependencies, but we’re also kind of mocking the actual app lifecycle. In our code example above, we manually called the viewDidLoad method of our view controller, which would’ve been called by the system in a regular app session. But we might just as well have forgotten to call this or any other system-triggered events, which could render our tests useless at best, and hide bugs at worst. Similar problems could also arise if we were to write tests that simulated user interaction that would not be possible in the actual app.
So, the usual pros and cons. If you have thoughts on this and perhaps even experience of doing something similar in a project, I would really like to hear about it on Twitter.
Thank you so much for reading!
Anders Mannberg is an iOS developer at Knowit in Malmö. When he's not coding (or writing posts like these), he's fiddling around with music or having fun with his kids.
This post was originally published on Anders' blog: https://mannberg.github.io.