If you’re a Ruby or Rails developer looking for some advice on how to get better at integration testing: congratulations! You’ve reached the highest level of difficulty in all of the areas of the stack you must conquer to become a great Ruby developer.

Integration testing is hard, but it doesn’t have to be. This the subtle of this truth lies in the fact that you must be skilled in both the backend and front-end of your app: you must understand factories and your Ruby objects, and if you have a Javascript-heavy app, the deep fundamentals of Javascript as well.

First things first, you will want to learn how to debug in both the front and back-ends. For the sake of this post, I’m going to assume you have learned a backend debugging tool like byebug. If not, try this tutorial now.

Second, you need to know that Capybara is a syntax for writing Ruby – “a DSL” – for telling a browser what to do. It can work against several of different browsers – Firefox, Chrome, and ‘headless’ browsers you can’t see. If you use a browser you can see, you get the neat effect of being able to view your results as they run, which can be fun (and you should do it) but may not work on a Continuous Testing / Continuous Integration platform.

A headless browser (of which I will discuss two: webkit and poltergeist) is complex to debug, and requires a command of all the parts of the stack.

Occasionally, you may write some Javascript code that will work in one browser and not another (you should learn to avoid this) – that’s why you can run Capybara with a single syntax against many different browsers.

The bad news is, in short, despite it being 2016 and Rails having been around for nearly 12 years none of the drivers is perfect.

Sometime ago I wrote about a neat little trick to view console messages while debugging Capybara webkit.

Here’s a list of other notes of things to keep in mind.

1. You should be using Capybara version 2.7.1 or higher. Earlier versions do not wait for all sessions to close before kicking off Database cleaner’s truncation. When truncation happens before all sessions are closed, bad things happen (like intermittent failing tests). Waiting and timing is explained in detail below.
2. This applies to you if you app makes PATCH requests: Make sure you are on Phantom JS 2.0 or higher. Note this is a binary to install and on CI server it probably is a global (shell) configuration. (On ours, Semaphore, you need to specify the Phantom JS in the global build commands, not just in your Gemfile.) You to be running on Phantom JS 2.0 or higher to be able to process PATCH requests corruptly. When they aren’t, they come through on the server side as empty requests, which can lead to unexpected results.
3. Capybara-webkit sucks. It just does. Don’t use it. The intermittent issues alone are enough to throw it out. Use Poltergeist instead. It was an older technology and by and large it has been replaced by Poltergeist. Experienced developers know this and don’t use webkit for this reason. Junior developers fight in vein trying to get webkit to work and waste lots of time believing in something that simply is a shitty piece of technology.
4. When working with ChromeDriver note that it is annoyingly difficult to open the Developer Tools while the test is running. This is a knonw-issue, and the Chrome developers advise you pause your test to open Chrome Dev toos. This is explained here.
5. When using Database cleaner with Truncation, Make sure you have it in an append_after hook and not in config.after(:each) (several tutorials will mistakenly lead you down the wrong path here.) It should look like this:
   config.append_after(:each) do
    DatabaseCleaner.clean
   end
6. Prefer transaction instead of truncation for all non-Javascript tests (unit tests, controller tests, etc). For Javascript integration specs, you need truncation. An explanation about why can be found at https://github.com/DatabaseCleaner/database_cleaner#rspec-with-capybara-example
7. Use Factories and don’t use fixture data. Fixture data can lead to brittle tests. Generally the entire Rails community has learned from the Dark Days and recommends factories over fixtures.
8. Don’t use connection pooling. Some people on the internet will tell you to use connection pooling to solve thread-locking problems – don’t listen to them. Capybara already has dealt with this under the hood, make sure you are on a recent version of Capybara.
9. Avoid using .trigger. Sometimes if an element isn’t visible Capybara will advise you when it fails you can ‘work around’ the element not being on the page by referencing the element and calling .trigger. You’re just trying to get around the on-screen-and-visible enforcement by Capybara, but this isn’t a good idea. If the thing isn’t on the screen and visible, it probably means there’s a bug and you want to catch that as a failure. Remember your tests are only as valuable as what they catch when things mess up.
10. Circular Dependancy when trying to load ___
    This development issue that causes race condition (intermittent) failures has been explained on this Thoughtbot blog post.

    To fix if you’re on Rails 4.1 or prior, set allow_concurrency = false in test.rb (Rails 4.1 + earlier only)

    Set this in your config/environments/test.rb file set this:

    config.allow_concurrency = false

    You do not need this if you are on Rails 4.2 and above.

Timing

Timing is super hard to debug, but there’s an art to it. Tame your Capy specs like a lion tamer. Make them jump through hoops and bedazzel them to calm them down. You need to understand 3 things: Capybara’s native waiting, (2) a wait helper, and (3) an explicit sleep.

Capybara Native Waiting Behavior
If you’re on Capy 2.7 understand that Capybara natively waits for content on a page when you assert it to be there, even when Ajax and rendering might not have it ready to be there at the very moment the assertion runs. Thomas Wolpole, author of Capybara, advises me:

The way 2.7.1 is handling this is through middleware that keeps a counter of any current requests being processed by the app. First it tells the browser to visit about:blank and waits for that to happen, at which point the browser should not be initiating any more requests to the app. Then it waits for the active request counter to be 0, and then continues on.

Instead of using sleep, use expect(page).to have_content(…) to wait for the content you want to appear. Specifically I believe that using expect/have_content waits for the page to have the content you want it to have, but expect/value/to eq does not actually wait. For this reason, sprinkle in some expect(page).to have_content even when you don’t have to just to get Capybara to pause until the page is re-rendered.

You often find yourself writing

over and over again. Contrary to the instinct to not Repeat Yourself, that’s a good thing! If this really irks you may write yourself helpers to make this repeated step more encapsulated. What you are really doing is putting the UX through it paces, so think of it like a player piano instructions not like the code you so work on to make beautiful.

It will be easier to do this if your app has natural-language responses like “You have logged in successfully.” For this reason you should encourage your Product Owners/Stakeholders to put in such natural language indicators – it makes your site easier, safer, and your regression suite more solid. And your users will appreciate instant feedback it too. If your product managers insist on ‘silent’ feedback, remember you can use Capybara to assert that things are or aren’t disabled, grayed-out, etc.

Basically, although you can sometimes get away with expectations that do direct Ruby object lookup, you really shouldn’t, or should use it as infrequently as possible

Wait Helpers (Ruby metaprograms Javascript)

Sometimes you’ll see a spec failure that will pass if you add sleep 1 or sleep 2. Avoid this, but use a very fast sleep (I use 0.1) when necessary. Instead of sleeping, turn off animations and write wait helpers for yourself to pause until certain conditions are met.

You should use wait helpers to wait for:

– Ajax requests that Capybara doesn’t seem to pick up natively (Later versions of Capy are supposed to count the number of outstanding Ajax requests but I’ve had difficulty getting this to work consistently. You can and should assert content is on page and prefer Capybara’s native waiting to a wait helper)

– Your app is doing something like initializing (you can even write your app to set itself a global flag when initialization has finished which can be checked from Capy helpers)

Here’s an example of a wait helper that waits for an Ajax request. Note here we are using page.evaluate_script to metaprogram Javascript by way of Ruby code, waiting until a condition is met before continuing the spec.

Here’s an example of a wait helper that would wait for your app’s own initialization cycle, provided yourApp is the variable in Javascript where you app is namespaced, and when it is finished with its own initialization cycle it sets _initialized to true (on itself). You can write your own wait helpers appropriate to things you app does.

Explicit Sleeps

When all else fails sometimes you just need a sleep, which you just do in ruby as sleep X where [X] is the number of seconds you want to sleep.

You should use sleeps very rarely, but I’ve found they are needed in these cases:
– After an Ajax request, sometimes a sleep is needed to let the database catch up. (I try to keep these at about 0.5 seconds)
– A small timing delay (no longer than 0.1 seconds) for your app doing something like re-rendering

In theory you can wait for anything, so try to use Capybara’s internal waiting mechanisms first. In this order, your toolkit is:

1. Capybara’s internal waiting
2. A wait helper (as explained above)
3. An actual explicit sleep (try to keep all sleeps under 0.2 secs)

Remember each knife is shaper than the next, and so you should strive for minimal intrusiveness, but know that a combination of all three is likely necessary. The more astray you go from the art the more likely you experience timing delays.

Warning for anyone who has an expires_in set as a cache-control header in your controller endpoints (html or json).

Yes you! Go look in your code right now for expires_in set in your controllers and if you have any pay attention.

As I documented here, you’ve got to watch out if you have endpoints that have non-Zero cache-control headers on them. The headless driver (poltergeist or webkit) will hang onto the HTTP response between specs. This can be detrimental to you, if, say, the content of that endpoint’s response is what you are testing. In my case, I just used an inelegant hack to work around this- suggestions for improvements welcome.

Conclusion

Try to keep your feature specs to about 1-3 minutes to run per file, also maybe split them off when they are about 200-300 lines long. Be mindful of the total run time – since they are so valuable you can afford a little leeway here but keep in mind it slows down you time to develop new features.

Be careful about assertions that reach back into the database. Although you can get it to work, reloading objects and asserting things have changed is prone to race conditions, particularly with database cleaner. Remember that you have two threads operating separately, and even if you are able to do .reload on the object to get it into the right state, it’s actually nearly always better when writing Capybara specs to just assert the UX has changed the way you think it will.

And finally: Patience, discipline, know that others have been here before you and others will come here again. You are on the pinnacle of Rails development – don’t fall! Patience and faith.

Suggestion or feedback? Log in with your Stackoverflow, Github, Facebook or Google account to leave a comment.