For my first iOS app (I'll provide link when it's published on App Store) I wanted to make some decent backend. I chose Google App Engine for several like scalability, very good latency worldwide, hosting included, etc.
Since I wanted to do everything right from the very beginning I wanted to properly unit test every feature.
This made me thinking because I wanted to run all unit tests in several different environments:
Locally in terminal.
It's very easy to run just a single test case and it doesn't have to be dependent on GAE modules.
Locally on GAE test server.
This should give the same results like running tests in terminal but this time in GAE sandbox. I think it's quiet unlikely that some tests would pass in terminal but fail on GAE server. This gives me more confidence that my code is not dependent on some local modules that are not provided by GAE.
Testing GAE apps in sandbox is very important because some methods (like
os.chdirfor example) are disabled and throw an exception.
On real GAE production server hosted on Google infrastructure.
After this I'm sure that my local modules work as well as those provided by GAE. I think if you're locally using up-to-date GAE server all test should pass without any problem.
For me was very important to realize that although my app uses some Python modules like lxml that is provided also by GAE environment I still have to install it locally because it's not part of GAE test server. Read more about standard third-party libraries on GAE.
Particularly, you can specify which version of lxml you want to use but you never know. I might upgrade my local lxml unintentionally and it might break some functionality. That's why I want to be able to run tests also in production GAE server hosted on Google.
Running unit tests in terminal is simple and it's well described in many other tutorials all over the Internet. Running tests for an app that uses some GAE modules isn't that simple because you have to load some GAE modules and I think it's easier to leave this stuff on somebody else.
I chose nosetests with GAE extension because it's easy to use in terminal and you can run all tests (using auto discovery) or just particular test that you want. You can install both with
Finally this is how I use it:
nosetests -s -v --with-gae --without-sandbox
nosetests --help to see what each parameter does but the most important is
The first one loads GAE modules so you can import them in your app (for example
The second one disables GAE sandbox which is necessary only if you want to import some third-party libraries including those provided by GAE! For example my app uses lxml, thats why I have to run tests with
--without-sandbox. Of course, you don't have to use this option if you have no dependencies.
Running unit tests on GAE server
In order to run unit tests on local/production GAE server I had to make some URL handler that actually runs all tests. At first I thought I can do this easily with
nosetests which has methods for running tests programmatically but after a couple of hours I gave it up because it uses system methods like
chdir which is disabled on GAE server and I think I wouldn't be able to bypass it with our modifying
nosetests source code.
So, at the end I sticked to normal
unittest module with my own very simple autodiscovery method.
Just a note here, all my tests inherit
unittest.TestCase, each files contains just one test case and both files and class names begin with the word
Test (for example in
TestUtils.py I have just one test case
By the way, I'm using webapp2 framework but I guess it would be easy for you to modify it for another framework
Well, last thing is to assign it to a URL:
routes = [ ('/unit-tests', RunUnitTests.RunUnitTests) ] app = webapp2.WSGIApplication(routes, debug=True)
This prints results from each test case with total number of tests.
Well that's it. I hope there's some easier and maybe more straight forward method but for now this is all I have and I think all three steps are necessary to automate the testing process.
If you're using some different approach let me know in the comments bellow. I would love the hear it.