SLTest Class Reference
| Inherits from | NSObject |
| Declared in | SLTest.h |
Overview
SLTest is the abstract superclass of Subliminal integration tests.
To write a test, developers create a new subclass of SLTest. They then add
test case methods and implement set-up and tear-down methods as necessary.
The methods in the SLTest (SLTestCase) category are used to set up before
and clean up after individual test case methods. Test case methods are methods,
defined on a subclass of SLTest:
- whose names have the prefix “test”,
- with
voidreturn types, and - which take no arguments.
When a test is run,
it discovers, sets up, runs, and tears down all its test cases.
The method descriptions below specify when each method will be called,
and [SLTestTests testCompleteTestRunSequence] gives an example.
A test case “passes” if it throws no exceptions in its set-up, tear-down, or the body of the test case itself; otherwise, it “fails”. That failure is “expected” if it was caused by a test assertion failing. Any other exception causes an “unexpected” failure.
Tasks
Retrieving Tests to Run
-
+ allTestsReturns all tests linked against the current target.
-
+ testNamed:Returns the
SLTestsubclass with the specified name.
Conditionalizing Test Runs
-
+ isAbstractReturns
YESif this class does not define test cases. -
+ supportsCurrentPlatformReturns YES if this test has at least one test case which can be run given the current device, screen, etc.
-
+ isFocusedReturns YES if the test has at least one test case which is focused and which can run on the current platform.
Ordering Test Runs
-
+ runGroupReturns a value identifying the group of tests to which the receiver belongs.
Running a Test
-
– runAndReportNumExecuted:failed:failedUnexpectedly:Runs all test cases defined on the receiver’s class, and reports statistics about their execution.
Running Test Cases
-
+ testCaseWithSelectorSupportsCurrentPlatform:Returns YES if this test case can be run given the current device, screen, etc.
-
– setUpTestCalled before any test cases are run.
-
– tearDownTestCalled after all test cases are run.
-
– setUpTestCaseWithSelector:Called before each test case is run.
-
– tearDownTestCaseWithSelector:Called after each test case is run.
-
– wait:Suspends test execution for the specified time interval.
-
– recordLastKnownFile:line:Records a filename and line number to attach to an exception thrown at that source line.
Class Methods
allTests
Returns all tests linked against the current target.
+ (NSSet *)allTestsReturn Value
All tests (SLTest subclasses) linked against the current target.
Discussion
The recommended way to run Subliminal tests is to invoke [SLTestController runTests:withCompletionBlock:]
with the set returned by this method. That way, new tests will automatically
be discovered and run.
Without modifying the argument to [SLTestController runTests:withCompletionBlock:],
tests may be conditionalized to run only in certain circumstances using APIs
like isAbstract, supportsCurrentPlatform, and isFocused.
Declared In
SLTest.hisAbstract
Returns YES if this class does not define test cases.
+ (BOOL)isAbstractReturn Value
YES if the class is without test cases, otherwise NO.
Discussion
An abstract test will not itself be run. Subclasses which do define test cases will be run, however, allowing a single base class to define set-up and tear-down work shared among related subclasses. Abstract classes can also be used to define the run group shared by subclasses.
Declared In
SLTest.hisFocused
Returns YES if the test has at least one test case which is focused and which can run on the current platform.
+ (BOOL)isFocusedReturn Value
YES if any test cases are focused and can be run on the current platform,
NO otherwise.
Discussion
When a test is run, if any of its test cases are focused, only those test cases will run. This may be useful when writing or debugging tests.
A test case is focused by prefixing its name with “focus_”, like so:
- (void)focus_testFoo;
It is also possible to implicitly focus all test cases by prefixing
their test’s name with “Focus_”. But if some test cases are explicitly focused
(as above), only those test cases will run—the narrowest focus applies.
If a test is focused, that focus will apply to any tests which descend from it.
Warning: Methods that take test case selectors as arguments (like
setUpTestCaseWithSelector:) are invoked with the unfocused form of the selectors
—they need not (and should not) be modified when a test case is focused.
Warning: Focused test cases will not be run if their test is not run (e.g. if it is not included in the set of tests to be run, or if it does not support the current platform).
Declared In
SLTest.hrunGroup
Returns a value identifying the group of tests to which the receiver belongs.
+ (NSUInteger)runGroupReturn Value
A value identifying the group of tests to which the receiver belongs.
The default implementation returns 1: all tests will be part of a single run group.
Discussion
SLTestController will run tests in ascending order of group, and then within
each group, in a randomized order. This allows test writers to provide a rough
order to tests, where necessary, while minimizing the test pollution that can
result from an absolute ordering.
A common use for run groups is to divide tests into two groups, those that
need to occur before some “startup” event (an onboarding flow, an import process, etc.)
(of run group 1) and those that need to occur afterward (of run group 2).
In this scenario, the “post-startup” tests subclass an abstract test
that, in its implementation of setUpTest, causes the startup event to happen.
Altogether, this ensures that all of the “pre-startup” tests run before
any of the “post-startup” tests run—and that startup happens before any of the
“post-startup” tests happen—while allowing the tests within each group to run
in any order.
Declared In
SLTest.hsupportsCurrentPlatform
Returns YES if this test has at least one test case which can be run given the current device, screen, etc.
+ (BOOL)supportsCurrentPlatformReturn Value
YES if this class has test cases that can currently run, NO otherwise.
Discussion
Subclasses of SLTest should override this method if some run-time condition
should determine whether or not all test cases should run.
Typical checks might include checking the user interface idiom (phone or pad)
of the current device, or checking the scale of the main screen.
As a convenience, test writers may specify the device type(s) on which a test can run by suffixing tests' names in the following fashion:
- A test whose name has the suffix “
_iPhone,” like “TestFoo_iPhone”, will be executed only when([[UIDevice currentDevice] userInterfaceIdiom] == UIUserInterfaceIdiomPhone)is true. - A test whose name has the suffix “
_iPad” will be executed only when the current device user interface idiom isUIUserInterfaceIdiomPad. - A test whose name has neither the “
_iPhone” nor the “_iPad” suffix will be executed on all devices regardless of the user interface idiom.
The default implementation of this method checks that the class is suffixed
appropriately and that there is at least one test case for which
testCaseWithSelectorSupportsCurrentPlatform: returns YES.
If this method returns NO, none of this test’s cases will run.
Declared In
SLTest.htestCaseWithSelectorSupportsCurrentPlatform:
Returns YES if this test case can be run given the current device, screen, etc.
+ (BOOL)testCaseWithSelectorSupportsCurrentPlatform:(SEL)testCaseSelectorParameters
- testCaseSelector
A selector identifying a test case.
Return Value
YES if the test case can be run, NO otherwise.
Discussion
Subclasses of SLTest should override this method if they need to do any run-time checks to determine whether or not specific test cases can run. Typical checks might include checking the user interface idiom (phone or pad) of the current device, or checking the scale of the main screen.
As a convenience, test writers may specify the device type(s) on which a test case can run by suffixing test cases' names in the following fashion:
- A test case whose name has the suffix “
_iPhone,” like “testFoo_iPhone”, will be executed only when([[UIDevice currentDevice] userInterfaceIdiom] == UIUserInterfaceIdiomPhone)is true. - A test case whose name has the suffix “
_iPad” will be executed only when the current device user interface idiom isUIUserInterfaceIdiomPad. - A test case whose name has neither the “
_iPhone” nor the “_iPad” suffix will be executed on all devices regardless of the user interface idiom.
The default implementation of this method checks that the selector is suffixed appropriately.
Warning: If the test does not support the current platform, that test’s cases will not be run regardless of this method’s return value.
See Also
Declared In
SLTest.htestNamed:
Returns the SLTest subclass with the specified name.
+ (Class)testNamed:(NSString *)nameParameters
- name
The name of the test (
SLTestsubclass) to return.
Return Value
The SLTest subclass with the specified name, or nil if no SLTest
subclass with that name is linked against the current target.
Discussion
This method may be used to retrieve a single SLTest, e.g. to pass to
[SLTestController runTests:withCompletionBlock:], without having to import
that test’s interface.
Note that it may be easier to run a single test by focusing that test
than by modifying the arguments to [SLTestController runTests:withCompletionBlock:].
Declared In
SLTest.hInstance Methods
recordLastKnownFile:line:
Records a filename and line number to attach to an exception thrown at that source line.
- (void)recordLastKnownFile:(const char *)filename line:(int)lineNumberParameters
- filename
A filename, i.e. the last component of the
__FILE__macro’s expansion.
- lineNumber
A line number, i.e. the
__LINE__macro’s expansion.
Discussion
Used by the UIAElement and test assertion macros so that exceptions thrown
by SLUIAElement methods and/or test assertions may be traced to their origins.
Declared In
SLTest.hrunAndReportNumExecuted:failed:failedUnexpectedly:
Runs all test cases defined on the receiver’s class, and reports statistics about their execution.
- (BOOL)runAndReportNumExecuted:(NSUInteger *)numCasesExecuted failed:(NSUInteger *)numCasesFailed failedUnexpectedly:(NSUInteger *)numCasesFailedUnexpectedlyParameters
- numCasesExecuted
If this is non-
NULL, on return, this will be set to the number of test cases that were executed—which will be the number of test cases defined by the receiver’s class.
- numCasesFailed
If this is non-
NULL, on return, this will be set to the number of test cases that failed (the number of test cases that threw exceptions).
- numCasesFailedUnexpectedly
If this is non-
NULL, on return, this will be set to the number of test cases that failed unexpectedly (those test cases that threw exceptions for other reasons than test assertion failures).
Return Value
YES if the test successfully finished (all test cases were executed, regardless of their individual
success or failure), NO otherwise (an exception occurred in test case set-up or tear-down ).
Discussion
See SLTest (SLTestCase) for a discussion of test case execution.
Warning: If an exception occurs in test case set-up, the test’s cases will be skipped.
Thus, the caller should use the values returned in numCasesExecuted, numCasesFailed,
and numCasesFailedUnexpectedly if and only if this method returns YES.
Declared In
SLTest.hsetUpTest
Called before any test cases are run.
- (void)setUpTestDiscussion
In this method, tests should establish any state shared by all test cases, including navigating to the part of the app being exercised by the test cases.
In this method, tests can (and should) use test assertions to ensure that set-up was successful.
Warning: If set-up fails, this test will be aborted and its cases skipped.
However, tearDownTest will still be executed.
Warning: Unlike the -setUp method found in OCUnit and other JUnit-inspired
frameworks, -setUpTest is called only once per test.
See Also
Declared In
SLTest.hsetUpTestCaseWithSelector:
Called before each test case is run.
- (void)setUpTestCaseWithSelector:(SEL)testCaseSelectorParameters
- testCaseSelector
The selector identifying the test case about to be run.
Discussion
In this method, tests should establish any state particular to the specified test case.
In this method, tests can (and should) use test assertions to ensure that set-up was successful.
Warning: If set-up fails, the test case will be logged as having failed, and the test case itself will be skipped. However, tearDownTestCaseWithSelector: will still be executed.
See Also
Declared In
SLTest.htearDownTest
Called after all test cases are run.
- (void)tearDownTestDiscussion
In this method, tests should clean up any state shared by all test cases, such as that which was established in setUpTest.
In this method, tests can (and should) use test assertions to ensure that tear-down was successful.
Warning: If tear-down fails, the test will be logged as having terminated abnormally rather than finished, but its test cases' logs will be preserved.
Warning: Unlike the -setUp method found in OCUnit and other JUnit-inspired
frameworks, setUpTest is called only once per test.
See Also
Declared In
SLTest.htearDownTestCaseWithSelector:
Called after each test case is run.
- (void)tearDownTestCaseWithSelector:(SEL)testCaseSelectorParameters
- testCaseSelector
The selector identifying the test case that was run.
Discussion
In this method, tests should clean up state particular to the specified test case, such as that which was established in setUpTestCaseWithSelector:.
In this method, tests can (and should) use test assertions to ensure that tear-down was successful.
Warning: If tear-down fails, this test case will be logged as having failed even if the test case itself succeeded. However, the test case’s logs will be preserved.
See Also
Declared In
SLTest.hwait:
Suspends test execution for the specified time interval.
- (void)wait:(NSTimeInterval)intervalParameters
- interval
The time interval for which to wait.
Discussion
Only use this method to wait (for the UI to update, or for the application to complete some operation) if a delay is found to be necessary, and it is not possible to describe a specific condition on which to wait.
It should not be necessary to wait before attempting to access interface elements when the delay would be less than the default timeout: elements automatically wait to become valid and/or tappable if access requires waiting.
Where the delay would be more than the default timeout, or where the condition
on which to wait involves application state not made apparent by the UI,
using the SLAssertTrueWithTimeout macro will result in a clearer, more
efficient test than using -wait:. See the definition of SLAssertTrueWithTimeout
for examples.
Declared In
SLTest.h