Contents

  1. Phase 2
  2. Phase 1
    1. Running tests
    2. Suggested order of migration
    3. Current test suite assignments and wish-list
    4. General notes on test support for Amara 2.x
    5. General notes on migration process
    6. Notes for specific assignments
    7. More notes
    8. Outdated assignments

Phase 2

Andrew Dalke (with contribution by Andrew Kuchling) converted the test suite to nose, and made some improvements.

See: http://groups.google.com/group/akara-dev/browse_thread/thread/5519a8ecf38e6fc

You can still run the test suite as follows: 

cd $AMARASRC/test
sh testall.sh

Phase 1

This effort was pretty much completed by early 2009, thanks to help from the community. These notes are obsolete, but preserved for the record.

We'll be migrating the test suites from the old Ft.Lib.TestSuite infrastructure to unittest, and porting tests from Amara 1.x tests to match the new APIs.

We'll need plenty of help. If you're up to help, please notify the core developers. Start by joining the Akara dev mailing list and reviewing Amara/Developer_notes

Contents

  1. Phase 2
  2. Phase 1
    1. Running tests
    2. Suggested order of migration
    3. Current test suite assignments and wish-list
    4. General notes on test support for Amara 2.x
    5. General notes on migration process
    6. Notes for specific assignments
    7. More notes
    8. Outdated assignments

Running tests

On UNIXy systems, you should be able to just tun test/testall.sh .

Suggested order of migration

This is a broad view, but most of the action will be in specific, present assignments, tracked in the next section.

Current test suite assignments and wish-list

Any of the above selections can be done in a few hours' work.

General notes on test support for Amara 2.x

We will need to provide a test support module because unittest's loader is pretty dumb out of the box. It's probably enough to just pull in Pythons ( http://svn.python.org/view/python/branches/release25-maint/Lib/test/test_support.py?rev=60337&view=markup ). Until we sort that out top-level test suites will not automatically run, but please don't avoid them for that reason.

Update: we now have amara.test, which handles this. Just use: 

from amara.test import test_main, test_case

#Test code here...
class a_test_suite(unittest.TestSuite):
    class a_test_case(test_case):
        def test_method(self):
            self.assert_(True)

if __name__ == '__main__':
    test_main()

the entire module is wrapped in a TestSuite with the declared TestSuite subclasses added in alphabetical order followed by any amara.test.test_case (unittest.TestCase subclasses) added after that also sorted likewise.

Note: The amara.test.test_case subclasses in the TestSuite subclass are added in alphabetical order; their corresponding suite test methods are already added sorted by unittest, so we just extended that idea for amara.test.test_case

General notes on migration process

For Amara the main task is to update imports. For 4Suite we also need to port from the old Ft.Lib.Test machinery to unittest.

Each inner startGroup...groupDone will become a new class that subclasses amara.test.test_case, which in turn subclasses unittest.TestCase.

Each outer level startGroup...groupDone becomes either an instance of unittest.TestSuite or a new test module file, depending on your judgment. Additional levels of groups become nested TestSuite or further broken-down files. You can add either a amara.test.test_case or a nested TestSuite to a TestSuite.

Everything from tester.startTest to tester.testDone will become a test function (a method that starts with test).

The description string passed into startTest will become the first line for a docstring of the new test function.

tester.compare will map to one of the assert... functions of unittest, based on your judgment.

An example: 

    tester.startGroup('file:/// and file://localhost/ equivalence')
    tester.startTest('equivalent key in UriDict')
    uris = Uri.UriDict()
    uris['file:///path/to/resource'] = 0
    tester.compare(True, 'file://localhost/path/to/resource' in uris, 'RFC 1738 localhost support failed')
    tester.testDone()
    tester.startTest('value of 2 equivalent keys')
    uris = Uri.UriDict()
    uris['file:///path/to/resource'] = 1
    uris['file://localhost/path/to/resource'] = 2
    tester.compare(2, uris['file:///path/to/resource'], 'RFC 1738 localhost support failed')
    tester.testDone()
    tester.groupDone()

would become: 

class test_file_uri_localhost_equiv:
    '''file:/// and file://localhost/ equivalence'''
    def test_uri_dict(self):
        '''equivalent key in UriDict'''
        uris = Uri.UriDict()
        uris['file:///path/to/resource'] = 0
        self.assert_('file://localhost/path/to/resource' in uris, 'RFC 1738 localhost support failed')
        return

    def test_equiv_keys(self):
        '''value of 2 equivalent keys'''
        uris = Uri.UriDict()
        uris['file:///path/to/resource'] = 1
        uris['file://localhost/path/to/resource'] = 2
        assertEqual(2, uris['file:///path/to/resource'], 'RFC 1738 localhost support failed')

This is part of the port from

http://cvs.4suite.org/viewcvs/4Suite/test/Lib/test_uri.py?rev=1.34&view=markup

to

http://hg.4suite.org/amara/trunk/file/tip/test/lib/test_iri.py

In the old test suite we often used parameter-driven tests with the like of an (input, expected) tuple over which we iterated. Here is a suggestion for how to port those

Before: 

    tester.startGroup("PercentEncode and PercentDecode")
    for unencoded, encoded in percentEncodeTests:
        if len(unencoded) > 10:
            test_title = unencoded[:11] + '...'
        else:
            test_title = unencoded
        tester.startTest(repr(test_title))
        tester.compare(encoded, Uri.PercentEncode(unencoded))
        tester.compare(unencoded, Uri.PercentDecode(encoded))
        tester.testDone()
...
    tester.groupDone()

After: 

from amara.test import test_main, test_case

class Test_percent_encode_decode(test_case):
    '''PercentEncode and PercentDecode'''
    #def test_percent_encode(self):
    @classmethod
    def create_test_percent_encodes(cls):
        '''Percent encode'''
        for count, (unencoded, encoded) in enumerate(percent_encode_tests):
            #print "Creating test", "test_percent_encode_%i"%count
            def test_percent_encode_template(self, count=count, unencoded=unencoded, encoded=encoded):
                if len(unencoded) > 10:
                    test_title = unencoded[:11] + '...'
                else:
                    test_title = unencoded
                self.assertEqual(encoded, iri.percent_encode(unencoded))
                self.assertEqual(unencoded, iri.percent_decode(encoded))
            setattr(cls, "test_percent_encode_%i"%count, test_percent_encode_template)
        return

Test_percent_encode_decode.create_test_percent_encodes()

Another approach to migrating batch tests

The new test

http://hg.4suite.org/amara/trunk/file/tip/test/xpath/test_expressions.py

Is basically a roll-up of

You can run it, and it delegates to each of the other modules in turn (these can still be run individually). It's also an example of aggregating multiple modules into a single TestSuite.

Basically this approach uses metaclasses instead of @classmethod. As Jeremy puts it: "lots of fun with metaclasses!"

treecompare

Amara comes with a markup-savvy comparison function, smart enough to ignore order of attributes and such. For a comprehensive example of usage, see the _assert_result function in amara/test/xslt/init.py . The basic usage, for XML comparison, is: 

            diff = treecompare.xml_diff(out, ATOMENTRY1)
            diff = '\n'.join(diff)
            self.assertFalse(diff, msg=(None, diff))

Notes for specific assignments

amara2/test/xslt (especially "Borrowed")

The old Xml/Xslt/Borrowed tests will be ported to a new source/transform/expected trio of files.

Here is an example of an already ported test case (Xml/Xslt/Borrowed/af_2000922.py): 

    from Xml.Xslt import test_harness
    source_1 = "..."
    sheet_1 = "..."
    expected_1 = "..."
    def Test(tester):
        source = test_harness.FileInfo(string=source_1)
        sheet = test_harness.FileInfo(string=sheet_1)
        test_harness.XsltTest(tester, source, [sheet], expected_1,
                          title='output bug')
        return

would become three files in the amara/test/xslt/borrowed directory:

For those tests that have multiple tests in one Python source file, simply append a _nn to the base name, where nn matches the number suffix of the string inputs.

If an XsltTest() tests for an exception or uses top-level parameters, skip it, but indicate those tests here as they will need more attention.

More notes

For now we'll replace the test files and reorg as makes sense. We may ned some lightweight harness machinery for 4Suite, e.g. a test_support module that each test module will need to import (this will also help abstract the platform differences). In particular we might need to come up with our own TestLoader to give us flexibility for combination of TestSuites and amara.test.test_case into each file. We might also need setUp() and tearDown() methods, which are called before each test function (amara.test.test_case.testXXX).

http://oakwinter.com/code/category/python/unittest/

Use decorators to e.g. mark knownFail tests. Keep all extensions simple, in one file that's easy to find and easy to follow (i.e. well-documented. If there must be magic, given unittest's limitations, keep it all in one place, and well explained.

The "core" and "borrowed" tests should be merged into one directory, with the "borrowed test" terminology changing, perhaps to "field test" or "user test" (i.e. a test based on an observation by a user in the field). This will tend to explode the number of test files within the directory, so good naming will be key.

Outdated assignments

This section is quite out of date. Present assignments are above. If you completed some work in this section, but didn't turn it in, please e-mail this info to the akara-dev list

Uche will work on test/Lib and Uche and Jeremy on specializations to the test harness

amara2/test/xmlcore

Ported from 4Suite/test/Xml/Core/:

Porting from 4Suite/test/Xml/Borrowed/:

If anyone finishes the above and is ready to move on, mention to the list and move on to port 4Suite/test/Xml/Borrowed/

amara2/test/bindery

Soon to be assigned. Mostly to Luis MM and Joel B

amara2/test/xpath

Jeremy is working on ports from 4Suite/test/Xml/XPath/Core/:

Need to assign tests from 4Suite/test/Xml/XPath/Borrowed/:

Amara/Developer_notes/Test_migration (last edited 2011-07-25 15:06:16 by LuisMiguel)