Source code for EntitySpaces unit tests to be released.

We are sure you will be pleased over the next few weeks as we lead up to the official release of EntitySpaces 1.0. We have decided to release the source code for our unit tests. Registered members will be able download the entire EntitySpaces test suite for free, including the C# source code for the test fixtures and the code generated for the test database from the EntitySpaces templates, a Microsoft Visual Studio solution, SQL scripts for creating and populating SQLExpress and Oracle test databases, an already populated Microsoft Access 2003 test database, and the NUnit project and config files.

In anticipation of the pending release, we thought we would blog a bit about our testing philosophy, the tools we are using, and what it all means to you.

Testing Philosophy:

While we do not ascribe to any single testing methodology, we are big believers in testing. We are looking for a number of benefits from our tests:

• Comprehensive tests reassure us and our users that the code meets specifications. Nothing guarantees bug free code, and we certainly do not make that claim. But, a full suite of tests with a high percentage of coverage means fewer problems for you and for us.
• Automated tests mean we can run them often and ensure that a new feature or a fixed issue does not break working code.
• When an issue is discovered, a test is added that duplicates the problem and then the problem is fixed. Thus, expanding coverage into unanticipated areas.
• Tests provide additional documentation of EntitySpaces functionality.

It is for this last reason that we decided to release the source code. Not only does it give you extra reference material, but once every function has a corresponding test, it will be a great resource of working sample code. Opening up the test source code also exposes some of our development process to peer review. We look forward to the feedback.

For the purists out there, we should point out that much of what we are doing is not, in the strictest sense, “unit testing.” When we refer to unit tests we mean pretty much anything that can be automated with NUnit. That includes unit, functional, acceptance, and integration testing. The main point of differentiation from the standard definition of a unit test as found in “Test-Driven Development Quick Reference” is that we are not testing in isolation. We are testing all the way through to the backend databases that we support.

A pure unit test assumes that no database or network connection exists. In the case of a database, mock objects are coded that simulate the connection. A good part of the benefit of our providers is that they shield the developer from some of the idiosyncrasies of their database. In this case, a set of functional or acceptance tests are needed that do test via a database connection. Therefore, our tests are against actual databases using specific .NET drivers. If we had both a unit test against a mock object and a separate functional test hitting the database, then the unit test seemed redundant. We felt the unit test would just be testing our mock object code.

Testing Tools:

NUnit 2.2.6:

In order to have a full set of tests that can be run against each supported database, we are taking advantage of two things:

• EntitySpaces ability to dynamically change connections as long as the databases have matching schemas.
• NUnit’s ability to match a specific NUnit config file to a corresponding NUnit project file.

Compiling the tests is no more difficult than compiling the SqlDemo. Make sure that app.config has the right connectionString for each database. All the NUnit project files are identical. The different names associate them with different NUnit config files. NUnit config files are identical in format to EntitySpaces app.config files. Consequently, you will see two NUnit config files for each database, one for each sqlAccessType; DynamicSQL and StoredProcedure. You need to open each config file and make sure the connectionString matches the one in app.config for that database. Double-click one of the .nunit files to run the tests in the NUnit GUI. Once the GUI is open, you can run other tests by opening the other project files.

The Base folder contains all the test fixtures and the generated abstract and concrete classes for SQLExpress. The only template that needed to be run for Access and Oracle was the EntitySpaces Concrete MetaDataMap template. It could have just as successfully been done the other way around – a full set for Access and the MetaDataMap for SQLExpress and Oracle. Just note that esDefault in app.config should then be for Access, not SQLExpress. You will find the generated source files for Access in the MSAccess\BusinessObjects folder. Oracle files are in the Oracle\BusinessObjects folder.

For instances where Access does not support a feature or behaves differently than SQLExpress, the test fixtures have switch/case statements so the correct assertion is tested. For example, in some of the aggregate tests, Access treats nulls differently than SQLExpress and separate asserts were needed for the different RowCounts. Unsupported features (like aggregates in a GroupBy) are flagged ignored for Access only. They will show up yellow in the NUnit GUI.

A very good series of articles on unit testing can be found here.

NCover 1.5.3 with NCoverExplorer 1.3.1:

We are using these to backfill some of the areas that are not currently covered by our tests.

Tests.exe:

A little GUI we coded to try different EntitySpaces queries and to debug failing tests. There is a grid for the query results, a multi-line text box for displaying exceptions and/or Query.LastQuery, and RowCounts can be displayed in the status bar.

We invite you to help us make EntitySpaces the best it can be.

Today’s development world forces us to become a jack of all trades, master of none. We would much rather be EntitySpaces experts, not testing experts, but being the former requires at least a healthy dose of the latter. We are always looking for ways to improve our software. And one of those ways is to improve our testing. We realize that we still have a long way to go before we are satisfied with our tests. We welcome any comments or suggestions regarding the tests and our testing methods.

The EntitySpaces Team