9 Generating testthat test files

When your R code is offensive programming instrumented, it becomes possible to generate testthat unit test files, thus improving greatly developers productivity.

To be able to generate testthat unit test files, you must ensure that your R code is function return type instrumented and test cases instrumented. Both are required for this generation.

9.1 Package wyz.code.testthat in action

9.1.1 Setting up the context

To create testthat unit test files, you must provide a target folder, to store the generated unit test files. In this session, generated files will be stored onto folder generated-testthat. You may change this, but be warned that generated files might be overwritten without any reminder. Be careful, if you set the target folder to tests/testthat as you may loose previous work. You may save frequently your results in configuration management to be able to retrieve original file content whenever required.

Here, I reuse 4 files from package wyz.code.offensiveProgramming to generate unit test files from.

library(wyz.code.testthat)

target_folder <- 'generated-testthat'
if (!dir.exists(target_folder)) dir.create(target_folder)

generateTests <- function(sourceFile_s_1, sourcePackage_s_1, object_o_1) {
  g <- gautfo(object_o_1, sourceFile_s_1, sourcePackage_s_1, target_folder)
  print(g)
  g
}

source_package <- 'wyz.code.offensiveProgramming'

source_files <- c(
  'code-samples/both-defs/good/full/AdditionTCFIG1.R',
  'code-samples/no-defs/Addition.R',
  'code-samples/frt-defs/good/partial/AdditionFIPartial.R',
  'code-samples/tcd-defs/good/partial/AdditionTCPartial.R'
)

invisible(sapply(source_files, function(e) {
  source(system.file(e, package = source_package))
}))

9.1.2 Unit test file generation

Generation is done on a per class basis. Simply use function gautfo or function generateAllUnitTestsFromObject.

9.1.2.1 Nominal case

Class AdditionTCFIG1 is offensive programming instrumented. It uses both function return type instrumentation and test case instrumentation. Indeed, this class is partially instrumented. Not all the methods of the class have test cases declared for.

Generation is quite straightforward.

print(gautfo(AdditionTCFIG1(), source_files[1], source_package, target_folder))
#> $class
#> [1] "AdditionTCFIG1"
#> 
#> $filenames
#>                                                    filename overwritten
#> 1:       generated-testthat/test_AdditionTCFIG1-addDouble.R        TRUE
#> 2:      generated-testthat/test_AdditionTCFIG1-addInteger.R        TRUE
#> 3:    generated-testthat/test_AdditionTCFIG1-divideByZero.R        TRUE
#> 4: generated-testthat/test_AdditionTCFIG1-generateWarning.R        TRUE
#> 5:   generated-testthat/test_AdditionTCFIG1-generateError.R        TRUE
#> 6:  generated-testthat/test_AdditionTCFIG1-addMultiDouble.R        TRUE
#> 7: generated-testthat/test_AdditionTCFIG1-addMultiInteger.R        TRUE

Results is a list with two entries. Entry named filenames holds a data.table providing insight about created files.

9.1.2.2 Exception cases

Class Addition is NOT offensive programming instrumented.

Class AdditionFIPartial is NOT FULLY offensive programming instrumented. It lacks test cases instrumentation.

Class AdditionTCPartial is NOT offensive programming instrumented. It lacks function return type instrumentation.

Expected results is no unit test file generation.

print(gautfo(Addition(), source_files[2], source_package, target_folder))
#> [1] "Class [Addition] apparently owns no test instrumentation. No test created."

print(gautfo(AdditionFIPartial(), source_files[3], source_package, target_folder))
#> [1] "Class [AdditionFIPartial] apparently owns no test instrumentation. No test created."

print(gautfo(AdditionTCPartial(), source_files[4], source_package, target_folder))
#> [1] "Class [AdditionTCPartial] apparently owns no function return type instrumentation. No test created."

9.2 Generated unit test file content

Typically, generated R code will looks like following unit test code. Note, that comments are provided to ease cross-referencing and to link back easily to wyz.code.offensiveProgramming test case number. This is helpful when facing dysfunctions.

source(system.file("code-samples/both-defs/good/full/AdditionTCFIG1.R",
                   package = "wyz.code.offensiveProgramming"))
object_o_1 <- AdditionTCFIG1()
emsre <- EvaluationMode("standard_R_evaluation")
rtcsre24 <- runTestCase(object_o_1, 24, emsre)
rtcsre25 <- runTestCase(object_o_1, 25, emsre)
rtcsre26 <- runTestCase(object_o_1, 26, emsre)

test_that('addMultiDouble', {

# test 24 - sum of 1 integer and 1 double - correct
expect_true(rtcsre24$synthesis$status)
expect_true(rtcsre24$synthesis$value_check)


# test 25 - sum of 1 double, 2 integers and 1 NA_integer_ - correct
expect_true(rtcsre25$synthesis$status)
expect_true(rtcsre25$synthesis$value_check)


# test 26 - sum of nothing - correct
expect_true(rtcsre26$synthesis$status)
expect_true(rtcsre26$synthesis$value_check)

})

emtce <- EvaluationMode("type_checking_enforcement")
rtctce24 <- runTestCase(object_o_1, 24, emtce)
rtctce25 <- runTestCase(object_o_1, 25, emtce)
rtctce26 <- runTestCase(object_o_1, 26, emtce)

test_that('addMultiDouble', {

# test 24 - sum of 1 integer and 1 double - correct
expect_true(rtctce24$synthesis$status)
expect_true(rtctce24$synthesis$value_check)


# test 25 - sum of 1 double, 2 integers and 1 NA_integer_ - correct
expect_true(rtctce25$synthesis$status)
expect_true(rtctce25$synthesis$value_check)


# test 26 - sum of nothing - correct
expect_true(rtctce26$synthesis$status)
expect_true(rtctce26$synthesis$value_check)

})

9.3 Known-limits

Generation of unit test file uses meta-programmation based on call function, and aims to produce R valid code. Indeed, format and presentation are not managed, in generated file. Used RStudio editing facilities to ensure nice presentation, although neither mandatory nor required.

Generated tests cases are ready to run. Use the standard way to run your testthat test cases onto them. If you face some test failures, verify following points

1. make sure offensive programming evaluation is running fine for ALL evaluation schemes, and that you have no errors at this level
2. make sure your generated test case source is well up to date with the your R offensive programming code. You may regenerate your tests cases at any time if you have any doubt.

Note that unit test file generated is fully dependent of your R source and of the instrumented scope. If there are function not instrumented in your source code, do not expect to have unit test cases for them.

Also note, when working incrementally, you need to regenerate the unit test cases each time you change the R source code or the offensive programming instrumentation.

Best way to put wyz.code.testthat in practice, is to apply following procedure

1. apply offensive programming at the required scope and ensure wyz.code.offensiveProgramming test cases are valid, using runTestCase function
2. generate testthat test cases in one single pass using wyz.code.testthat. To do so, create an R script. This will ease your pain, and will provide consistent results through calls
3. apply testthat testing practice, to verify that generated tests are running fine.